Allow symbol substitution in CICS reading the LE runtime options

Closing RFE with customer's consent.

Thank you. For Case TS002585030, we were instructed to build an LE Runtime Options module for the JVMServer to enable this Heap Check. That is where we found the undesired behavior that was investigated with Case TS002785451 referenced in the RFE. That investigation led to this RFE. With the RFE Team's suggestion of using a JVMProfile directive to enable the Heap Check instead of a LE Runtime Options module, the behavior has been as desired. The files are organized such that we are immediately able to identify which CICS region and which JVMServer generated the CEEDUMP file. As such, we would be ok with this RFE being closed and moving forward, using the JVMProfile method of enabling a Heap Check when necessary.

Sorry to come back again on this, I just want to be sure we fully understand the problems reported in the Case (and the subsequent RFE) before looking at potential solutions.

Despite the information written in Case TS002785451, our internal testing results show that we can successfully use the following JVM profile settings to create HEAP trace, and to direct the output based on symbolic substitutions in _CEE_DMPTARG. This has been independently verified by our service team, who cannot reproduce the reported problem (or can no longer reproduce the problem, but they recall thinking they had previously reproduced it).

_CEE_DMPTARG=./&APPLID;/&JVMSERVER;
_CEE_RUNOPTS=HEAPCHK(ON,1,0,10,10,40960,0)

If the INIT, MAIN, and TERM portions of the LE enclave do not propagate these environment variables as was suggested, then none of these values should ever take effect. However, our observations show that they do take effect, and in our testing - consistently. The testing was done at the CICS TS v5.5 and the current development release of CICS with a Liberty JVM server.

What this could suggest is that the problem is more complex than the Case diagnosis suggests. Perhaps there is another factor affecting the behaviour, I don't think it is fully understood what conditions could lead to the _CEE_DMPTARG value not being respected on a system.

Please would you be able to confirm that the exact settings above, coded in the JVM profile definitely do NOT take effect? The values should be written to the JVM profile before the JVMSERVER is enabled, and then the JVMSERVER must be fully disabled before any output is expected. If, with these settings it is confirmed that the CEEDUMP.* file(s) are not seen in the specified location, then I would suggest the Case/PMR should be reopened for further investigation prior to this RFE being progressed (I have engaged with the service team so they are aware).

I also note there was a suggestion of inconsistency of behaviour? Was the inconsistency between different JVMSERVERs? different CICS regions? or was it that the same JVMSERVER with the same settings might produce the CEEDUMP.* files at the right location during one enable cycle, but not the next? Any information that could help narrow down the suspected problem would be welcomed.

Again, sorry to extend the discussion, your patience is appreciated and will hopefully lead to a more satisfactory solution.

In Case TS002785451 I asked why "_CEE_DMPTARG=&CONFIGROOT;&APPLID;/&JVMSERVER;" was NOT always putting the CEEDUMP files in that directory and instead, some files are at the root work directory instead. "Setting _CEE_DMPTARG let us put MOST of the CEEDUMPs in the directory of the region's JVMServer. However, we still have a high number of CEEDUMPs produced in the work directory root level. Those make it very difficult to figure out what CICS region and JVMServer generated the file."

Mark Pocock @ CICS (Hursley) and Perry Buchalski @ LE discussed the specifics of how this is done and this resulted in finding "In this case, the setenv (for _CEE_DMPTARG) is being done 'too late' to affect the main/parent enclave. You noted that HEAPCHK is being set - can a test be done with _CEE_DMPTARG being set at this same time? Currently, setting _CEE_DMPTARG with the CALL_SUB will *not* update the main enclave -> and that is why it is *not* taking effect *from* the main enclave at termination" (Perry) and "Perhaps a CICS RFE should be raised to allow some sort of substitution of values within the runtime options file" (Mark).

This RFE is a result of Mark's comment in that Case since the current design does NOT control the output location in all cases since "_CEE_DMPTARG is only forwarded w/fork, exec, or spawn … as documented at https://www.ibm.com/support/knowledgecenter/SSLTBW_2.3.0/com.ibm.zos.v2r3.cbcpx01/cbc1p2349.htm … and _CEE_DMPTARG is being set 'too late' - and is thus not getting picked up by the main/parent enclave" (Perry).

In the end, if the design can be reworked so that _CEE_DMPTARG does indeed output all the files in the specified directory, that would be sufficient.

See previous comment. Is this sufficient ?

Using the generic LE runtime option 'ENVAR' to set LE environment variables is not required for JVM servers. If used, then symbol substitution is unsupported, the options must be recompiled for every single change, and there is a character length restriction which can easily be encountered. Instead, the JVM server is architected to use a JVM profile for setting LE environment variables in a more dynamic and flexible manner.

LE environment variables set in the JVM profile, such as _CEE_DMPTARG, influence the output locations of LE runtime options such as HEAPCHK. These values are capable of using substitution symbols. Typically one would use "&APPLID;" and "&JVMSERVER;" to ensure the locations are unique to a JVMSERVER instance. For example:

_CEE_DMPTARG=./&APPLID;/&JVMSERVER;

Since CICS TS v5.5 you can even define custom symbols in the JVM profile. The full set of symbols are described in topic 'Symbols used in the JVM profile' - http://releng-ilnx-u16.hursley.ibm.com:9055/kc/SSGMCP_5.5.0/configuring/java/dfha2_jvmprofile_symbols.html. The requirement for processing additional MVS symbols has been raised in RFE 129504.

CICS supplies a default pre-compiled LE runtime options (DFHAXRO) module, and the intention is that customers can also pre-compile a couple of variations of the module for test/dev purposes where debug options such as HEAPCHK, RPTO (report options), and RTPS (report storage) are ON. However, none of these values require location information or symbol substitution. Instead their output is controlled by the LE environment variables defined in the JVM profile - such as _CEE_DMPTARG and STDERR (for the LE reports).

Additionally, for dev/test purposes you may be able to use a currently undocumented, and unsupported feature to provide LE runtime option overrides directly in the JVM profile. For example:
_CEE_RUNOPTS=RPTOPTS(ON),RPTSTG(ON)
or
_CEE_RUNOPTS=HEAPCHK(ON

The former turns on LE reporting, the latter turns on HEAP checking - even if your JVM server uses the default DFHAXRO pre-compiled module!