IBM Z Software


This portal is to open public enhancement requests against IBM Z Software products. To view all of your ideas submitted to IBM, create and manage groups of Ideas, or create an idea explicitly set to be either visible by all (public) or visible only to you and IBM (private), use the IBM Unified Ideas Portal (https://ideas.ibm.com).


Shape the future of IBM!

We invite you to shape the future of IBM, including product roadmaps, by submitting ideas that matter to you the most. Here's how it works:

Search existing ideas

Start by searching and reviewing ideas and requests to enhance a product or service. Take a look at ideas others have posted, and add a comment, vote, or subscribe to updates on them if they matter to you. If you can't find what you are looking for,

Post your ideas

  1. Post an idea.

  2. Get feedback from the IBM team and other customers to refine your idea.

  3. Follow the idea through the IBM Ideas process.


Specific links you will want to bookmark for future use

Welcome to the IBM Ideas Portal (https://www.ibm.com/ideas) - Use this site to find out additional information and details about the IBM Ideas process and statuses.

IBM Unified Ideas Portal (https://ideas.ibm.com) - Use this site to view all of your ideas, create new ideas for any IBM product, or search for ideas across all of IBM.

ideasibm@us.ibm.com - Use this email to suggest enhancements to the Ideas process or request help from IBM for submitting your Ideas.

Add a new idea Subscribe
Status Not under consideration
Workspace z/OS Connect
Created by Guest
Created on Oct 21, 2020

RELATED IDEAS

Desire to propagate zCEE API name through to CICS request

When a CICS API request is made via zCEE, different APIs (and API names) have the potential to utilise the same transaction IDs in CICS, making it impossible to properly understand the workload being driven inside CICS in terms of the various zCEE APIs.

The desire here is simply to be able to associate the requesting zCEE API name, with the resultant CICS UOW.

CICS API requests from zCEE cannot easily be joined up with the resultant CICS UOW. I understand that with the latest versions of zCEE there is a unique tracking token that is present in zCEE SMF data, and that is propagated into CICS using OADATA1 field, which is materialised in CICS 110 SMF as well. So there is the possibility of joining the zCEE SMF with CICS SMF using the tracking token as the key, in order to connect the two together. This however, would require a huge amount of processing to do so, especially when our production CICS estate runs into many hundreds of millions of CICS txns per day.

It would therefore be highly desirable for the zCEE API name to be propagated into CICS in a similar manner as to how the tracking token is currently propagated, perhaps using one of the spare OADATA fields.

This would result in the zCEE API name being present in CICS 110 SMF records, and so appropriate association of API names with the resultant UOWs can be done directly using just the single CICS 110 SMF data source.


As a rough comparison, we already do such associations with CICS webservice names held within CICS 110 records, i.e. associating a webservice name with the resultant UOWs. This is extremely valuable in understanding workloads, and tracking changes to those workloads over time, especially when many transaction IDs are duplicated due to the 4 character limit.

We wish to be able to continue to do this in a similar fashion for zCEE-initated UOWs.

Idea priority Medium
  • Guest
    Reply
    |     Nov 27, 2020

    Thank you for your response.

    Whilst this requirement is valid, there are currently many tools and methods available to correlate end to end records for APIs workflows across nodes in near real time. This is a popular and strategic approach for API monitoring in the industry which ensures completeness and consistency of information that's not limited by the capabilities of any given node to receive or propagate large amounts of API request metadata. Specifically for SMF records, IBM Common Data Provider (CDP) provides a great way gather and summarise SMF records from multiple sub systems in near real time and present that data to analytics platforms like Splunk or ELK.

    As there is currently a solution to this need available, unfortunately, based on our current plans and priorities for IBM z/OS Connect Enterprise Edition, it is not likely that this could be implemented in the next 12 months. Correspondingly this requirement is being declined at this point. The requirement will be kept in the RFE system and might be reassessed in the future. You also have an opportunity to resubmit in twelve months time if you wish it to be reconsidered then.

  • Guest
    Reply
    |     Oct 29, 2020

    1) What monitoring tool do you use to track requests across z/OS?
    We do not have any tool that can track requests currently. What I am specifically referring being enhanced in this RFE, is to do with the post-processing & summarisation of SMF records, in order to provide meaningful insights into workload activity, that can be used to build an extensive & valuable historical record.

    2) Does it use exits/interceptors, SMF records, or both?
    SMF records

    3) What is preventing it from using a unique ID to correlate requests across subsystems?
    Associating a zCEE API name with a CICS UOW could potentially be done by match-merging records from both CICS & zCEE, but given the numbers of records will run into the hundreds of millions per day, this is not a particularly attractive prospect due to the massive amounts of resources & time that it will consume.

    4) By making the API name available in CICS SMF records what informed action would you be able to take that would be otherwise impossible?
    This will avoid the need to match-merge SMF records between CICS & zCEE. By having the zCEE API name already present in a CICS 110 field, will allow the CICS data to simply be summarised, so as to be able to associate a meaningful 'service name' to the resultant UOWs. This is already done today, to associate CICS web service names with their UOWs, which is extremely valuable to be able to properly understand the CICS workload, and how it is being driven from the originating sources. So would be a very simple matter to extend to cover the new field containing the zCEE API name.
    There would also be a side benefit in being able to see the zCEE API name in both 'live' and 'historical' CICS transactions in our CA-SYSVIEW monitor, which dynamically formats & records all SMF fields for all (recent) CICS transactions.

  • Guest
    Reply
    |     Oct 26, 2020

    Thank you for this RFE.

    There are currently a number of ways to monitor API workloads flowing through z/OS Connect EE covering a many different use cases including; Real Time Monitoring, End to End Business Request Tracking and Operational Analytics.
    Examples of these use cases and the types of monitoring tools that can be used are here: https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/yun-han-li1/2020/08/12/monitoring-api-workloads
    - Real Time Monitoring: https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/will-meyerink1/2020/08/11/monitoring-apis-with-omegamon-for-jvm
    - End to End Business Request Tracking: https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/yun-han-li1/2020/08/11/using-appdynamics-with-zos-connect-ee
    - Operational Analytics: https://community.ibm.com/community/user/ibmz-and-linuxone/blogs/yun-han-li1/2020/08/10/using-splunk-with-zos-connect-ee

    These examples provide the ability to associate an API request with a given Task, UOW etc in the backend system. As you rightly point out this achieved through the use of a unique correlation ID passed from z/OS Connect EE to the target subsystem (in the CICS case placed in OADATA1). This approach provides a subsystem agnostic way for these, and other, monitoring solutions to gain rich visibility at the request level as to the target of any given request, regardless if they are using z/OS Connect EE's interceptor framework directly or using the unique ID to correlate SMF records. Ultimately this enables the association of any and all data from a given API request (inc method, path, API name, service name, headers calling IP etc etc) with any down stream task or work.

    To help us better understand your use case, and what it requires beyond what is possible today, we'd appreciate if you could clarify the following points:
    1) What monitoring tool do you use to track requests across z/OS?
    2) Does it use exits/interceptors, SMF records, or both?
    3) What is preventing it from using a unique ID to correlate requests across subsystems?
    4) By making the API name available in CICS SMF records what informed action would you be able to take that would be otherwise impossible?

    Thanks again for raising this requirement and we look forward to your response to help us explore it further.

By clicking the "Post Comment" or "Submit Idea" button, you are agreeing to the IBM Ideas Portal Terms of Use.
Do not place IBM confidential, company confidential, or personal information into any field.