By John Alvord, IBM Corporation

jalvord@us.ibm.com

Inspiration

I recently had this problem report:

In the kdsmain.msg, I saw below error messages:

04/07/13 21:35:27 KDSPM021   Storage limit exceeded attempting to process situation _Z_KLOLOGPEVT17.
04/07/13 21:35:27 KO41039    Error in request _Z_KLOLOGPEVT17. Status= 1178. Reason= 50.
04/07/13 21:35:27 KO41039    Error in request _Z_KLOLOGPEVT17. Status= 1131. Reason= 1131.

How can I find out the real name for the situation _Z_KLOLOGPEVT17. I tried use tacmd viewsit:

tacmd viewsit -s _Z_KLOLOGPEVT17

KUICVS005E: The situation _Z_KLOLOGPEVT17 does not exist on server https://XXXX.com:3661.

The command did not complete because the specified situation was not found on the server.

Specify a valid situation name and try running the viewsit command again.

Overview

A situation whose name starts _Z_ is a duper situation. When you first see “duper” you will imagine it is an insider joke… and in some ways it is.

The z/OS platform was the second major target after AS/400 in 1997. The CICS product is widely used in z/OS and you often see hundreds or thousands of CICS processes in a single z/OS instance. Running multiple situations against all those processes was quite expensive – almost impractical. The idea came up to meld situations together. Instead of multiple situations running, you could run just a single merged situation and then when results were returned the TEMS would figure out which original situations the results applied to. The benefit was a sharply lower number of data collection operations. This made the process practical.

This is known as formally as Situation Synchronization and informally Super Duper situation processing. Situations that were merged together were called duperized situations. The process of merging was called duperization. The _Z_ named situation is called a duper situation. The form of the name is _Z_ concatenated with the attribute group table name concatenated with an integer.

Situation synchronization enables multiple situations to collectively perform a single data collection and all result sets are calculated from a single data collection. Situation synchronization enables more situations to be evaluated with a constant processing requirement.

Limits on Duperized situations

There are very significant limitations on duperized situations. Here is the list of requirements and exclusions:

Duperized Situation Eligibility Requirements:

1. Same attribute group required
2. Same attribute group required
3. Must be autostarted
4. Maximum of 10 expressions per situation
5. Can combine situations with different distribution lists

Duperized Situation Exclusions:

1. Display Item (atomize) [allowed in ITM 630 and ITM 623 FP2]
2. Take action [allowed in ITM 630 and ITM 623 FP2]
3. Used in a Policy
4. Embedded situations
5. Until clauses
6. *MISSING
7. *STR or *SCAN
8. Any group functions *COUNT/*MAX/*MIN/*SUM etc
9. JOIN between two single-row attribute groups
10. Some complex cases are excluded because of SITDB PREDICATE column limit
11. New/Updated situation requires TEMS recycle

Diagnosing Duper Issues

Duper situations are created when needed on each TEMS.

Many issues can be identified by adding this to the KBBENV file:

            CMS_DUPER=NO

When the TEMS is restarted it will run without Duperization logic. At that point most issues become a lot easier to understand.

I have several times figured out the duper situation components by studying the SITDB PREDICATE [a SQL representation of the situation logic]. From that I could determine the attribute group, the attribute names, the sampling interval and the tests performed. Then I studied the TSITDESC PDT [situation formula]. You can eliminate most when they don’t use the right sampling interval or attribute group. Then a manual scan can usually figure it out. That isn’t an easy/quick job and requires some special tools.

Happily, from ITM 630 and ITM 623 FP2 the following trace filter can clarify things:

error (unit:ko4lodge,Entry=”newSitRec::buildNameAndPredicate” all er)

On Linux /Unix add this to the TEMS .config file:

KBB_RAS1=’error (unit:ko4lodge,Entry=”newSitRec::buildNameAndPredicate” all er)’

Setting by service console or tacmd settrace is not useful because almost all the work happens at TEMS startup time.

Duperization logic is performed during TEMS startup. With the above trace filter you will see messages like this:

Created new sit <%s> by melding <%s>

That will clarify what situations compose the duper situation.

You may see multiple _Z_ with the same attribute table and different trailing numbers. That usually means there were situations that might have been melded but the sampling interval was different. Getting the intervals identical is critical to this process. On the other hand if you want to prevent situations from being duperized, setting a different sampling interval will accomplish that neatly.

Remote TEMS Duper Processing

If duper situations are running on a remote TEMS and the remote TEMS loses contact with the hub TEMS, all the duper situations are un-melded into the original situations. When the remote TEMS is connected again, the duper situations are recreated. The melding might change if the situation definitions change.

There are cases where a hub TEMS stop resulted in a severe remote TEMS CPU usage spike.

Summary

Duper situations are an amazing benefit for cases like z/OS CICS Agent, where there can be hundreds or thousands of CICS regions in a z/OS image. When just a few are combined or melded the benefit is not so large.

After ITM 630 and ITM 623 FP2 you will see more of these duper situations because the two exclusions were removed. On the other hand, the base exclusions are pretty severe and there won’t be that much change. The big difference will be for customer using Duper situations that now have more flexibility using DisplayItem and Action Commands.

Sitworld: Table of Contents

NOTE: Photo is hummingbirds visiting my front porch.

Nowadays, I use 2 48oz feeders and at times go through 10 pounds of sugar a week.