Blogs

About this blog

I'm a well-known mainframe performance guy, with almost 30 years of experience helping customers manage systems. I also dabble in lots of other technology. I've sought to widen the Performance role, incorporating aspects of infrastructural architecture.
I'm a world-famous podcaster and screencaster (albeit VERY thinly spread). :-)

Find us on Facebook

While I was putting together the original three posts in this series a number of thoughts struck me, amongst which two really cried out for further investigation: I don't know how your XML data arrives on z/OS but quite a lot of scenarios don't have the data all as one document (file). XSLT looks complex - particularly if recursion does your head in. Thought 2 I'll deal with in a... [More]

Frank Yaeger of DFSORT Development suggested I pass this tip along to y'all. It's his solution to a problem set by Brian Peterson of UnitedHealth Group... In z/OS Release 12 two new messages were introduced: IEF032I and IEF033I replace IEF374I and IEF376I. The older messages were single-line step- and job-end messages. The new ones are their multiple-line analogues: IEF032I is 3 lines and... [More]

As you probably know Hardware Data Compression has been supported by MVS
and IBM mainframes for around 20 years. In several recent batch studies
I've conducted it's been evident in a widespread way. (In
this post I'm not talking about DB2 compression of either flavour or
VSAM compression - though some of the information here applies to these
functions as well.) It's not as
simple... [More]

Have you ever had the suspicion a sort was unnecessary in your batch? I bet you have. In recent Batch Performance studies I've had the suspicion that many of the sorts are unnecessary: Either they should be merges or not done at all. But how do you prove it? But first, what do I mean by a sort not needing to be done at all? Clearly if the data is reformatted then something has to be done to... [More]

Following on from this post and this one , this post discusses the DFSORT piece. The DFSORT code in this post parses the Comma-Separated Variable (CSV) file produced by XSLT processing. In this simple example it merely produces a flat file report, but the post has a few additional details you might find valuable. First, here's the SORTIN DD JCL statement. It's not like a regular sequential file... [More]

This is the second part of a (currently) three-part series on processing XML data with DFSORT, given a little help from standard XML processing tools. The first part - which you should read before reading on - is here . To recap, getting XML data into DFSORT is a two stage process: Flatten the XML data so that it consists of records with fields in sensible places. Process this flattened data with... [More]

In the distant past I've written about using DFSORT to parse XML. This post (and two follow-on posts) will describe
an experiment to make such processing much more robust. In this post I'll talk about what the problem I'm trying to solve is. And why. And a brief outline of my solution. About XML This isn't meant to be the most detailed description of XML, nor a complete list of where it's used. I... [More]

I concluded Batch Architecture - Part One with a brief mention of inter-relationships and data. I'd like to expand on that on this part. Often the inter-relationships between applications are data driven - which is why I'm linking the two in this post (and in my thinking). But let's think about the inter-relationships that matter. There are four levels: Between applications. Between jobs. Between... [More]

Thanks to Scott Drummond for reminding me of last Autumn's DFSORT Function PTFs - UK90025 and UK90026. They're mentioned in the preview for z/OS Release 13 so now is not such a bad time to be talking about them. So let me pick out a few highlights: Translation Between ASCII And EBCDIC, And To And From Hex and Binary For a long time DFSORT has been able to translate to upper case (TRAN=LTOU), to... [More]

First a word of thanks to Ferdy for his insightful comment on Batch Architecture, Part Zero . And also to my IBM colleague Torsten Michelmann for his offline note on the subject. As I indicated in Part Zero I hoped to talk about jobs in a subsequent post. And this is that post. In particular I want to discuss Viewing jobs as part of distinct applications, and Generating a high-level understanding... [More]

A new set of function was recently made available for DFSORT via PTFs UK51706 and UK51707. In this post I want to talk about the new JOINKEYS function, and try to add a little value by discussing some performance considerations. I've had the code for a couple of months and have played with it but not extensively. So much of what follows is based on thinking about the function (described in this... [More]

Over the years I’ve written emails with data collection requirements dozens of times, with varying degrees of clarity. It would be better, wouldn’t it, to write it once. I don’t think I can get out of the business of writing such emails entirely but here’s a goodly chunk of it. Another thing that struck me is that the value of some types of data has increased enormously... [More]

First, I’m indebted to Ray Mullins for his help in figuring out how to solve the problems I identified as remaining in Alternate Macro Libraries: A Way To Document Assembler Tables .
Indeed he wrote a very handy blog post of his own:
Wrapping your own macro code around vendor-supplied macros without changing source
This post, to summarise, shows how you can use the OPSYN instruction to create... [More]

When I first heard of Flash Express as part of the zEC12 announcement -
some time before announcement - I thought of one use case above all,
and one of particularly poignant resonance with some of my readers: Dump capture amelioration. Then, in the marketing materials, I heard of others.
And the discussions have grown more numerous recently.
So it’s time I expressed (pardon the pun) my... [More]

It's said that imitation is the sincerest form of flattery. In this Social Media age I'd say indirection comes a pretty close second. Indeed there's a nice term for it: "link love". Standard advice is not to just post links to the content of others. In reality the word "just" should probably be inserted: Don't just post links to the content of others. In that vein I want to... [More]

DFSORT's Arithmetic operators can do many things but the one thing they can't do is take the square root of a number. You might think that's minor but it means you can't calculate a Standard Deviation.
(Variance is fine but dimensionally not so nice when used in conjunction with the Mean.)
And I needed Standard Deviation in a real live customer situation. So I set out to "roll my... [More]

The CICS Monitor Trace SMF Record (Type 110) has got to be one of the most complicated SMF records in existence - and for good reason. Which is precisely why I'm not going to attempt to process the raw records in my code. (And why PMCICS doesn't support releases after a certain point.) But the "and for good reason" hints at the fact I think this is a tremendously valuable type of... [More]

Now with free map , this is the concluding part of a four part series on batch parallelisation, with especial focus on cloning. In previous parts I discussed: Motivation Classification Issues This part wraps up with thoughts on implementation. I'm going to break it down into: Analysis Making Changes Monitoring While there probably are iterations of this, this is the essential 1-2-3 sequence within... [More]

Maybe you've all heard of BPXWUNIX. Maybe it's just me that hasn't. Hence the question mark in the title. Seriously, I doubt this REXX function is as widely known about as it should be. And this post hopes to illustrate a little of its value. Suppose you're writing a REXX program with a stem variable in it. Suppose you want to sort the strings in the stem variable: You might write your own... [More]

I can't believe it's been almost a week since I wrote Batch Capacity Planning, Part 1 - CPU . Where did the time go? Re-reading it I'm struck by the overwhelming theme of Batch's unpredictability and lumpiness. This is true of memory, as well, but to a much lesser degree. Why to a lesser degree? Well, in most systems I look at the memory usage is mostly fairly constant and dominated by big... [More]

Following on from The Best Sort Is The One You Don't Do here's another reason for eliminating sorts. I think it's worth a post in its own right. (In this post, again, I'm talking about resequencing passes over data - not copying or merging.) With a sort it's possible the last record read in might be the first record written out. So you can never overlap input and output phases. (There might... [More]

To whoever (In the USA I think) Googled "what is REMOVECC" and got to this blog let me give you my thoughts on the matter. (And yes I know, it being a search, you might never come back to read this.) To understand REMOVECC you have to understand that the "CC" refers to ASA (American Standards Association) Carriage Control, described here . ASA (formerly ANSI) control characters appear in Position... [More]

For whoever got to my blog with this Google search here's how you sort data without a header: First you need to be using z/OS DFSORT Release 10 or to have applied the PTFs for UK90013 (July 2008). Second, the vehicle for doing this is ICETOOL rather than DFSORT itself. Use the new DATASORT operator. Here's an example: If you code DATASORT FROM(DD1) TO(DD) FIRST(3) USING(CTL1) and code a CTL1CNTL... [More]

So here's yet another way of parsing the WLM XML Service Definition. This time it's on z/OS, using DFSORT. Relatively recent features in DFSORT have made it easier to do useful things with XML. In this example I've made some attempt to make the output pretty - by creating a HTML table. Below is the SYSIN you need to parse the <SchedulingEnvironment> elements. I'm assuming you know how to... [More]

As I mentioned in this blog post DFSORT just shipped a new FINDREP function to do "find and replace". I mentioned an example of where I might use it to replace SMFIDs in SMF records. That example generally works well. But suppose (say, for SMF 42-6 Data Set Performance records) I want to replace "SYS1" with "MVSA" but don't want to replace "SYS1." with "MVSA.". There's something useful about... [More]

Yesterday DFSORT announced a new set of functions - as PTF UK90013. The documentation for it can be found here . Every year or so there's a new set of DFSORT functions - and generally they're "out of cycle" with z/OS releases - although they are incorporated into subsequent releases of z/OS. This means that fewer of you will know about the functions, particularly as we don't make a big fuss about... [More]

In September 1997 DFSORT Release 13 was shipped (to coincide with the release of OS/390 Release 4). It took a nice idea from Syncsort and extended it. In case you didn't know OUTFIL allows you to read an input data set (and perhaps sort it) and write to multiple output files from the resulting records - perhaps selecting subsets of the records and reformatting them (and differently to each output... [More]

!Somehow I seem to have end up writing a "Memories of..." series of blog posts. That wasn't the intention but a set of threads on IBM-MAIN Listserver got me to thinking about these nice venerable technologies - VIO, Hiperbatch, Batch LSR and Pipes. By couching these posts in terms of "memories of" it sounds like they're perhaps obsolete. With the possible exception of Hiperbatch that probably... [More]

(Also posted to MXG-L Listserver (MXG-L@PEACH.EASE.LSOFT.COM) which I highly recommend as a place where mainframe performance people hang out.) I must confess I feel slightly foolish about this... :-) After having asked for input into my UKCMG BOF one of the themes that came out was better ways of doing what IFASMFDP does. (I already have this on my agenda - as you'll know if you follow my blog.)... [More]

( Scroll down if you're not interested in DB2 ) This past week I've been updating our SQL Analysis code to cope with Unicode SQL statements. (If you bind a package or plan once you've gone to DB2 Version 8 New Function Mode the statement text is stored in the Catalog in Unicode.) These statements are in Unicode UTF-8 which is extremely similar to ASCII. So I snaffled some existing code of mine... [More]