Hidden Value

Search

September 22, 2017

Importing CSV Text Into COBOL II

I'm importing a Comma Separated Value (CSV) text file into a COBOL II program. I want to compare a numeric field from the file to a number. But the input text field can be different for each record. How do I code in COBOL to accommodate the different number sizes in the text file?

Walter Murray, who worked inside HP's Language Labs where COBOL II was developed before moving out into the user community, noted that Suprtool was likely the best solution to the problem. But after someone suggested that COBOL's UNSTRING statement could be useful, he had his doubts.

Along with suggesting that "importing the file into an Excel spreadsheet, and saving it in a more civilized format," Murray had these notes.

The UNSTRING statement will be problematic, because one of your fields may have one (or more?) commas in it, and you may have an empty field not surrounded by quotation marks. You might have to roll your own code to break the record into fields. If you are comfortable with reference modification in COBOL, your code will be a lot cleaner.

Once you do isolate the check amount in a data item by itself, you should be able to use FUNCTION NUMVAL-C to convert it. Yes, NUMVAL and NUMVAL-C are supported by COBOL II/iX, as long as you turn on the POST85 option.

Olav Kappert offered a long but consistent process.

First thing to do is to not use CVS; use tab-delimited. No problem with UNSTRING. Just use the length field and determine if the length = 0.

Do an UNSTRING of the fields delimited by the tab. Then strip out the quotes. Determine the length of each field and right-justify each field and zero-fill them with a leading zero. Then move the field to a numeric field.

You now have your values. Do this for each field from the unstring. You can create a loop and keep finding the ",". By the way, determine the record length and set the last byte+1 to "~" so that the unstring can determine the end of record. Long process, but consistent in method.

In addition to generating a CSV file with leading zeroes, Alan Yeo suggested using the X field.

Move the CSV value to a full size X field, then strip trailing spaces, and then move the result to an X redefines of your numeric. Please note, as your numeric is V99, you might also want to strip all "." and "," before the compare.

Get e-mail notice when the NewsWire blog gets a new entry.
Just say "Blog Me" in a message to editor@3000newswire.com.

September 15, 2017

Friday Fine-tune: Disk and memory checks

The utility cstm has the ability to show the configuration of your current memory installation: the makeup of 3000 memory in terms of boards used. What command delivers this information? First, enter the MAP command to see a map of the hardware on your system. Each item on the resulting list has a line number. Note the line number for “memory” and use it in the “select device” command, then enter the “info” command. For example, if the memory is device 64:

cstm>select device 64 cstm>info

If you enter the map command now, you will see the status of the memory will be “Information starting” or “information running”. When the status changes to “Information Successful,” you can display the result with the “il” (information log) command. Note: You can avoid the necessity of repeatedly looking at MAP to determine whether the info function has completed by entering “wait” at the prompt following the “info” command. You will not receive another prompt until the info process has completed. Another answer without using cstm is to run SYSINFO.PRVXL.TELESUP and at the prompt type MEMMAP. You should avoid this solution if using Mirror/iX, since it will break the mirror.]

What MPE command shows me much total hard disk space I have available to me, and how much of that is being used? Also is it possible to break that up per account? For instance, can it tell me how much hard drive space I would gain by purging a particular account? Use :DISCFREE C for checking disc space used and available by drive and in total. :REPORT z.@ will let you know how much your accounts are using. You may want to run :FSCHECK and do a SYNCACCOUNTING first.

September 08, 2017

Fine-tune Friday: Moving systems quickly

Here in the 14th year after HP stopped building 3000s, customers continue to use them. They use them up, too, and when that happens it's time to move a system from one machine to another. Here's some timeless advice from a net.digest column of the NewsWire on how to move quickly.

How do you move a large system from one machine to a completely new system, including disk drives, in the quickest way possible and minimizing downtime? In this particular case, it is a 7x24 shop and its online backup to a DLT4000 takes 16 hours.

Stan Sieler came up with an interesting approach to this particular problem, an approach that can be extended to solve a variety of problems in large 7x24 shops.

• Buy a Seagate external disk drive.

• Configure the Seagate on both the old system and the new system.

• Connect the Seagate on the old system.

• volutil/newset the Seagate to be a new volume set, “XFER” (REMEMBER: Volume set names can and should be short names!)

• Do one (or more) STORE-to-disks using compression with the target disk being the new Seagate drive.

• When the entire system is backed up onto the XFER disk, VSCLOSE it and unplug it (Caution: The safest approach is to power off your system first.)

• Attach the new disk to the new system (see caution above) and reboot.

• Set up the XFER group on the new system.

:newgroup xfer.sys

:altgroup xfer.sys; homevs=XFER

• restore the data

:file xferA; dev=99 (or whatever ldev XFER is)

:restore *xferA; /; olddate;create (if necessary)

Obviously, this leaves out interesting things like setting up UDCs, directory structure, etc. The point of this note is to introduce the concept of using a 36Gb disk drive as a transfer media.

Bijo Kappen and Patrick Santucci both pointed out that TurboStore’s store-to-disk module is smart enough to create another “reel” when the 4Gb file limit is reached. From the TurboStore/iX documentation:

If STORE fills up the first disk file specified for the backup, it creates as many additional disk files as needed, or uses existing disk files. They will be built with the same default file characteristics as the first disk file. The naming convention used for additional files is to append the reel number to the end of the first disk filename. The resulting name will be an HFS-syntax name. For example, if STORE needed three disk files to store all files, they would be named:

/SYS/MYBACKUP/STORDISC

/SYS/MYBACKUP/STORDISC.2

/SYS/MYBACKUP/STORDISC.3

John Lee reported doing the very thing Stan suggested:

“This does work. We do it all the time here when moving information between systems.

“Another variation we’ve found useful is using large, inexpensive, disks for archive purposes. Instead of purchasing often expensive archival devices such as CD or optical jukeboxes, just throw the information on some cheap hard disks inside a cheap enclosure and hang it off your system. Users then have access to all this information online. It might not be right for everybody, but in many cases it is."

September 01, 2017

Steps for a Final Shutdown

We're hearing a story about pulling the next-to-last application off an HP 3000 that's run a port facility. At some point, every HP 3000 has to be guided into dock for the last time. These are business critical systems with sensitive data—which requires a rigorous shutdown for sending a 3000 into a salvage yard.

While this is a sad time for the IT expert who's built a career on MPE expertise, doing a shutdown by the numbers is in keeping with the rest of the professional skill-set you can expect from a 3000 manager. I am reminded of the line from Citizen Kane. "Then, as it must for every man, death came to Charles Foster Kane." Nothing escapes death, but a proper burial seems in order for such a legendary system.

Chris Bartram, whose 3k Associates website offers a fine list of public domain MPE/iX software, has chronicled all the details of turning off an HP 3000. "I have performed last rites for a 9x8 server at a customer site," he says, "and have been through the exercise a couple times before."

There are 10 steps that Bartram does before switching off the 3000's power button for the last time.

Bartram reported that he first purged all accounts except sys, hpspool, and 3000devs (and had to log off all jobs, shut down the network, and disable system UDCs to do that). Then:

2) Reset/blanked all system passwords (groups, users, accounts)

3) Purged all groups from SYS account that I could (aside from in-use groups) as well as all users except MANAGER.SYS,OPERATOR.SYS, MANAGER.HPSPOOL.

4) Went through PUB.SYS listf (file by file) looking for anything that might be a job stream or contain user data (or anything not critical to keeping the system up) and PURGEd it

5) Went into VOLUTIL and condensed my discs

6) Created a group called JUNK.SYS (you would need to do this on each volset; this box only had the system vol set)

7) Wrote and ran a short script that copied NL.PUB.SYS (the largest file remaining on the system) into JUNK.SYS in a loop using filenames A####### and X####### until all disc space was used up

8) Typed the command PURGEGROUP JUNK.SYS

9) Went into NMMGR and changed IP addresses on the box to something bland/different; including the default gateway (also deleted any entries in the NS directory if there are any)

August 25, 2017

SSDs: Not a long-shot to work with MPE/iX

Hewlett-Packard Enterprise is not entirely sure if it's leaving the support forum it once devoted to the HP e3000. (After so many years, "e3000" is still the go-to keyword while looking at the HPE resources that remain online.) To test if the forum was still alive, or entirely in archive stasis, I posed a question. Could an IDE SSD drive get a hookup to a 3000 using an SCSI converter?

Is there enough bus connectivity in the e3000 to install an SSD on the server? There are SSD drives that can link up via IDE and I've found a SCSI to IDE converter.

One reply bounced back on the forum. "Torsten" said, "If this is really an HP3000 server, this sounds like the most crazy tuning idea."

Not so crazy, we've seen. In 2015, after one 3000-L newsgroup user compared putting SSDs in 3000s to a McLaren racing engine in an SUV, a more plausible solution emerged: using SSDs to support a virtualized 3000 running on an Intel-based PC. "You could house your 3000 in a Stromasys emulator running on a Linux box with VMware," said Gilles Schipper of GSA, "employing as many SATA SSD disks as you want on your host."

But there was a time in another May when SSDs running native in HP's 3000 hardware was a possibility worth investigating. It was also a necessity, because it was the only way.

It was almost six years ago, and the Charon emulator was still in development. Extending the future of the HP hardware still a necessity for a homesteading user. Stan Sieler of Allegro said he'd be looking into what would be needed to bring solid state storage to MPE.

"I'm thinking about SSD and SATA/SCSI adapters to speed up the 'obsolete' -- but still world's-best -- business computer, the HP 3000," Sieler said in May, 2009. "I'm hoping to do some tests in the near future."

Sieler said that those SATA/SCSI adapters would be a crucial part of putting SSD on its MPE feet. "Few SSD drives have SCSI interfaces... hence the SATA/SCSI adapter component," he said. "An SSD with a SCSI interface would look completely like an SCSI disk drive."

This kind of design, to mimic the SCSI interface, would've helped to avoid using the SCSI Pass-Through code HP engineered during 2007. The community still hasn't heard reports of how the pass-through works, and HP said that employing it is "not for the faint of heart."

A HP-hardware based 3000 would need to ensure that old hardware could be represented in its original form. An IT shop preserving MPE applications, instead of the platform -- not so much. A virtualized 3000 will do. The Stromasys Charon installations can be run from an SSD.

August 18, 2017

Fine-tune Friday: SCSI codes, and clean-ups for UDCs and 3000 power supplies

I need to clean up COMMAND.PUB.SYS on my 3000. There's a problem with BULDACCT. Is there a utility to help manage the UDC catalog?

Stan Sieler replies:

One option is "PURGE," which ships on all MPE systems :) Of course, that means you have to rebuild the UDC catalog. We recently encountered a site where, somehow, an HFS filename had gotten into COMMAND.PUB.SYS. You can't delete UDC entries with HFS filenames, nor can you add them. I had to edit the file with Debug to change the name into something that could be deleted.

Keven Miller adds:

I believe you want the utility UDCSORT from the CSL, the UDC sorting and reorganization program.

There are so many SCSI types. It's got to be the most confusing four letter acronym. Is there a guide?

Ultra160: same as LVD, but data signals double-clocked, i.e. transfers on both clock transitions like DDR DRAM. LVD and Ultra160 can co-exist on the same bus with SE devices, but will operate in SE mode. HVD doesn’t co-exist with anything else.

Upon arrival this morning my console had locked up. I re-started the unit, but the SCSI drives do not seem to be powering up. The green lights flash for a second after the power is applied, but that is it. The cooling fan does not turn either. The fan that is built into the supply was making noise last week. I can’t believe the amount of dust inside.

Tom Emerson responded:

This sounds very familiar. I’d say the power supply on the drive cabinet is either going or gone [does the fan ‘not spin’ due to being gunked up with dust and grease, or just ‘no power’?] I’m thinking that the power supply is detecting a problem and shutting down moments after powering up [hence why you see a ‘momentary flicker’].

Denys Beauchemin added:

The dust inside the power supply probably contributed to its early demise. It is a good idea to get a couple of cans of compressed air and clean out the fans and power supplies every once in a while. The electrical current is a magnet for dust bunnies and other such putrid creatures.

Tom Emerson reminisced:

Years ago at the first shop where I worked we had a Series III and a Series 48. Roughly every 3-6 months an HP technician would stop by our office to perform Preventative Maintenance. Amazingly, we had very few hardware problems with those old beasts. Once we didn't have a tech coming out to do PMs anymore, we had hardware related failures, including a choked-up power supply fan on a disk cabinet.

Finally, Wayne Boyer said"

Any modular power supply like these is relatively easy to service. It is good advice to stock up on spares for older equipment. Just because it’s available somewhere and not too expensive doesn’t mean that you can afford to be down while fussing around with getting a spare shipped in.

The compressed air cans work—but to really do a good job on blowing out computer equipment, you need to use an air compressor and strip the covers off of the equipment. We run our air compressor at 100 PSI. Note that you want to do the blasting outside! Otherwise you will get the dust all over where ever you are working. This is especially important with printers as you get paper dust, excess toner and so forth building up inside the equipment. I try and give our equipment a blow-out once a year or so. Good to do that whenever a system is powered down for some other reason.

Both are, well, difficult to use. (HP-UX also switched from sysdiag to stm.) Both have some modules that require passwords, and some that don't.

The offline diagnostics are on a bootable CD or tape. The lastest offline diagnostics CD (for PA-RISC) that I could find was labelled "2004."

That CD has seven diagnostics/utilities. I tried running all of them on an A-Class system. The "ODE" one is special; it's a program that itself hosts a number of diagnostics/utilities (some of which require passwords).

I'm not saying these diagnostics are "password-protected," because that implies they need protecting. "Password restricted" or "password deprived" might be a more accurate phrase. :)

ISL> tdiag...probably doesn't require a password (can't run on A-Class)

ISL> clkutilno password

ISL> edproc...probably doesn't require a password (can't run on A-Class)

ISL> edbc...probably doesn't require a password (can't run on A-Class)

ISL> xmap...probably doesn't require a password (can't run on A-Class)

ISL> multidiag****** MULTIDIAG ************ Version A.01.12 ******Enter password or a <CR> to exit:

All the rest from here on are ODE utilities/diagnostics. I ran each one, and document whether or not it requires a password. (A few utilities seem to have little or no use because HP hasn't provided a method of saying "Hey, my disk drive isn't an HP drive, and it's over here.")

ODE> wdiag****** WDIAG ************ Version A.01.53 ******Enter password or a <CR> to exit:

(from a friend:)

WDIAG is the PCXW ODE based diagnostic program. It is intended to test the Processor of the various PCXW based systems in the offline environment. The program consists of 150 sections, 1/150, and are organized into the following groups:

August 04, 2017

Friday Fine-Tune: HP 3000 DLT vs. DDS

Backing up enterprise-grade 3000s presents interesting choices. Back in the 1990s when the 3000 being built and sold by HP, DDS at first had only two generations, neither of which were reliable. A DDS tape used to be the common coin for OS updates and software upgrades. The media has advanced beyond a DDS-4 generation to DAT-360, but Digital Linear Tape (DLT) has a higher capacity and more reliability than DDS.

When a DDS tape backup runs slower than DLT, however, something is amiss. DLT is supposed to supply a native transfer rate of 15 MBps in the SureStore line of tape libraries. You can look at an HP PDF datasheet on the Ultrium SureStore devices certified by HP for MPE/iX at this link.

HP 3000 community partners such as Pivital Solutions offer these DLTs, At an estimated cost of about $1,300 or more per device, you'll expect them to beat the DDS-3 transfers of 5 MBps.

When Ray Shahan didn't see the speed he expected after moving to DLT and asked the 3000 newsgroup community what might be wrong. Advice ranged from TurboStore commands, to channels where the drives are installed, to the 3000's bandwidth and CPU power to deliver data to the DLT. Even the lifespan of the DLT tape can be a factor. HP's MPE/iX IO expert Jim Hawkins weighed in among the answers, while users and third-party support providers gave advice on how to get the speed which you pay extra for from DLT.

Dave Gale wrote in an answer that device configuration and CPU are potential problems:

If you are using a DLT it likes to get data in a timely manner. Otherwise it will do the old 'shoe shine'. This means that other devices on the line can affect the bandwidth on the channel and starve the DLT. If you are using something like RoadRunner, then the CPU can be a real factor in this equation (especially single-CPU machines). So, you may not only want to check the statistics portion of the report, but monitor your machine during backup with Glance or SOS.

Gilles Schipper of GSA said that a TurboStore command is essential. "If you're using HP TurboStore, are you using MAXTAPEBUF option on STORE command?" MAXTAPEBUF and INTER can make a major difference, cutting a backup to DLT from 7 hours to under 2 by adding the parms.

HP's Hawkins said channel configurations of backup devices are key to ensuring that DLT tops the DDS speed:

Generally this shouldn’t happen. It might happen if the DLT and disc are on the same channel while the DAT/DDS was on a separate one. Might also happen with large numbers of small files on semi-busy system as some DAT are better at start/stop than DLT. If you are running STORE the STATISTICS option can give a broad indication of throughput for A/B comparison.

July 28, 2017

Fine-Tune Friday: Moving Backup Files

Editor's note: Today's 3000 technique talk is written by Brian Edminster, an HP 3000 veteran who's been maintaining MPE/iX apps in a support role for many years. Edminster's experience goes back into the 1980s, making him a good match for some of the classic MPE/iX apps that are still serving companies. He's available for contract or long-term engagements through his company Applied Technologies.

By Brian Edminster

First of two parts

Once store-to-disk backups are regularly being processed, it’s highly desirable to move them offsite — for the same reasons that it’s desirable to rotate tape media to offsite storage. You want to protect against site-wide catastrophic failures. It could be something as simple as fire, flood, or a disgruntled employee, or as unusual as earthquake or act of war.

Regardless of the most pressing reason, it really is important to keep at least some of your backups offsite, so as to facilitate rebuilding / recovering from scratch, either at your own facility, or at a backup/recovery site.

The problem comes in that the MPE/iX file system is far more structured than Unix, Windows, or any other non-MPE/iX file system-based storage mechanisms. While transferring a file off MPE/iX is easy via FTP, sftp/scp, or rsync, retrieving it is problematic, at least if you wish the retrieved files and the original store-to-disk files to be identical (i.e., with the same file characteristics: filecode, recsize, blockfactor, type, and so forth).

What would be optimal is automatic preservation of these attributes, so that a file could be moved to any offsite storage that could communicate with the MPE/iX system. Posix on MPE/iX comes to the rescue.

For FTP transfers between late-model MPE/iX systems this retrieval is automatic, because the FTP client and server recognize themselves as MPE/iX systems. For retrieving files from other systems, HP has made that somewhat easier by making its FTP client able to specify ‘;REC= , CODE= , and ;DISC=’ on a ‘GET’:

If you do not specify the ‘buildparms’ for a file being retrieved, it will default to the file-type implied by the FTP transfer mode: ASCII (the default), binary, or byte-stream (often called ‘tenex’ on Unix systems). The respective defaults used are shown below:

What follows is an example of automatic preservation of these attributes, so that a file could be moved to any offsite storage that could communicate with the MPE/iX system. And this is yet again where Posix comes to the rescue, via the venerable ‘tar’ (Tape ARchiver), or ‘pax’ archiving utilities.

‘pax’ is a newer backup tool, designed to be able to read/write with tar format archives, newer ‘ustar’ format (that includes Extended Attributes of files). At the same time it has a more ‘normal/consistent’ command syntax (as Unix/Posix stuff goes, anyway), plus a number of other improvements. Think of it as tar’s younger (and supposedly more handsome) brother.

A little known feature of most ‘late-model’ tar and all pax commands is the ability for it to recognize and utilize Extended Attributes. These will vary with the target implementation platform, but for the tar and pax commands included with releases after v5.5 of MPE/iX this capability is not only present — but contrary to the man command’s output and HP’s Posix Command Line manual, it’s the default! You use the -A switch to turn it off, returning tar to a bytestream-only tool.

While not externally documented, via a little experimentation I’ve determined that the following series of Extended Attributes value-pairs are in the MPE/iX Posix implementation of a tar or pax ‘file header’ for each non-Posix file archived:

July 21, 2017

Friday Fine-Tune: Nike Arrays 101

Newswire Classicby John Burke

Many 3000 homesteaders have picked up used HP Nike Model 20 disk arrays. For many years there has been a glut of these inexpensive devices on the market and they work with older models of HP 3000s. However, there is a lot of misinformation floating around about how and when to use them. One company posted the following to 3000-L:

“We’re upgrading from a Model 10 to a Model 20 Nike array. We're deciding whether to keep it in hardware RAID configuration or to switch to MPE/iX mirroring, since you can now do it on the system volume set. We’re considering the performance issue of keeping Nike hardware RAID versus the safety of MPE Mirroring. How do you switch from one to the other? You can use the second Fast and Wide card on the array when using MPE mirroring, but you can’t when using Model 20 hardware RAID."

“So, with hardware RAID, you have to consider the single point of failure of the controller card. If we ‘split the bus’ on the array mechanism into two separate groups of drives, and then connect a separate controller to the other half of the bus, you can’t have the hardware mirrored drive on the other controller. It must be on the same path as the ‘master’ drive because MPE sees them as a single device.

"Using software mirroring, you can do this because both drives are independently configured in MPE. Software mirroring adds overhead to the CPU, but it’s a trade-off. We are evaluating the combination of efficiency, performance, fault tolerance and cost.”

First of all, Mirrored Disk/iX does not support mirroring of the System Volume Set – never did and never will. Secondly, you most certainly can use a second FWSCSI card with a Model 20 attached to an HP 3000.

Another poster elaborated on the second controller. All of the drives are accessible from either controller but of course via different addresses. Your installer should set the DEFAULT ownership of drives to each controller. To improve throughput each controller should share the load. Only one controller is necessary to address all of the drives, but where MPE falls short is not having a mechanism for auto failover of a failing controller.

In other words, sysgen reconfiguration would be necessary to run on a single controller after SP failure in a dual SP configuration. You could have alternate configurations stored on your system to cover both cases of a single failing controller but the best solution is to get it fixed when it breaks. The best news is that SP failures are not very common.”

There is a mechanism in MPE for ‘failover’ called HAFO: High Availability FailOver. Unfortunately it is only supported with XP and VA arrays and not on Nike arrays or AutoRAIDs.

“We have seven Nike SP20 arrays," said Andrew Popay, "totaling 140 discs spread across all the arrays, using a combination of RAID 1 (for performance) and RAID 5 (for capacity). We use both SP’s on all arrays, with six arrays used over three systems (two per system). One of our systems has two arrays “daisy-chained.” The only failures we have suffered on any of the arrays have been due to a disc mechanism failing.

"We never find any issues with the hardware raiding; in fact, as a lot of people have mentioned, hardware raiding is much more preferred to software raiding. Software raiding has several issues, system volume, performance, ease of use, etc. Hardware raiding is far more resilient.

“As for anyone concerned about single points of failure, I would not worry too much about the Nike arrays, I would say they are almost bullet proof. For those who require a 24x7 system and can’t afford any downtime what so ever, maybe they should consider upgrading to an N-Class, with a VA or XP. Bottom line: SP20’s are sound arrays on the HP 3000s, easy to configure, setup and maintain.”

July 07, 2017

Fine-tune Friday: opening disk, adding HASS

I need contiguous file space for my XM log file. How do I get this?

Many operations on the HP 3000 require contiguous disk space. Other files also require contiguous space; for example, consider the contiguous disk space on LDEV 1 required for an OS update. If you do not have one of the several third-party products that will create contiguous disk space on a drive, you may still be able to get enough free space by using CONTIGVOL.

However, occasionally, CONTIGVOL will stop with a message of “*Warning: Contigvol - Inverse Extent Table Full, Internal resource limit.” What can you do? Run it again. HP’s Goetz Neumann reported the message "is a warning that an internal table has filled up. It appears CONTIGVOL only handles looking at 40,000 extents at a time. You can run CONTIGVOL multiple times if the first run does not condense the free space enough because of this limitation.

I am adding two drives to a HASS (Jamaica) enclosure that already has several drives. How do I do this?

Gilles Schipper, Lars Appel and Chris Bartram reply:

First, a note of caution. If you dynamically add disk drives to, say, your MPEXL_SYSTEM_VOLUME_SET, you could find yourself in a pickle if you subsequently perform a START RECOVERY by accident or design. So while you can add drives dynamically as a convenience, it is a good idea to schedule a SHUTDOWN, START NORECOVERY as soon as possible to “fix” the new drives in your base configuration.

You do not even have to take down the system to add the drives to an HASS enclosure. The following steps will do the job.

• Set proper SCSI IDs. Make sure the SCSI addresses of the HASS enclosures are what you believe them to be. Do not make any assumptions. You need to set the SCSI address dip switches properly and ensure they are unique for the controller they are attached to. You will probably need a little flashlight to check the settings.

• Plug in the new drives.

• Use IOCONFIG to add the appropriate paths and device IDs. Note that the ldevs cannot be in use by, for example, vt or telnet sessions. So, you may still need to do this “off hours.”

• Use VOLUTIL to NEWVOL or NEWSET. For example,

:volutil>newvol mpexl_system_volume_set:member99 99 100 100

(This example is for LDEV 99 — the “99” in member99 does not need to correspond to the LDEV number, it only needs to be unique for that volume set.)

It might be a good idea to first run the drives in a NEWSET for a while, exercising them a little. You could also use that extra volume set to exercise seldom used VOLUTIL commands or NEWACCT options like ONVS/HOMEVS. Finally, SCRATCHVOL them and add them to the desired volume set.

June 30, 2017

Fine-Tune Friday: Jazz refuge, Query-JCL tip

Editor's Note: We're taking Monday July 3 off to celebrate Independence Day, and we'll be back on July 5 with a new report.

Where did the files HP hosted on jazz.external.hp.com go? So many articles reference that HP 3000 labs site.

The Jazz server contents were moved to several servers. A system at Speedware (a company now called Fresche Legacy) has much of what was hosted on Jazz. Those Jazz links at Speedware (now Fresche Legacy, and deeply absorbed with IBM iSeries work) are tucked away under hpmigrations.com. Not exactly the place where you'd look for homesteading tools, but available anyway.

How can I supply to QUERY a variable from within a JCL job? The physical paperwork for a file on the 3000 is being copied to digital format. We mark the files as deleted, (logically, not physically). Tracking this destroyed paperwork is done manually (a tedious and error prone process). How can I write something on the 3000 to create a comma delimited file of files shredded the day before?

John Long replies

This may help:

:SETVAR PREVDATE, !HPYYYYMMDD - 1:SHOWVAR PREVDATEPREVDATE = 20170607

I'm not sure how (or even if) query reads variables. (Nothing in the manual about it). Which is why you might have to 'build' your jobstream daily and replace the date in the 'USE file.'

June 23, 2017

Friday Fine-Tune: A 3000's Intrinsic Savvy

As the clock counts down to the 10-year deadline for calendar services changes, our thoughts turn to HPCALENDAR. That's the intrinsic HP wrote for the 6.0 and 7.x releases of the 3000 OS, a new tool to solve an old problem. Alas, HPCALENDAR is fresher than the bedrock CALENDAR, but it's only callable in the 3000's Native Mode.

But poking into the online resources for MPE Intrinsics, I learned that once more HP's re-shelved its 3000 docs. Things have gotten better: everything now lives on the much-better-focused HP Enterprise website. You can, for the moment, locate the guidelines to intrinsics for MPE/iX at hpe.com.

The Intrinsics Manual for MPE/iX 7.x is also a PDF file at Team NA Consulting. Independents like Neil Armstrong help the community that's using HP's resources for 3000s these days. It used to be much simpler. In the 1990s, the Interex user group ran a collection of well-written white papers by George Stachnik. We're lucky enough to have them with us today, cut loose from ownership and firewalls. One is devoted to the system's intrinsics.

By the time The HP 3000--for Complete Novices, Part 17: Using Intrinsics was posted on the 3K Associates website, Stachnik was working in technical training in HP's Network Server Division. He'd first written these papers for Interact, the technical journal devoted to 3000 savvy for more than two decades. Even though Interact is long out of print, Stachnik's savvy is preserved in multiple web outposts.

Stachnik explains why intrinsics tap the inherent advantage of using an HP 3000.

When an application program calls an MPE/iX intrinsic, the intrinsic places itself in MPE/iX's "privileged mode." The concept of privileged mode is one of the key reasons for the HP 3000's legendary reputation for reliability. Experienced IT managers have learned to be very wary of application programs that access system internal data structures directly. They demand that MPE/iX place restrictions on HP 3000 applications, to prevent them from doing anything that could foul up the system. This is what led to the development of the intrinsics. Application programs running in user mode can interact with the operating system only by invoking intrinsics.

Even if your company has a migration in mind, or doesn't have an unlimited lifespan for the 3000, knowing how intrinsics work is an intrinsic part of learning 3000 fine-tuning that might be inside classic applications. Tools can help to hunt down intrinsics, but it helps to know what they do and what they're called. You can fine-tune your 3000 knowledge using Stachnik's papers and HP's Intrinsic documentation.

June 16, 2017

Friday Fine-Tune: Cleaning Up Correctly

Classic 3000 AdviceBy John Burke

Good intentions about maintenance sometimes stumble in their implementation. As an example, here’s a request for help on cleaning up.

“We have a 989/650 system. Every weekend we identify about 70,000 files to delete off the system. I build a jobstream that basically executes a file that has about 70 thousand lines. Each line says ‘PURGE file.group.account’. This job has become a real hog. It launches at 6 AM on Sunday morning, but by 7 PM on Sunday night it has only purged about 20,000 files. While this job is running, logons take upwards of 30 seconds. What can I do?”

This reminds me of the old joke where the guy goes to the doctor and complains “Gee, doc, my arm hurts like hell when I move it like this. What can I do?” The doctor looks at him and says “Stop moving it like that.” But seriously, the user above is lucky the files are not all in the same group or he would be experiencing system failures like the poor user two years ago who was only trying to purge 40,000 files.

In either case, the advice is the same; purge the files in reverse alphabetic order. This will avoid a system failure if you already have too many files in a group or HFS directory, and it will dramatically improve system performance in all cases. However, several people on the 3000-L list have pointed out that if you find you need to purge 70,000 files per week, you should consider altering your procedures to use temporary files. Or if that will not work, purge the files as soon as you no longer need them rather than wait until it becomes a huge task.

If all the files are in one group and you want to purge only a subset of the files in the group, you have to purge the files in reverse alphabetical order to avoid the System Abort (probably SA2200). PURGEGROUP and PURGEACCT will be successful, but at the expense of having to recreate the accounting structure and restoring the files you want to keep. Note that if you log onto the group and then do PURGEGROUP you will not have to recreate the group.

Craig Fairchild, MPE/iX File System Architect explained what is going on. “Your system abort [or performance issues] stem from the fact that the system is trying desperately to make sure that all the changes to your directory are permanently recorded. To do this, MPE uses its Transaction Management (XM) facility on all directory operations.

“To make sure that the directories are not corrupted, XM takes a beginning image of the area of the directory being changed, and after the directory operation is complete, it takes an after image. In this way, should the system ever crash in the middle of a directory operation, XM can always recover the directory to a consistent state - either before or after the operation, but not in a corrupted in-between state.

“On MPE, directories are actually just special files with records for each other file or directory that is contained in them. They are stored in sorted alphabetical order, with the disk address of the file label for that file. Because we must keep this list of files in alphabetical order, if you add or delete a file, the remaining contents of the file need to be “shifted” to make room, or to compact the directory. So if you purge the first file alphabetically, XM must record the entire contents of the directory file as the before image, and the entire remaining file as the after image.

“So purging from the top of the directory causes us to log data equal to twice the size of the directory. Purging from the bottom of directory causes XM to log much less data, since most of the records stay in the same place and their contents don’t change. The system abort comes from the fact that more data is being logged to XM than it can reliably record. When its logs fill completely and it can no longer provide protection for the transactions that have been initiated, XM will crash the system to ensure data integrity.”

Goetz Neumann added, “PURGEGROUP (and PURGEACCT) do not cause a SA2200 risk, since they actually traverse the directory in reverse alphabetical order internally. This is useful to know for performance reasons. Since these commands cause much smaller XM transactions, it is faster to empty a group by logging into it and then PURGEGROUP it, instead of using PURGE @.

“There is a little-known tool to help prevent you from running into these situations in the first place: DIRLIMIT.MPEXL.TELESUP. A suggested (soft) limit for directory files would be 2MB. This would limit MPE to not have more than 50,000 files in one group, and (very much depending on the filenames) much less than 50,000 files per HFS directory. (These are XM protected just as well, and tens of thousands of files in an HFS directory is not a good idea from a performance standpoint, either.)

“Another way to reduce the risk of SA2200 in these situations would be to increase the size of the XM system log file (on the volume set that holds the group with the large number of files), which is available in a VOLUTIL command.

June 09, 2017

Friday Fine-Tune: Split up MPE/iX versions

Has anybody had success trying to split a single HP 3000 system into two different OS versions by using two sets of disks? There is no need for sharing of information between the two operating systems, and they would run independently of each other at different times of the day.

Guy Paul replies: This is certainly possible, as we have done this in the past for customers who couldn’t tolerate any downtime for OS upgrades. Hence, we came up with a solution to have a duplicate set of SYSVS discs that we upgraded while they were still on the old OS. Come day of the ‘real’ OS upgrade, we brought them down, stored off any modified files, switched over to the new OS, restored any modified files and they had an OS upgrade in about 45 minutes. So it is possible.

You should probably consider using BULDACCT to synchronize the accounting structure.

Gilles Schipper adds: This should be entirely possible. I do this sort of thing all the time. By simply booting from the appropriate boot path, you can do exactly as you wish. In fact, I have even shared common volume sets among different LDEV 1 system volume sets, with different MPE versions.

What's the name and syntax of the Posix utility that allows you to assign a new file, give the file the same name as the one you're replacing—and the new file will replace the old file when the last user closes it?

Andreas Schmidt replies:You want to use mv, a utility to rename and move files and directories. The syntax is

Where do I start to configure my HP Laser printer to be a network printer off the HP 3000. Don’t I need steps for I need steps for the NMMGR process?

Jeff Woods replies:Odd as it might seem, network printing isn’t configured using NMMGR but instead depends on SYSGEN devices with the ID HPTCPJD, plus a text file called NPCONFIG.PUB.SYS defining each printer LDEV.

(This might have more than one LDEV associated with each IP address, perhaps to give different environments such as landscape vs. portrait or duplex printing or special margins or line spacing as needed).

June 02, 2017

Sendmail fine-tunes, if you still need delivery

By Andreas Schmidt,with Mark Bixby, and Jens von Bülow

Relying on the HP 3000, you may want to use this box for incoming and outgoing e-mails as well. This is possible using a collection of software bundled with the HP 3000, or available in the public domain:

• Sendmail/iX, the mail transport agent well known on HP-UX that was ported to MPE/iX;

• Syslog/iX, the event logging subsystem required by Sendmail/iX;

• MAILX.HPBIN.SYS, the mail reader, and;

• Qpopper, the POP3 protocol for downloading.

Sendmail/iX and Syslog/iX have been ported to MPE. MAILX.HPBIN.SYS is part of the Posix shell part of each MPE/iX release since 4.5. The combination of all four of these utilities will enable your HP 3000 to receive Internet e-mails sent to you@host.domain and send Internet e-mails into intranets.

Syslog/iX

Syslog is the standard event logging subsystem for Unix. It consists of a server daemon, a client function library, and a client command line utility. It is possible to log to files, terminal devices logged on users, or forward to other syslog systems. Syslog can accept data from the local system via an AF_UNIX socket, or from any system on the network via an AIF_INET UDP socket on port 514. The sendmail mail transport package is one of the Internet tools which log to syslog. Syslog/iX is bundled with MPE/iX in the SYSLOG account. If somebody was a little too aggressive about cleaning up unused FOS files, you can restore the SYSLOG account from the backup of your OS. Otherwise, you can locate your FOS tape and manually extract and install the SYSLOG account.

Sendmail/iX

Sendmail is a mail transport that accepts fully formatted e-mail messages from local host system users, queues the messages, and then delivers the messages to local or remote users. It listens on TCP port 25 for incoming SMTP messages from remote systems, and delivers these messages to local host system users by appending the message text to the user’s mailbox file.

Sendmail is not a mail user agent. It does not have the ability to compose or to read e-mail. To cover this functionality, HP bundled the program /SYS/HPBIN/MAILX into the shell utilities. Sendmail is also not a POP3 server that will enable network clients to access Sendmail/iX mailboxes.

MAILX.HPBIN.SYS

This program helps read and send electronic mail messages. It has no built-in facilities for sending messages to other systems. But combined with other programs (a mail routing agent and a transport agent like Sendmail/iX) it can send messages to other systems. MAILX only offers limited support for various message headers (i.e. Subject:, From:, To:, Cc:, etc). If you need to do anything fancy, like MIME headers, you’ll need to call SENDMAIL.PUB.SENDMAIL directly and pass it a fully formatted message containing all headers and body text.

To read messages from your mailbox in /usr/mail/ type :MAILX.HPBIN.SYS

The files in /usr/mail/ are named USER.ACCOUNT and are accessible only for this user.

Qpopper

Qpopper is a server that supports the POP3 protocol for downloading Internet e-mail from software clients. Qpopper does not include a message transfer agent or SMTP support but normally works with standard Unix mail transfer agents such as sendmail. On MPE/iX it works therefore perfectly with Sendmail/iX.

The installation procedure basics are:

• The link /usr/local/bin/popper must point to /SYS/ARPA/POPPER.

• In SERVICES.NET.SYS, port 110/tcp must be reserved for pop3 service.

Having successfully installed this, you may now change your Internet browser so that your HP 3000 is the incoming POP3 server. You may do as we did: we created a new account POP3 with plain vanilla users per mailbox. The PC e-mail client needs to be configured in the following way:

Server Name: your POP3-enabled HP 3000

Server Type: POP3 Server, User Name: USER1.POP3 (e.g., SCHMIDA.POP3)

You may want to remember to set a password and an adequate check time for new e-mail. It’s up to you whether you want to download the new messages to the PC and not to keep on the host or not.

A nice feature is the aliasing in Sendmail/iX. Your HP 3000 acts as a POP3 and SMTP server for all Internet e-mail software agents.

Is there a proper way to shut down sendmail?

• Use the Posix kill signal from SERVER.SENDMAIL or any user with SM capability. (The following can be easily turned into a job.)

kill $(head -n 1 /etc/mail/sendmail.pid)

• Only use :ABORTJOB as a last resort! (This is true for all of the Posix things that got ported to MPE)

If you don't need to run a mail server (e.g. sendmail) on your 3000, you shouldn't. In most cases, using a mail client will be "just the ticket." Point the client at your in-house (SMTP) mail server and enjoy.

May 26, 2017

Friday Fine-Tune: Tape to Disk, Posix time fix

Editor's Note: Monday is the Memorial Day holiday here in the Newswire's office. We'll be back with a new article on May 31. With every day that passes, HP's original hardware gets older and more likely to fail. Virtualization of hardware brings newer hardware into service with 3000s and helps with this problem, a risk that's greatest on the 3000's moving components. Especially tape drives like the C1539 above; even HP's discs are less likely to fail. Enjoy this advice on how to put stored data from tape into a store-to-disk file, as well as keeping dates accurate in MPE/iX's Posix name space.

I have some information on a tape. How do I create a store to disc file with it?

There are a few solutions. The first and easiest is to simply restore the info to a system (RESTORE *T;/;SHOW;CREATE;ACCOUNT=WORKSTOR) where WORKSTOR is an account you create to pull the data in. Then a simple FILE D=REGSFILE;DEV=DISC and STORE /WORKSTOR/;*D;whatever else should create the disc store.

The second is to use FCOPY. The STORE format is FILE TAPEIN;DEV=TAPE;REC=8192,,U,BINARY.

John Pitman adds, "If you mean copy it off tape to disk store file, I’m not sure if that can be done. In my experience with tapes, there is a file mark between files, and EOT is signified by multiple file marks in a row. But it may be possible. If you do a file equate and FCOPY as shown below, you should be able to look at the raw data, and it should show separate files, after a file list at the front.

FILE TX;DEV=TAPE;REC=32767FCOPYFROM=*TX;TO=;CHAR;FILES=ALL

Here is our current store command, and the message it provokes. MAXTAPEBUF speeds it up somewhat

STORE !INSTOREX.NEW.STOCK2K;*DDS777;FILES=100000;DIRECTORY;MAXTAPEBUF

Why is the date/time in the Posix shell way off from the time on MPE, and what can be done to fix it? It’s over three weeks off.

Homesteading Editor Gilles Schipper replies:

First, check to ensure your timezone offset is correct and there are no pending time clock changes.

May 12, 2017

How to Step Through a CSLT Reinstall

Assume you've seen your Series 918 crash to its bones. You replace the 3000 and need to reinstall from CSLT. Many a 3000 site hasn't done this in a long time. This is when you're getting in touch with your independent support company for advice and walking the steps to recovery.

Oh. You don't have a support company to call. One more thing that's been dropped from the budget. You could ask on the 3000 newsgroup for help, so long as your downtime isn't serious. This support strategy is one way to go, and James Byrne got lucky this week. One expert walked him through the critical steps.

"I am working my way through the check-lists for reinstalling from a CSLT," Byrne said, "and I have come to the conclusion this stuff was written more to obscure than to illuminate." The HP documentation advised him to boot from disk, something he couldn't do. "Fortunately this is a backup 3000, and nothing too bad can happen yet."

Gilles Schipper, our esteemed homesteading contributor, provided the answers. The problem lay in a bad boot drive, but how do you discover that's true? We'll get to that in a bit. First, the CSLT reinstall.

Modify step 6 to read as follows:6. Use VOLUTIL to add additional discs to MPEXL_SYSTEM_VOLUME_SET, as well as any additional user volumesets that you have.The approprite VOLUTIL commands you will need are:1. NEWVOL command to add additional volumes to the existing MPEXL_SYSTEM_VOLUME_SET2. NEWSET command to add first (master) volume of user-volumeset1, say3. NEWVOL command to add additional volumes to user-volumeset14. repeat 2 and 3 for each additional user volumeset

Then, prior to step 7, add step 6A, which is the repeat of step 5, which is to AGAIN restore the directories from the tape containing the directories so that they are restored to the master volumes of their respective user volumesets.

And then step 7 remains as before.

The bad LDEV1 discovery? You can always swap out a known good drive. Byrne did this and noted he had to restock his cache of backup drives. "Only three left," he said. "Time to order more, perhaps." Advice on checking for bad drives before a boot:

"Instead of INSTALL, type ODE," Schipper said. "Then run Mapper. That should show you all of your devices including hard drives. Although that may even show drives that cannot be INSTALL'ed to. Also, check all of your SCSI terminators."

Stan Sieler added this to the remedies for a bad disk. "Try doing:

start norecovery message

"That 'message' option will cause 'start' to generate a **LOT** of output. Hopefully, the last few dozen lines might provide a clue to the problem."

Do all the backup that you need when protecting that HP 3000. The success of its backups falls in the lap of the system's managers. (Running CHECKSLT from the TELESUP account will verify if a CSLT is still good enough to boot your system. You'll want to check that CSLT to ensure it'll run on any tape drive, not just the one it usually runs on. Alignment issues kick DDS drives out of service regularly.) Don't forget about keeping your CSLT healthy.

April 28, 2017

Friday Fine-tune: directories and tombstones

A 3000 manager wanted to know about adjusting privileges on their server. When the community's veterans started to respond, extra information rose up. Some of was about the management of files in MPE/iX, the kind of legacy recorded on what's known as a tombstone.

Tombstones are data used to solve 3000 problems and establish file access. HP says in its manual for programming in MPE/iX that "It's frequently necessary to obtain status information on a file to determine the cause of an error." A File Information Display is frequently called a tombstone, providing:

Actual physical and operational file characteristics.

Current file information, pertaining to end of file, record pointer, and logical and physical transfer count. Information on the last error for the file and the last HPFOPEN or FOPEN error.

When a file is opened, the final characteristics may be different from those originally requested because of defaults, overrides, :FILE commands, and the file label.

You can use the PRINTFILEINFO intrinsic to print a tombstone. It requires that you specify the file number returned when the file is opened by HPFOPEN or FOPEN. The tombstone can display either a full or short format. If the file is open, it provides a full display. Otherwise, it provides a short display. Calling this intrinsic does not automatically abort the program.

You can call the PRINTFILEINFO intrinsic from programs written in COBOL II/XL and HP FORTRAN 77/iX. When calling from COBOL II/XL, use the FD filename. You can call the name PRINTFILEINFO directly from HP FORTRAN 77/iX programs. You can obtain the required file number by using the FNUM intrinsic.

Tombstones came up after one list member resurrected an answer about privileges from a 11-year-old post. Ray Shahan, still managing archival systems for Republic Title of Texas, heard his name in discussion about TD and RD privileges and how to control them. He quipped about not being heard from in ages.

"I have been asked by our security group to remove TD and RD privileges from our HP 3000," Reggie Monroe wrote this week. "These are for Reading and Traversing Directories. Does anyone know what the impact of this would be, if any?"

Tracy Johnson replied that "Unless your users have access to Posix files, you can categorically state you don't have any to remove."

There is an old comp.sys.hp.mpe posting where Ray Shahan wants to add TD and RD privileges. Just do the opposite, though that may be a bad thing if applied to MPE groups and accounts treated as directories.

The advice from the 2005 discussion included using Posix to enable "execute" permissions on all directories needed to get to the directory you want. So the opposite would be to disable those permissions. The ALTSEC command does this.The process will also include adding ACDs to the directory.

ACDs are ordered lists of pairs.The pairs are made up of access permissions and user specifications that control access to objects. Objects are passive entities that contain or receive information, such as files, directories, and devices. Each entry in the ACD specifies object access permissions granted to a specific user or group of users. In addition to being granted access to an object protected by an ACD, users can also be granted access to read the ACD itself.

ACDs can be applied to any MPE/iX files using the ALTSEC command. This command was enhanced to support directories. If a file has an ACD, this method of specifying access to the file takes precedence over other security features, such as lockwords and the file access matrix. ACDs cannot be placed on root, account, group, or directories.

April 10, 2017

3000 backup strategy for closing Sundays

Easter Sunday is on this week's horizon. While it's a rare day of closure at our local HEB grocery chain, Sundays are another sort of closure for 3000 managers. Nearly all of them want their partial backups of the weekdays to wrap up before the backup begins that will serve the work week. If you do full backups every night and want to make the new strategy to do partials during the week and a full on Sunday, there's a way to make that work. Donna Hofmeister, one of the former OpenMPE directors, explained the strategy in a message to 3000 managers.

First you need to decide what kind of partial you want to do. On Tuesday, do you want to backup all files changed since Sunday's full backup or do you want to backup all files changed since Monday's backup? (and so on....)

There are some things to think about here. If your "line in the sand" is always Sunday, then you have to deal with knowing that by Friday/Saturday your "partial" backup is likely going to be sizeable and will take longer to run. On the other hand, if you ever have to do a big restore, your restore plan is plan is pretty simple -- you'll need your last partial and your last full backup.

If your "line in the sand" is always yesterday, then your "partial" backups will be relatively small and quick. The flip-side is your big restore could be very complicated, since you'll need every partial backup through Monday plus your full backup. I think most people set the backup date as "Sunday" and do partials from there. But there's a technical bit that's also important.

The bit concerns the commandrestore ....;date>=!my_lastfull. Donna went on to explain.

"!my_lastfull" is a CI variable that contains a date in "mm/dd/[yy]yy" format. This is a value that your *full backup job* needs to establish. So your full backup job should do something like:

file lastfull,old echo !hpmonth/!hpday/!hpyear>*lastful

Your partial job will need to do something like this:

file lastfull,old input my_lastfull < *lastfull

There's plenty of logic that needs to be added to the above examples. (You'll note that there is NO error checking/what-if handling, etc.) This should be enough to get you started.

On incremental vs. differential backups — the former is a backup of changed files since the last full OR partial. The latter is a backup of changed files since the last full.

April 07, 2017

Friday Fine-Tune: Creating the perfect CSLT

A question about creating a CSLT for a Disaster Recovery test turned into a general discussion about what the perfect CSLT should look like. A system manager wanted to use the STORE option on the SYSGEN TAPE command to store additional files onto the CSLT he was creating for his DR test but was running into trouble trying to specify STORE options as part of the SYSGEN TAPE command. In particular, he wanted to simply add ;SHOW to get a listing of all files stored.

The answer to his original question is to use an indirect file, as in

sysgen>TAPE MODE=VERBOSE DEST=OFFLINE STORE=^CSLT.INDIRECT.SYS

where the indirect file contains whatever STORE directives you want in addition to the file list.

One contributor recommended robust efforts to get a listing: “A backup tape is of limited value without a listing. For Disaster Recovery purposes it is also a good idea to have the original HP tapes and patches with you as it is possible to create an SLT that does not install or work on a different HP 3000 system.” This same contributor also suggested creating a disk file with a listing of all the stored files.

However, several people questioned whether this list of files which the thread originator had proposed storing was sufficient. Stan Sieler probably said it best:

“I’d put much more in the STORE section, at the minimum:

/SYS - /SYS/MPEXL/DUMPAREA - /SYS/PUB/NL - /SYS/PUB/SL -/SYS/PUB/XL

(To explain, NL, SL, XL are dumped in the CSLT portion, so no need to dump them in the STORE section; DUMPAREA is a 32Mb file created at INSTALL time and there’s absolutely no need to dump it to a tape.)

/TELESUP - /TELESUP/DUMPS (or wherever you put your dumps.)

/ALLEGRO, /LPSTOOLS, /REGO, /ROBELLE, /VESOFT (It’s surprising how much you might want tools at an early stage.)

“If most of your user data is in one or two accounts, other than SYS, TELESUP, and the rest of the system might fit well onto one DDS/DLT, you might find it more useful to do:

/ - /USERS - /SALES

(where USERS, SALES are the ‘user’ accounts)

“Why? It guarantees you’ll get everything you’re likely to need in a recovery/install situation (except, of course, for the major portion of the user’s data). I’d also specify:

March 30, 2017

Puts, Gets, and Serving Up Transfers Faster

HP 3000s are exchanging files with other servers, a process that's included the FTP protocol for more than 15 years. This capability was once so magic that the arrival of Samba file exchange on MPE/iX was lauded as a breakthrough. FTP is quite a ways off the most current of transfer protocols. One manager's started a discussion about how to improve transfer speeds to and from the 3000, though. He's using DSCOPY as well, but prefers the PUTs and GETs of FTP.

The advice that's current about FTP/iX says that hard-coding the 3000's ports (100mb full duplex, or 10mb half) is one way to speed things up. Ensuring your traffic is not running through a proxy (called "being proxied) is another idea. Measuring the speed of a PUT against a GET is one step in discovering why the 3000's FTP might seem slow.

In 2008, MPE/iX gained a secure version of FTP—at least part of one. This SFTP functionality arrived at the end of the Hewlett-Packard lab era for 3000s, a period when new tools were not being placed into wide use. Sites were locking down their 3000 scope of operations to ensure stability. The port of this then-current functionality fell short of complete: only an FTP secured client got created. PUTs could be secured, as well as GETs. But only from Windows, Unix, or Linux hosts to the 3000. The 3000 wasn't going to dish out files using secured FTP. There are notes in place to carry the work forward, though.

MPE/iX tools and components are also out there to complete this securing of file transfers. OpenSSH is the best-known protocol. A quick-start bundle can be downloaded from the MPE-OpenSource website run by Applied Technologies. There are SFTP installation instructions at Applied, too. Someone who's got a need for securing FTP transfers will need to do the server side of the porting, which was completed on the client side by Ken Hirsh, Mark Bixby, and Mark Klein. Requests to speed up FTP are a sign this porting would be more than just an open source hobby project.

Brian Edminster, the senior consultant at Applied Technologies, explained that "with a bit of work, you could get OpenSSH v 3.7.1p2 working. The issue is that 'select' is busted under MPE/iX, and that's what's required for ssh to work correctly."

The fact remains: ssh cannot connect to a remote system and execute commands that produce any output. Ken Hirsch did the original port, but he only really needed the SFTP client -- so the issue with ssh wasn't addressed.

Ken also posted on the 3000-L newsgroup in 2008, asking if there was any interest in getting an ssh and sshd/sftp-server working (server daemon) -- so the 3000 could do port forwarding, act as a SFTP server, receive inbound ssh connections, and so on. Apparently he didn't get enough response to carry forward.

Back in 2005, Hirsch posted his goal.

I could get an interactive ssh client to work on MPE/iX. I don't know how, but I know it's possible! It would not be possible to get an ssh server working in such as way that an ssh client could run any program. But it would be possible to get enough of the server running so that you could use the server to do port forwarding.

In 2008, he added the note which Edminster referenced. "If anybody knows a way to actually write to a terminal while there is a read pending," Hirsch said, "I could use OpenSSH as a server on the HP 3000. Apparently there are undocumented MPE/iX sendio() and rendezvousio() calls, of which I know nothing. There are also tread()/twrite() routines in libbsd.a that I think are intended for this, but there's no documentation for these, either."

March 17, 2017

Friday Fine-Tune: Moving all disks at once

I want to take all my disks, system volume set, and the user volumes, and move them to another machine that has no disks. How can I best do this?

Lars Appel replies

You might use SYSGEN to create two different config groups in the SYS account, one for the old system (e.g. CONFOLD), and one for the new system (e.g. CONFNEW). To create the new config you might start with one of the existing config templates, e.g. CONF9x8.SYS for 3000/9x8 systems. Use BASEGROUP to open it, adjust IO to your needs and KEEP it as CONFNEW.

Just make sure that every disk gets an LDEV in the new config. Paths and LDEVs (except for LDEV 1) do not need to be the same on the new system. MPE/iX will notice (and "collect") all available volumes during bootup (as long as they are "visible" by a configured LDEV number).

Shutdown the system, plug disks into the new box, power it on, boot from the primary path (or enter the appropriate path for LDEV 1) and at the ISL prompt do a START NORECOVERY [NOSYSSTART] GROUP=CONFNEW.

You might also need to adjust NMCONFIG with NMMGR as a different system will probably have the LAN cards at a different physical path. In case you make a copy of NMCONFIG, make sure it stays on LDEV 1 (FILE;DEV=1).

We replaced LDEV1 on our HP 3000, did an INSTALL. Then we didRESTORE *T;/;DIRECTORY;SHOW;KEEP;OLDDATE The MPE-filespace was restored normally, but the HFS filespace was not. RESTORE told me to use CREATE=PATH which I then did. The DIRECTORY is on my STORE tape - so what went wrong?

Wolfgang Kinscher, and Gilles Schipper reply:

If you are planning an install keep this in mind: Do not specify the option "DIRECTORY" with a partial (DATE>=) backup. STORE will not put the HFS directory structure on tape. Do the following instead: STORE ;*T;DIRECTORY stores all of your MPE/HFS directories on tape. Then do your partial backup without DIRECTORY option

After the install, first restore your DIRECTORY with RESTORE *T;;DIRECTORY and then restore your partial and your latest full backup respectively.

March 10, 2017

Friday Fine-Tune: Going Beyond JBOD

By Gilles Schipper

One of the most cost-effective ways of advancing the reliability of your legacy system may be to replace your existing “JBOD” disk system with a much more reliable disk system. MOD20 units, still a better deal than individual disks, can provide a good starting point to implement RAID. JBOD is an acronym meaning “just a bunch of disks” — which would characterize the majority of HP 3000 systems as they were initially sold. JBOD disk systems comprise a set of independent — typically SCSI-connected — disks, which are each seen by the HP 3000 as a single logical device number or LDEV. Each disk LDEV is associated with a “volume set” and the failure of a single disk renders the “volume set” to which it belongs inoperable and un-accessible.

Traditionally, most 3000 systems have comprised a single volume set (specifically, the required SYSTEM volume set, with the brevity-challenged label “MPEXL_SYSTEM_VOLUME_SET”).

Systems comprising a large number of “JBOD” LDEVs increased the likelihood of system downtime, since the failure of a single, old disk effectively resulted in a “down” system — requiring a time-consuming disk replacement and system reload before the system could properly function once again.

To mitigate such delicate exposure to a single disk failure, many installations implemented the “User Volume Set” feature built in to MPE/iX, then constructed multiple volume sets so that the failure of a single disk affected only the volume set to which it belonged.

For practical purposes, the only real benefit to this approach was to reduce the amount of time required to replace the disk and reload only the data residing on the affected volume set. (In reality, it was usually quite unusual for a system to continue normal, or even minimal operation with even a single unavailable volume set).

To further improve system reliability and minimize downtime an optional, additional-cost software product was available in the form of software mirroring — aka “MPE/iX mirroring.”

This enabled the system administrator to configure non-system volume set disk drives to be associated with identical corresponding “mirror” disks. The software was responsible for dynamically duplicating the contents of both disk drive “mirrors” such the failure of one of the two mirror drives could be tolerated without affecting the continuous operation of the system. The damaged disk could then be replaced and the dynamic disk duplication would resume.

Only if both mirror pairs failed would there be a corresponding system outage and data loss. However, software mirroring was still far from ideal. Since it was unavailable for the MPEXL_SYSTEM_VOLUME_SET, the failure of a system disk, unprotected by mirroring software, would result in certain system down time.

Further, software mirroring exacted a price in terms of CPU and I/O overhead that could otherwise be utilized for actual “useful” processing.

And, as a wise person once said, given a choice, a feature is almost always better implemented in hardware than software. This certainly applies to disk mirroring and nicely aligns with the the Nike MOD20 RAID disk system, which is (one of the) HP 3000’s solutions to the compromises associated with software mirroring.

The MOD20 features dual controllers, duplicated (even triplicated) power supplies, and up to 20 disk drives housed in a single frame/enclosure that provides significant improvements over the MPE/iX software mirroring functionality.

Each MOD20 provides for a maximum of 8 logical units (LUN’s) to be configured — each of which appears as a single logical device no. (LDEV no.) to the HP 3000. A maximally and optimally configured MOD20 will include 20 disk drives and be configured as follows:

14 disks to be defined as type RAID1, using up 7 LUNS—since each LUN comprises two separate mirrored disks. RAID level 1 is equivalent to simple mirroring whereby one disk is dynamically maintained as a duplicated mirror image on its mirrored twin disk, which must of identical size and model.

If one disk of the mirrored pair fails, the other disk can take over the responsibility of presenting the requisite data IO to and from the host system with no perceived performance degradation. The remaining 6 disks can be configured as a single LUN comprising 4 RAID 1/0 disks and 2 hot spares.

A RAID 1/0 configuration takes an even number of disks and duplicates the contents of half of them (as a group) onto the other half.

The hot spares would act as dynamic replacements for any disk in the MOD20 that fails, such that even the failure of one or two disks would not prevent the entire disk subsystem from maintaining its fail-safe mirroring capability. Without the hot-spare feature, failure of a single disk would allow normal system activity to continue but without further fail-safe capability for the failing LUN only.

Chances of both disks in the same LUN failing are extremely remote. That is why I advise you to forgo the hot spare capability. Utilize a 6-disk RAID 1/0 LUN instead of a 4-disk RAID 1/0 LUN, giving you additional usable disk space overall.

March 03, 2017

When and how to back up 3000 directories

Editor's Note: Homesteading Editor Gilles Schipper weighs in on using the 3000's store directory option, rather than invoking the buldacct program, to make clean backups.

By Gilles SchipperHomesteading Editor

It's common to see confusion surrounding the use of the ;directory store option versus the buldacct directory creation program. In order to benefit from the store ;directory option, one has to utilize the option almost perfectly in both the store and the restore following a system INSTALL. Consequently, it becomes much easier to fall back on the buldjob options to re-create the directory -- although that option is inferior.

In order to be able to effectively utilize the directory option, the first thing that must be done properly is to ensure that the appropriate ;onvs= option is also used in the case where user volumesets are utilized. Otherwise, the non-system volumeset directories do not get restored after the INSTALL since they are not on the tape.

But even if the store part is done correctly, the other opportunity to go wrong presents itself during the reload process.

Of course, for those that do not utilize user volumesets, the directory option becomes much less error-prone. And, for those that utilize third-party backup utilities, the ;directory option -- as utilized in the MPE store command -- is generally replaced with a similar option in the various backup utilities.

The bottom line: for those that utilize the MPE store command to perform their backups, the properly-used ;DIRECTORY (and, if appropriate) corresponding ;ONVS= options) -- together with correct restore procedures as indicated above -- will get the desired result 100 percent of the time. Notwithstanding, of course, any tape issues occurring, which would be problematic no matter which directory re-creation option is used.

The bottom line is that the proper way to perform a full backup if you're using the MPE backup facility:

Of course, upon further reflection, and better than the store command: use sysgen to create a backup tape that not only contains all files, but also the SLT -- so that this one tape alone can be used to INSTALL and RELOAD your system. The use of sysgen for such purpose will require use of an indirect file.

February 24, 2017

Friday Fine-Tune: Opening Up MPE's Shell

Way back in the middle 1990s HP added the Posix shell to the HP 3000. The improvement meant customers who had Unix and MPE running in the same shop could train operators and managers with a single set of commands. Posix was a plus, making the 3000 appear more Unix-like (which seemed important at the time).

Over the years, however, Posix has been a feature waiting be discovered for most 3000 managers and operators. The computer's operating system was renamed from MPE/XL to MPE/iX just for this added Posix feature. But enough history; Posix is still on the 3000 and remains a powerful interface tool, an alternative to the CI interface that HP created for the system. You can even call Posix commands from the CI, a nifty piece of engineering when it can be done.

That's not always possible, though. A customer wanted to know how to "expand wildcard shells" using Posix. He tried from the CI and had this story to relate.

:LL /BACKUPS/HARTLYNE/S* ls: File or directory is not found

So how do I do this? I need to be able to tell tar to archive all of the reels of a STD STORE set via a regexp. It does not work in tar, and it apparently does not in ls—so I speculate that there is something special about the innovation of Posix utilities from the CI that I am not aware of. What is it?

Jeff Vance, the 3000 CI guru at while at HP, replied "Wildcards on most (all) Unix systems, including Posix implementations, are done by the shell, not the individual programs or in-lined shell commands, like ls in your example. A solution is to run the shell and execute ll from within.

The magic Posix shell command to do the expansion:

An interesting footnote if you've read this far: The Posix shell for the 3000 is one part of the operating system that was not built by HP. The shell was licensed by HP from MKS, and Hewlett-Packard paid royalties to MKS so Posix could work inside of MPE/iX. That was an issue that posed a potential snag for source code licensing from HP. But the outside license issues never ended up blocking emulation or source-license arrangements. Managers have used Posix on the 3000 as a way to get familiar with commands in Unix systems. In the great majority of instances, these commands are the same.

February 03, 2017

Fine-tune Friday: Care and feeding of UDCs

Mercury Insurance is a long-time HP 3000 shop still running a server in production. Last week Reggie Monroe reached out for a refresher on administration of HP 3000 User Defined Commands (UDCs). These are the HP3000's equivalent of scripting in Unix environments. UDCs are a better version of Command Files, according to Jon Diercks and his MPE/iX System Administration Handbook. UDCs are catalogued, Diercks says, so they can be loaded for individual user accounts.

Click for details

There's a superior PowerPoint slide deck online at the 3K Associates website that covers how to create and use UDCs. But the Diercks book (no longer in print, but available online) is more concise on the use of UDCs. It's also only available as an $80 book today on the used market; put yours in a safe place. Monroe's question asked about "a command to list all users, and the logon UDC associated with them, if one is set."

The initial answer was the command HELP SHOWCATALOG,ALL. This brings an administrator to

SHOWCATALOG [listfile][;USER=username[.accountname]]

But Alan Yeo pointed out that the MPE/iX command only locates system-level UDCs.

You don't actually get what you think you asked for, so whilst :showcatalog ;user=@.@ sounds very hopeful, in fact it only shows the system level UDCs not account ones. As far as I'm aware the only place you can find them all is in the BULDJOB2 file in PUB.SYS. You do have a BULDJOB2 file don't you? And it's up to date?

And here's where Vesoft's utility does a job the 3000's OS cannot. VEAUDIT LISTUDC @.@ finds UDCs of all kinds.

We have chronicled much of MPEX during the 21 years of the NewsWire's publication. The utility was even the sole subject of the Inside Vesoft column back in the era when HP was starting to lock down 3000 futures. In 2002 Steve Hammond illustrated the distinction of UDC administration under VEAUDIT. It becomes important because security on a 3000 includes management of the UDC catalogs. And yes, there's a tool for the security, too.

VEsoft’s Vladimir Volokh told me he had been asked to find out if any users on a system had the VEsoft utility GOD in one of their UDCs and if it had the lockword embedded in the UDC. He gave me a series of two commands that did the trick and they had some added value to boot. Once I saw the commands, I was impressed with the simple elegance, but like a good programmer, I had to deconstruct it, break it down and reassemble the whole thing. If you’d like to play along, you need: MPEX, VEAUDIT (both available from, who else, VEsoft) and a healthy programmer’s curiosity (you’re going to have to provide that yourself).

The details of the exercise show "GOD.PUB.VESOFT’ [is found using VEAUDIT] and we have accomplished our mission. But wait. What are those CIERR907 files? Those are files in the list that don’t exist! But they are UDCs that have been set! Looks like you can do some housecleaning and those UDCs can be un-set. How about that — you got some value added, you killed two birds with one stone, (insert your favorite cliche here). Time to play system manager again."

When you add a third party tool to your administrator's box, you can make a purge of such files foolproof. MPE/iX cannot select to show a complete set files by attributes such as program capability. Or for that matter, by last accessed time, or file size, or file security. It's a long list of things that MPE makes an administrator do on their own. Missing something might be the path to looking foolish.

VEAudit and MPEX will root out UDCs and do a foolproof purge, including file names. VEAudit will list all of the UDCs on a server, regardless of user -- not just the ones associated with the user who's logged in and looking for UDCs. The list VEAudit creates can be inverted so the filename is the first item on each line. Then MPEX will go to work to do a PURGE. Not MPE's, but a user-defined purge that looks for attributes, then warns you about which ones you want to delete, or would rather not.

January 27, 2017

Friday Fine-Tune: Memory and disk behavior

Along with the relationship between your CPU measurements and overall performance, memory and disk make up the other two components of your HP 3000 performance picture. Main memory is the scratch pad for all the work that the CPU performs. Every item of data that the CPU needs to perform calculations on or updating to must be brought into main memory.

The CPU must manage memory. It must cycle through the memory pages, marking some as Overlay Candidates (this means that new data from disk may be placed here), noting that some are in continued use, and swapping others out to virtual or what is called transient storage. Swapping to disk occurs when data is in continued use but a higher priority process needs room for its data.

To accommodate this higher priority process and its need for memory space, the Memory Manager will swap the memory for the lower priority process out to disk. The more activity the Memory Manager performs, the more CPU it takes to do this. Therefore it is the percentage of CPU used to manage memory that we use as a measurement.

Page Faults per Second

A Page Fault occurs each time a memory object is not found in memory. The threshold for the number of Page Faults per second that can be incurred before a memory problem is indicated varies with the size and the power of the CPU. Larger machines can handle more Page Faults per second while a smaller box will encounter problems with far fewer. We have found that the number of Page Faults per second a system can endure without problem rises with the relative performance rating of the machine.

An exceptional number of Page Faults should never be used as the sole indicator of memory problems but when observed should be tested with the memory manager percentage. If both agree, you have a memory shortage. There are some strange things that I have observed with Page Faults, so it does not stand alone as an indicator of memory shortage.

The number of Page Faults per second and the amount of CPU needed to manage Memory are always evaluated in conjunction with each other. That is to say the high Page Fault Rate will not be considered a problem if the Memory Manager Percentage is not above 4 percent.

Disk Environment

The Disk Environment is referred to as Secondary Storage. This is where all the data needed for system use is stored. Since Main Memory is not large enough to store all of the data that will be needed by all the processes, there must be a location for this larger pool of data. Even though the Disk Environment does not have the significance it once had, this area can still be a bottleneck. As the CPU speeds increase, bottlenecks become more significant.

Several different factors can affect the Disk Environment. One of these is data locality. Data locality pertains to two different types. There is data locality within Image datasets and data locality across the disk itself.

Data locality across disk: This refers to the location of separate pieces of files (called extents). When files are placed on the disk, they can be placed in contiguous sectors or sections of files, or they can be placed in non-contiguous locations or even on many different disks. When files are not in contiguous locations they are said to be fragmented. The advantage of contiguous location is that greater efficiencies are allowed in retrieving data. When files need to be read, the head movement of the disk drive is minimal if files are in contiguous locations. The head moves to the location and the retrieval begins.

As the disk fills up the system cannot find one contiguous location to build any new file. Therefore, the system breaks the file up into extents and places the file wherever it can. A system reload will put files back into contiguous location (usually back on the location of the files file label) or products such as Lund Performance Solutions De-Frag/X can be used to put the files back into contiguous location.

Operating systems allocate disk space in chunks as they create and expand files and transient disk space (swap areas, etc.). When files are purged, these chunks are released for reuse. Over time the disc space may end up fragmented into many small pieces, which can slow the performance and the reliability of the system.

To observe and correct MPE fragmentation, you can use the De-Frag/X product from Lund Performance Software or use the Contigvol command of MPE/iX's Volutil program. The latter creates contiguous free disk space on a volume. Contigvol work about as well as VINIT CONDense did -- that is, it's stable and reliable, but requires multiple passes to get the best results.

Data locality within IMAGE datasets is the other area of major concern. There there are two different types of datasets to be concerned with, detail datasets and automatic or master sets.

The Detail Datasets

This type of set holds the day to day data input. Detail sets begin with nothing in them. When records are added 1 is added to something called the high-water-mark, a number that tells how many records have been in the set, and the record is placed in the set.

The problem is that IMAGE automatically reuses space that is given up when a record is deleted. This space is often called the delete chain. New records are placed in the most recent location available on the "delete chain." This means that new records are not in the same physical locality as the rest of the records and may be far removed from the other records.

The ideal state for a detail database is one where the detail entries are sorted by the key field. This allows the data to be retrieved in the smallest amount of IOs making efficient use of the MPE systems pre-fetching of data. When this is not the case we can measure the dataset lack of efficiency with something called the Elongation factor. This is simply a measure of how many more IOs the user must perform to retrieve desired data.

The Master Datasets

These have unique identifiers (field names). There are two types of master sets, a manual master and an automatic master set. Manual masters have user-entered master entries while automatic masters have automatic entries placed in them to accommodate access to detail records. The issue of importance to performance here is something called the hashing algorithm. This is the method used by the database to calculate the location of the next record placed in the database. The intent is to cause the master set to be as equally distributed as possible.

The hashing algorithm uses the size of the set in its calculation. A poor size or a size that is not large enough will result in an unequally distributed database. A poor size is most easily described as one that does not consist of a prime number. This means that when the hashing algorithm calculates a location there is a higher potential that a record will already exist in that location. When this happens a secondary position must be calculated. When secondaries are placed in another block within the database, another IO must occur to retrieve needed data. Since IO to disk is the slowest type of access, we want to avoid this at all costs.

January 06, 2017

Friday Fine-Tune: Logging, IP logins, SNMP

Due to a disk crash, I had to reload my HP 3000 system recently. I’ve just discovered that system logging has been suspended. How do I resume system logging?

Paul Christidis replies:

The reason for the suspension of logging was most likely due to a duplicate log file name. When the SLT was created the then-current log number was recorded, and when you restarted the system from your most recent SLT it tried to open the sequentially next log file. Said file already existed.

MOVE the existing log files to a hold area

Determine what logfile the system resumed on

Perform a series of SWITCHLOG commands until the logfile number advances to one more than the highest number in the hold area

Then move the held logfiles back to the pub.sys group — replacing the ones created by the series of ‘switchlog’ commands.

Is there a way to see the IP address associated with a particular login?

The command :listf ci.pub.sys,8 will list all sessions and will show their associated IP address.

I’ve got an older model HP 3000 and I'd like to start monitoring it with SNMP for things like CPU utilized, jobs running or whatever other cool stat I can SNMP-grab. The problem I have is I can’t find the MIBs for it anywhere.

Andreas Schmidt replies:

First of all, I do not recommend the use of SNMP on the 3000, for performance but also security reasons. SNMP is not the securest protocol, as you know. Nevertheless, here are some hints:

• In the group NET.SYS you will find the SNMPUDC. This should be set in any case for MANAGER.SYS or on system level.• Having set this, a SNMPCONTROL STATUS will show you the status of the SNMP subsystem.• SNMPCONTROL START / STOP are self-explaining.• The MIBs specific for MPE can be found in the document HP SNMP/XL User’s Guide

December 16, 2016

Friday Fine-Tune: Login recovery strategies

We have changed passwords on MANAGER.SYS—and now we cannot locate the changed passwords, thanks to some staff reductions that made the new passwords unavailable. Any ideas on how to recover them?

John Stevens says

If you're logged on as OPERATOR.SYS, do LISTACCT on all the accounts that may have SM capability, then logon as the MGR/MANAGER of those accounts: TELESUP, and SUPPORT, and there are others; LISTACCT will find them. Login as those users (unless you don't have those passwords either) and LISTUSER MANAGER.SYS. Vesoft's MPEX might help ease some of this as well.

Using your favorite editor or other utility search for the string: "ALTUSER MANAGER SYS"You will notice: PAS=<the pwd> which is your clue.

Steve Ritenour suggests that a logon to the TELSUP account will unlock the passwords.

Some 3000 managers believe the subject itself should be filed in a place not easily found. "These responses are all well and good," said Bruce Collins of Softvoyage, "but shouldn't we be thinking twice about posting this kind of information (i.e. how to hack an HP 3000) to the 3000 newsgroup?"

Bill Lancaster disagreed. Secrecy about password recovery is not really a secret, he said.

A Google search with the right words will yield far more dangerous information about the 3000 that anything in this thread. The genesis of this information being on public networks came through BBS’s in the 1980s. I’m afraid the barn door is already open.

Can the password for MANAGER.SYS be reset?

Gilles Schipper says

Not easily. If you can log on as operator.sys, you should be able to store off the system directory to tape, as follows:

:file t;dev=tape:store command.pub;*t;directory

Now that you have the directory on tape, you should be able to look around with fcopy (and the ;char and ;hex options) to find passwords for manager.sys.

John Stevens says

If you're logged on as OPERATOR.SYS, do LISTACCT on all the accounts that may have SM capability, then logon as the MGR/MANAGER of those accounts: TELESUP, SUPPORT, (I can't think of others right now, but LISTACCT will find them). Login as those users (unless you don't have those passwords either), and LISTUSER MANAGER.SYS MPEX might help ease some of this as well.

December 09, 2016

Friday Fine-Tune: Driving Filesystem Checks

In the middle of a full backup, the HP 3000 at James Byrne's shop came to a system halt at 3 AM. It was the kind of halt that puts up those puzzling abort messages not even HP has fully documented. For example, about SA 1458, Robelle's Neil Armstrong said, "My experience with SA 1458 is that it is a catch-all abort. You need to look at the subsystem information and the only way to truly know the root cause would be to get a dump and analyze it." He referenced a webpage that breaks down the process of doing 3000 system failure analysis, too.

When a halt occurs during a backup, there's always the chance the 3000's filesystem has been injured. "I'd say run FSCHECK.MPEXL.TELESUP and check your filesystem," said Keven Miller of 3k Ranger. He added that a former HP support expert, Lars Appel, "instructed me that System Abort messages are in subsystem 98. From the MPE Error Messages Volume 2, Chapter 4, System Aborts, 1458 MESSAGE means A critical process is being terminated due to a trap."

Sure enough, power interruptions at Byrne's shop introduced damage to an Image database.

We reached the point last summer were we were toying with going off-grid simply to avoid the repeated power interruptions. If this sort of thing is causing damage then we will have to consider it. And it seems to; we now have a broken backward chain in one of our Image databases. A thing that I cannot ever recall. Coincidence? We are doing a backup, and then I will be using Adager to go in and take a look.

FSCHECK is an included tool on the 3000. It's simply there to validate extents and scan the table cache for missing files. Better tools include not only the legendary Adager, but independent support suppliers for the 3000 owner. People who know the fast commands of tools like CSTM. "What does your support provider say?" asked one support vendor. Self-support can be backed up by 3000-L questions. Some of the advice about the halt even ran to looking at memory issues. A provider can help eliminate these possibilities.

Capricious power service from his utility — a government service that's been privatized — has extracted a price on Byrne's Series 918.

I am simply trying to find out if there is any way of examining whether or not we actually have a failing drive. We have spares but if there really is no need then I would rather not take the system down again after such a short interval. It has been a bad fall for our poor old 918. The system HDD was toasted by a whipsaw set of power outages on October 11; now our data disc is suspected of being ready to let go as well.

While running FSCHECK, Byrne was advised to use the commands

Check all Dev=allSyncaccountingFSCOUNT / 10000

He also received instruction from Mark Ranft on how to scan logs using CSTM to find disk errors.

Sign on as Manager.SYS. Do a LISTF LOG####,2 to find the start and ending Log files. Alter the log file number range in the commands and enter the commands in LOGTOOL.

The results are written to two files LOGOUT1 and LOGOUT2. I had this set up to run weekly on my systems. And if the files had errors in them, the job would email the results to me for review. You will see errors due to SCSI or FC resets on every boot, so check the timestamps to tell if the boot caused the error.

At Byrne's shop the Series 918 was recovered after Adager did its repairs. "During the Adager repair an infrequently occurring high-pitched but low volume sound was noted," he said. "This appeared to emanate from the 3000. We have not heard it since the Adager repair completed. A suspicion arose that we might have a disk about to lose a bearing."

Once the recovery was complete and the disk replaced, the trademark wisecracks of long-time 3000 vets began to arise. The sound during Adager's repairs "would have been the sound of the chains being dragged around the disk to put them back straight," said Alan Yeo, "and possibly the sound of a very tiny virtual Alfredo welding a few broken links back together."

Byrne noted that the problems with the Series 918 disks "started I was working on installing the 9.6 version of PostgreSQL on a new FreeBSD host. I wonder if the HP 3000 is throwing a temper tantrum? Naaah. Cannot be."

December 02, 2016

Friday Fine-Tune: Print classes, STORE, FTP

I need to add classes to a printer on my HP 3000. I keep getting a message that the LDEV doesn't exist from SYSGEN's I/O section when I issue the ACLASS command. I can see the device exists, so what's the problem?

Devices on your system that are configured through NMMGR must have their classes changed through NMMGR, which is probably why you're getting the error message. Run NMMGR and go to the PROFILE for the device. You should be able to add classes to the device at that point. After making changes in NMMGR, remember to VALIDATE these changes to NMCONFIG, and then cross-validate using the RDCC command from SYSGEN's SYSFILE section.

NMMGR changes take effect after the system has been reSTARTed and the power has been cycled on the DTCs. You should cut a System Load Tape (SLT) making these changes so you'll have them on tape for your next UPDATE.

If your device was configured through SYSGEN, the device should have its classes modified through SYSGEN. You're right, you do use the ACLASS command in the IO section. Changes to the IO section require a START to make them take effect.

In HP's MPE/iX manuals I am are directed to backup the system as follows:

STORE @.@.SYS,@.@.@-@.@.SYS;*T;DIRECTORY;SHOW

What is the practical benefit over

STORE @.@.@;*T;DIRECTORY;SHOW

People trot out the reason from the distant past: if you have to do a re-install it is good to have PUB.SYS (and maybe the TELESUP account) on the front of the tape so you can restore quicker.

Gilles Schipper woke us up with his response. “The reason one would perform the former over the latter is to ensure that all of the files residing in the SYS account be placed at the beginning of the backup. Whether that is a good thing is, in my opinion, dubious."

Gilles said, “Perhaps, in the days of half-inch, 1600-BPI magnetic tapes, it could provide for a quicker restore of files in the SYS account than, say, files in the YAHOO account. In any case, with tape technology, where retrieval of any group of files from any spot on DDS or DLT tape is quick, the raison-d’être for such file placement strategy disappears - assuming there ever really was a good reason.”

If you have user volume sets you MUST explicitly list them after the DIRECTORY keyword or else the directories for the user volume sets will not be stored.

Is there a way to use FTP to move an IMAGE database between two HP 3000s?

FTP by itself won't work, as it doesn't support user-labels, which are included with the IMAGE root file. DSCOPY, while not being as trendy as FTP, does know about user labels. The Web has freeware LZW compressor programs which know how to archive IMAGE datasets, so you can use FTP without having to change the file codes to non-priv values beforehand. Just archive the sets into one non-priv package, FTP it as a binary file, and then use LZW to re-expand the data-sets back to their original form. LZW programs, along with many other open source gems, are available at SourceForge.

November 25, 2016

Friday Fine-tune: Adding disks and IP blocks

Is it possible to add a disk drive "on the fly" without doing a reload?

Jeff Kell replied:

You generally have to shut the system down to install and cable the disk to avoid electrical/interface problems. The usual approach is to use SYSGEN to configure the new device on the path where it will reside, keep the new configuration, shutdown the system, install the disk and do a START NORECOVERY.

Once the disk is recognized by the system, you can add it to your running configuration as follows (assuming the new drive will be LDEV 5 in the system volume set):

This will add the volume to the system volume set, but it also has some side effects. Since the new volume is "empty" and the disk space allocation routines attempt to "balance" loads across drives, all of your new files and transient space will be allocated on the new drive until it's capacity approaches that of the other volumes. This will create an I/O bottleneck on that drive, at least initially.

You could selectively :RESTORE certain accounts (or the whole system) to try and balance the allocation. You could also perform an INSTALL and a :RESTORE for better efficiency, but at the cost of a great deal of time. There are also certain third-party utilities that will balance disk utilization across members of a volume set. These utilities work online on a running system and don't require any downtime.

The network configuration of our HP 3000 was originally set up with one block of IP addresses. Now I need to add another block of addresses. Where do I add these in NMMGR?

You can add an IP address using NMMGR the following way:

After typing NMMGR, select "Open Directory" .

Then select "Update Dir."

Now select the "Add" option (F5.)

You are placed in a screen where you can enter the IP Address of the machine. The type is generally set to 1( IP).

Now press the "Save Data" (F6) option, back out of NMMGR, and you are done.

November 23, 2016

Mailing news from the HP 3000: an old skill

Internal mail hosts remain a crucial tool in datacenters, even some running MPE/iX. "You still host your own email?" is not a question you'd only pose to a crazy manager. An organization's security standards can be so high that no outside mail server will be trusted. In the earliest days of email, 3k Associates built and sold a beautiful native MPE mailing system, Netmail/3000. It's a smart mailserver, meaning it doesn't require that an organization's e-mail be piped through an Internet provider's mail server for final delivery. Then in the late 1990s, HP's lab started the long process of porting sendmail to MPE/iX.

Now some 3000 sites are looking at how to replace their 3000-based mailing software as they migrate. One of them contacted us this week to ask about an alternative to sendmail. Linux is their migration target, after a history using the 3000 that goes back to the days of HP Deskmanager. Tim O'Neill shared a story while asking about an alternative to sendmail.

I saw that FreeBSD Unix has its version of sendmail. Seeing reference to FreeBSD made me recall a story about FreeBSD running on an old HP 3000, maybe a Series 70 or an early Spectrum system. I think I have read that FreeBSD is at some sites still running in production mode, as MPE and MPE/iX are. It also made me wonder what the installed base of FreeBSD might be — and how that compares to the installed base of MPE and MPE/iX on old hardware and on Charon hardware.

FreeBSD, like MPE/iX, has some surprisingly large companies using it. You might have heard about one of them called Netflix. Of course the Charon HPA emulator from Stromasys makes every remaining product and archival 3000 a candidate for the kind of longevity we see in FreeBSD.

Sendmail has a colorful history. The Unix Hater's Handbookdevoted a full chapter to the software's vulnerabilities; sendmail comes from the Unix heritage, after all. By 2003, HP was still patching sendmail to shut down security breaches, although the breaching wasn't nearly as serious on MPE/iX as on Unix variants including Linux. Sendmail's open source capabilities are now under the banner of ProofPoint, the company that purchased the sendmail resources in 2013.

Sendmail's worldwide release was last updated in 2014. HP announced it was testing sendmail to place in the Fundamental Operating System in November, 2001—a month that's famous in the 3000's history for other reasons. But the software moved along to an 8.13.1 release in FOS. It's only one major release behind the worldwide open source version, now advanced to an 8.14 release. Sendmail also includes encryption.

Sendmail has included encryption facilities since 8.11. That's where security capabilities descend onto the requirements. Encrypting mail is a common feature in commercial hosting solutions. Sendmail/iX sends mail created by and triggered from HP 3000 applications, given enough technical know-how.

There's a robust webpage about the 3000 mail solution that was started by Mark Bixby. He's the engineer responsible for lighting the fire of open source flames at HP. Keven Miller of 3K Ranger has updated and maintained the page and its knowledge about Sendmail/iX. The software itself is in your 3000's SENDMAIL account in a version-specific group named vuuff.

November 18, 2016

Friday Fine-Tune: Tricks with command files

I'm working on a command file on my HP 3000. Is there any way to have it copy part of itself into a separate (temporary) file?

Jeff Vance replies:

MPE does not support the Unix concept of ‘here’ files, where input data for the command can reside in the same file as the command, except in the case of jobs. But even in a job, you may not include inline data for a script or UDC invoked by that job.

The SPOOKHELP script may be of some use. This single script contains the help text for all of the SPOOK commands plus the code to search for and display that text once HELP xyzzy is entered.

How can we execute a command after a user enters the :bye command in MPE?

Olav Kappert replies:

It is possible to execute many commands after the bye has been entered. Simply create a UDC (maybe a cmd file) called bye.

The contents of the UDC for the command bye is up to you. This would be useful if you want to do statistics before the session terminates.

John Pittman adds:

Don’t let them do a bye. We don’t allow any users access to OP system prompt at all. They get a logon no break UDC that runs a menu, and when they end the menu, they get logged off.

Inside that UDC at exit time, we build a string giving user, connection point (LDEV or IP of their PC) connect time, CPU date etc and append it to a log file. Then we know when anybody last used the system, how many users are using different connections, or when different user names are using the same connection point.

November 11, 2016

Friday FineTune: Internal disk plus VA array

FiberChannel Storage Area Networks and shared tape libraries became popular in the years after HP stopped making its MPE/iX hardware. The HP 3000 supports SAN from the XP series of RAID devices to the VA7100 disk arrays. But how much should you rely on a RAID or SAN device? Internal storage devices might seem to be yesterday's tools, but the modest drive inside an HP 3000 can still be very useful — even if a company has invested in the FiberChannel storage solution of the VA7100.

Moving to the VA solution has great benefits, as reported in a story about using a VA7100 array with the 3000s. But booting directly from a VA array — well, you'll need an N-Class server (native FiberChannel installed) or a very expensive HP A5814A-003 Fiber/SCSI router (if you can find such a thing) to employ VA in the 9x9s.

The Crossroads SA-40 Fiber/SCSI switch will link a VA array to 9xx 3000s. It just won't let you boot your MPE/iX system from any of its drives. Craig Lalley of EchoTech recommends the affordable Mod 20 arrays for boot capability.

Internal drives remain as important as the VA arrays for a Series 9xx HP 3000, or even to the XP line of HP arrays. Even important enough to even duplicate them.

A second bootable disk inside your 3000 can take some forethought, but can be essential to smooth recovery of an LDEV 1 failure. James Killam of HP once reported to the 3000 mailing list, "Keeping a bootable image of MPE on one of the internal drives... saved me once at 2 AM when we lost total connectivity to the XP array the system was attached to and we had some serious troubleshooting to do."

Donna Hofmeister, former OpenMPE board member, has noted:

I did one internal disc and it was expressly for memory dumps. A second internal disc with a bootable image would be wonderful insurance. It would take some planning to be able to manage it all, but there’s no reason why you can’t have two bootable discs. I’ll point out the obvious: if LDEV 1 is internal, and you have a multi-disc system volume, and the remaining system storage is on a disc array — uh, what’s the point? If LDEV 1 fails, you’re toast.

Hofmeister said she's had a drive fail on a VA array. "That array worked perfectly and switched over to the spare without a blink. Given that I had two systems sharing this array, I was more than pleased with how well it worked."

Drive failures are among the most likely of hardware problems using HP's 3000 devices. A second internal drive in a system can make a big difference in recovery time. The other way around drive failures is to make your way onto emulated PA-RISC systems from Stromasys.

October 17, 2016

Rebuilding Systems Faster and Better

I'm looking at how to save as much time as possible in rebuilding an HP 3000's software and directories. My options seem to be using STORE, versus the sysgen tape command "tape store=@.@.@". What's the best way to go here?

Donna Hofmeister of Allegro replies

Unless your system is small (like a 918 with 8-12GB of disc), you don't want to try to do a full backup via sysgen. If you really do a full backup then I prefer this syntax “store /;...” as it is self-documenting and you know that the Posix files will be backed up as well. (On older releases of MPE, @.@.@ did not back up Posix files <eek>)

You want to make sure that you run 'buldacct' periodically (and routinely). You also want to make sure that you are somehow backing up your directory (store /;*t;directory, for example). Between the two, you have belts and suspenders (for recovering your accounting structure).

On older releases of MPE, you want to make sure that the network is shut down prior to making your SLT tape. And it's still a good idea to have the system quiesced when making an SLT, since everything in the sys account (and .pub.sys in particular) will be locked while the tape is being made. Nothing quite like grumpy users to make your day.

Just as a matter of preference, you should normally do a BULDACCT @ at the beginning of the weekly full backup, then the DIRECTORY option along with ;ONVS=MPEXL_SYSTEM_VOLUME_SET, PRODUCTION_SET, etc as a belt and suspenders approach for that day we all hope never comes.

Mark Ranft of Pro 3K outlined the use of sysgen

Here are instructions for a complete backup of basic MPE/iX system via SYSGEN's TAPE option. It is best if everything fits on a single DDS (or DDS-2 or DDS-3 or DDS-4) tape cartridge, but it will ask for a second (additional) tape(s) as needed.

Note: I included a 'second_volume_set' which can be changed or removed

Note 2: The line below is >80 characters, so you have to know how to create a file so that this does not wrap (or make other adjustments.)

Step One - Create an indirect file containing the following as a single line...

October 07, 2016

How to Make a Windows to 3000 FTP Move

I need to move a file to our 3000 from a Windows server with FTP and Windows doing the put. The Windows file has longish variable length records, but I would like them converted to fixed length on the 3000. When I tried, this was the result:

550 The FILE EQUATION STRING option (item# 52) is not in a valid file equation form. (FILE OPEN ERROR -449)

Keven Miller replies

Item #52 refers to HPFOPEN. From the intrinsic manual

52 File equation string:

Passes a character string that matches the file equation specification syntax exactly. (Refer to the FILE command in the MPE/iX Command Reference Manual.) This option allows the specification of options available in the FILE command.

I don't like the trailing ";move" in your command string. I'd remove that. Also, you have comma after "ascii" and it should be a semicolon, like this

;rec=-1024,1,f,ascii;disc=1000

Tony Summers adds

I'm pretty sure ";MOVE" is only valid if you're using the HP 3000 FTP client. Have a look at page 20 of the link below at TeamNA Consulting, another place where the MPE and HP 3000 manuals live

September 26, 2016

3000-L connects again after a silence

As if on cue after our report about its silence, the 3000-L mailing list and newsgroup sprang back to life over the weekend. One problem solved by the 415 readers was how to identify if a store to disk backup is a LZW compressed backup file. A Tracy Johnson report also confirmed that a LISTF,2 can report the time of each LISTF, by writing a specialized job.

Meanwhile, a 15-year-old HP 3000 with network connection troubles got advice from the newsgroup's readers. A Series 969 running MPE/iX 6.0 would not be the first thing you'd choose for interfacing to an internal website. But when a 3000 has data that a user needs over the Web, the server is the place to go.

Trouble started to surface when clients access a webpage which then opens a telnet session with the 3000, grabs the info, and then returns the data to the webpage.

We’ve been getting more and more errors over the last year, culminating in non-stop Could not initialize data in path with TCP, which then blocked anyone accessing us through our webserver. We’ve tried many changes but cannot seem to get past this.

When it locks up, the HP 3000 keeps running but won’t accept any new sessions. Which means our clients can’t run searches. Which is very bad for us. Sometimes we can stopnet and startnet and it will work for a while, but then the errors start again. Eventually, we have to coolstart to be able to have clients log in.

Mark Ranft suggested "If they are already running in the CS queue, here is the likely cause. Is there some new monitoring in place? If so, it may not be behaving well on the network.

What happens is someone uses a telnet or ftp connection to monitor whether the network is up on the 3000. They send the SYN, the hp3000 answers with the SYN ACK, and then the 3000 receives a reset before the connection is complete. This handshake sequence causes this exact error.

Also inetd and other HP3000 networking improved greatly in later versions of MPE/iX. If they must stay on 6.0, they should at least be sure to install the latest patches.

A third party support company served this troubled user until the support vendor folded, "and the only options we found weren't affordable." Getting the 3000 back up will trigger a revisit of that situation. Any server with critical customer data on it—and doesn't have a support vendor—relies on the largesse of the 3000 volunteers of 3000-L. That mailing list did go without a new message for more than a month, a troublesome response time for a critical server.

September 14, 2016

Dancing the Samba services tune, MPE/iX-style

Ten years ago this week we were promoting instructions on how to use Samba better on HP 3000s. Samba is "a group of programs that allows a Unix host to act as a fileserver for Windows platforms," according the MPE/iX documentation rolled out in 1999. The file-sharing and printer sharing software which has been a part of MPE/iX since the 6.0 release "allows Unix-like machines to be integrated into a Windows network without installing any additional software on the Windows machines. Many different platforms run Samba successfully; and there are nearly 40 different operating systems which support Samba." And many more now, a decade later.

HP brought features of Samba to the 3000 in a port called Samba/iX. "It is a solution for those wishing to access HP 3000 disk storage and printers (both networked and spooled from MPE/iX) from common PC client operating systems like Windows." Samba/iX allows access to disk and printer resources of MPE/iX by providing standard SMB file and printer services that are accessible from PC clients and their applications. An administration tool called SWAT makes Samba so much easier to use.

Samba 3.0.22 is distributed by the following MPE/iX base patches. Your independent support provider should be able to help you round one of these up. They've got the latest functionality.

SMBMXY6D (BT) for MPE/iX 6.5

SMBMXY6E (BT) for MPE/iX 7.0

SMBMXY6F (BT) for MPE/iX 7.5

The (BT) stands for Beta Test. HP never cut the 3.0.22 version loose as a general release (GR) version. For reference, the following are GR versions with less functionality.

SMBMXG3A (GR) for MPE/iX 6.5

SMBMXG3B (GR) for MPE/iX 7.0

SMBMXG3C (GR) for MPE/iX 7.5

Even a total 3000 network newbie can get Samba up and running. Samba must be running before you can run SWAT. Here's some useful info when getting SWAT going.

If you’re running an older version of Samba, you’ll need to modify ‘/usr/local....’ to point to where SWAT actually lives (and case is important). The user needs to match the user in your samba daemon jobs. (For many, it’s MANAGER.SYS, for you it may be MGR.SAMBA) When you're connected to your MPE/Samba server through a browser to access SWAT, you'll be asked for a logon and password. This is a good thing.

After changing your services and inetdcnf files, all that you should have to do is give inetd a swift kick (e,g, :inetd.net -c ) Check inetd’s $stdlist after doing that and you should see that it brought in the new configuration.

In your browser point to http://xxx.xxx.xxx.xxx:901/ (where xxx.xxx.xxx.xxx is the IP of your 3000). Or you can use the name of your 3000 too.

When updating Samba and Apache config files, some are picky about how their records are terminated. Robelle's Qedit can make the needed adjustments. Be sure to know what version of MPE/iX you have installed, including patches.

August 12, 2016

How to purge UDCs on the HP 3000 safely

The software vendors most likely to sell products for a flat rate -- with no license upgrade fees -- have been the system utility and administration providers. Products such as VEsoft's MPEX, Robelle's Suprtool, Adager's product of the same name -- came in one, or perhaps two versions, at most. The software was sold as the start of a relationship, and so the relationship focused on the understanding the product provided for people responsible for HP 3000s.

That kind of understanding might reveal a Lewis Carroll Cheshire Cat's smile inside many an HP 3000. The smile is possible if the 3000 uses UDC files and the manager uses only MPE to do a file PURGE. Of course, PURGE ships on all MPE systems. Using that means you'll have to rebuild the UDC catalog. But even that's not enough.

Stan Sieler of Allegro shared a story about this recently. "We recently encountered a site where—somehow—an HFS filename had gotten into COMMAND.PUB.SYS. You can't delete UDC entries with HFS filenames, nor can you add them. I had to edit the file with Debug to change the name into something delete-able." Then there's the rebuilding of the catalog. Keven Miller has contributed a program that sorts and reorganizes UDC files.

There is a more complete way to remove such things from a 3000's storage. You're careful about this because eliminating UDCs with only MPE might leave a user unable to use the server. That grin that lingers is the UDC's filename.

User Defined Commands are a powerful timesaver for 3000 users, but they have administrative overhead that can become foolproof using the right tools. These UDCs need to be maintained, and as users drop off and come on the 3000, their UDCs come and go. There's always a chance that a UDC file could be deleted, but that file's name could remain in the filesystem's UDC master catalog. When that happens, any other UDCs associated with the user will fail, too. It might include some crucial commands; you can put a wide range of operations into a UDC.

When you add a third party tool to your administrator's box, you can make a purge of such files foolproof. You can erase the Cheshire Cat's grin as well as the cat. It's important because that grin of a filename, noted above, can keep valid users from getting work done on the server with UDCs. This is not the reputation anybody expects from a 3000.

First you have to find all of your UDCs on a system, and MPE doesn't make that as straightforward as you might think. Using SHOWCATALOG is the standard, included tool for this. But it has its limitations. It can display the system-level UDC files of all users in all accounts. But that's not all the UDCs on a 3000.

MPE, after all, cannot select to show a complete set files by attributes such as program capability. Or for that matter, by last accessed time, or file size, or file security. It's a long list of things that MPE makes an administrator do on their own. Missing something might be the path to looking foolish.

Employing a couple of third party tools from VEsoft, VEAudit and MPEX, lets you root out UDCs and do a foolproof purge, including file names. VEAudit will list all of the UDCs on a server, regardless of user -- not just the ones associated with the user who's logged in and looking for UDCs. The list VEAudit creates can be inverted so the filename is the first item on each line. Then MPEX will go to work to do a PURGE. Not MPE's, but a user-defined purge that looks for attributes, then warns you about which ones you want to delete, or would rather not.

By using MPEX -- the X stands for extended functionality -- you can groom your own PURGE command to look out for files that have been recently used, not just recently created. MPE doesn't check if a purged file is a UDC file.

Such 3000 utilities provided the server and its managers with abilities that went far beyond what HP had built into MPE and its IMAGE database. Now that MPE is moving on, beyond HP's hardware, knowing these third party tools will transfer without extra upgrade fees is like ensuring that a foolproof MPE will be running on any virtualized HP 3000.

They're an extra-cost item, but how much they're worth depends on a manager's desire to maintain a good reputation.

In the earliest days of the sale of these tools, vendors were known for selling them for the price of the support contract alone. That's usually about 20 percent annually of the purchase price. If a $4,000 package got sold that way, the vendor billed for just $800 at first. It made the purchases easier to pass through a budget, since support at the manager-tool level was an easier sell. Think about it. Such third parties passed up $3,200 per sale in revenues in the earliest days. They also established relationships that were ongoing and growing. They were selling understanding of MPE, not just software.

This kind of practice would be useful for the community's remaining software vendors. This is not the time to be raising prices to sustain MPE computing, simply because there's a way to extend the life of the hardware that runs MPE. As the number of MPE experts declines, vendors will be expected to fill in the gaps in understanding. Those who can do this via support fees stand the best chance of moving into the virtualized future of 3000 computing.

August 10, 2016

Measure 3000 performance for datacenters

Measuring the performance of an HP 3000 used to be a leverage point for increasing investments. By now the numbers help justify continuing to use the server in a datacenter with newer boxes. "We think of our HP 3000s as stable, and even reducing in usage over time," says one systems manager, "though actually as the company grows, the data requirements and load on the 3000s increases."

One way to measure a 3000's footprint is the amount of memory it requires. Memory upgrades cost nothing like what they did even 15 years ago. But any spending at all makes that 15-year-old server suspect. HP's Steve Macsisak recommended sessions x 4, plus jobs x 16, plus 64 MB as the criteria for memory usage.

An HP 3000 uses as much of its memory as possible to make processing efficient. The design of the PA-RISC architecture makes memory the most important element of performance, after IO speed. It's not that unusual to see a 3000 using 100 percent of its memory, according to field reports. There's also CPU usage to measure.

CPU percentages can come via the REPORT command. Count up the CPU seconds used in the week, and divide by the total number of seconds available (604,800). But for all of this, it doesn't feel like a graphic report the rest of the datacenter gets from its Unix and Linux systems using SAR. There may be a program inside a 3000 that can help, even if the company never purchased performance tools from Lund. HP's Glance gives away its reporting power in its name, one manager has joked.

Since there's no port of SAR for MPE/iX, something else must stand in. Some systems have HP Scope, the software that dives in deep enough to produce report-ready numbers. It's not the smooth path a 3000 gets from Lund's Clearview Performance Manager. Scope is the HP Performance Collection Software sold by the vendor while it still had an active 3000 business.

SCOPEXL, if you are using an MPE/iX system), UTILITY, and EXTRACT. SCOPEXL is the performance data collector for MPE Systems. It continuously collects and summarizes performance data. UTILITY and EXTRACT are the host programs that let you interact with SCOPEXL and manage the data that it collects.

HP's Scope documentation describes how to use the collection and management software: how the host components interact; detailed command descriptions for each program; and suggestions on how to use the programs to analyze and archive data efficiently.

July 11, 2016

How to Use Input to Create Output Files

Intrinsics are a wonderful thing to power HP 3000 development and enhancement. There was a time when file information was hard to procure on a 3000. It was a long time ago, as I was reminded by Olav Kappert in a call about his HP 3000 history. "The high point in MPE software was the JOBINFO intrinsic," he said. Kappert started with the 3000 in 1979.

Fast-forward 37 years later and you'll find questions from a different programmer still working on a 3000, adding features to a system. The Obtaining File Information section of a KSAM manual on MPE/iX holds an answer to what seems like an advanced problem.

I'm still using our old HP 3000, and I have access to the HP COBOL compiler. We haven't migrated and aren't intending to. My problem is how to use the characteristics of an input file as HPFOPEN parameters to create an output file. I want that output file to be essentially an exact replica of the input file (give or take some of the data). I want to do this without knowing anything about the input file until it is opened by the COBOL program.

I'm using FFILEINFO and FLABELINFO to capture the characteristics of the input file, after I have opened it. After I get the opens/reads/writes working, I want to be able to alter the capacity of the output file.

Francois Desrochers replies

How about calling FFILEINFO on the input file to retrieve all the attributes you may need? Then apply them to the output file HPFOPEN call.

Donna Hofmeister adds

You might want to get a copy of the "Using KSAM XL and KSAM 64" manual. Chapters 3 and 4 seem to cover the areas you have questions about. Listfile,5 seems to be a rightly nifty thing.

But rather than beat yourself silly trying to get devise a pure COBOL solution, you might be well advised to augment what you're doing with some CI scripts that you call from your program.

In a lively tech discussion on the 3000-L list, Olav Kappert added,

Since you want to do this without knowing anything about the input file until it is opened by the COBOL program, the only way is to use one of the MPE intrinsics to determine all the characteristics of the file in question. Then do a command build after parsing that information.

Michael Anderson added details on how the 3000's CI scripting can build upon the fundamentals of file information and COBOL.

I like Donna's plan.This is a strategy that will also help whenever you want similar functionality on a NON-MPE platform. Also, although COBOL is very capable, an external script might be a better tool. You don't always need a hammer.

This is hypothetical, to try to make a point. From your MPE CI prompt, type HELP FINFO. You should be able to set some variables (SETVAR FILEA "XXX"), and using FINFO add some more variables. Then from COBOL using HPCIGETVAR, string together a BUILD command (with a bigger LIMIT maybe), and call "HPCICOMMAND". You could string the build command from a command, into a single variable, then COBOL only needs to HPCIGETVAR once.

You can also write a script to do everything you want, and call HPCICOMMAND to run the script, pass it parms. It's pretty cool, and it makes your COBOL application more portable. (Same program, different script).

For example: On MPE I once wrote (using COBOL) a small utility to CALL DBINFO, extract all the meta-data from any IMAGE database, and then create, and write to the NEW KSAM COPYLIB, ending up with all the COBOL copylib modules needed for all datasets for any database, including call statements and working storage. My point to all this: I used CI scripting to create and write to the copylib. I actually used ECHO to write the copylib ksam file from a CI script. Now, seeing how I work more on HP-UX and Linux, plus OpenCOBOL and Eloquence, I should be able to compile this same program on Linux with minimal modifications, only changing the external script.

I use this method to access SQL databases, and much more, using OpenCOBOL and the Tcl/Tk developer exchange. This way I can run the same program, same script almost anywhere, no matter, Windows, Mac, or Unix.

Eric Sand, another veteran of the 3000, commented that this kind of challenge really shows off the range of possibility for solving development problems. "You can create almost any cause and effect in MPE that you can imagine," he said. "Reading about your concern gave me a little rush, as I mentally organized what I wanted to do to address your concern."

Also note that since you're asking for the value of HPYYYYMMDD three different times, there's the tiniest chance that if you executed this right at last part of the second at 23:59:59, then the value could roll over to a new date in the 2nd or 3rd call, and you could possibly get the wrong date, or even year. Unlikely, I know. But to avoid that, you'd want to capture the current date once, then operate on that, as in:

Kudos to Barry and Michael for apparently correctly guessing that our manager was interested in a CI-based solution. Next time, you might want to make that clear. I was thinking along the lines of SPL/Pascal/COBOL and the CALENDAR intrinsic, as well as whether I should ask the obvious questions or not. Of course, the easiest method is: ECHO 06/29/16

Any code that wants a coherent date and time using separate sources (e.g., CLOCK and CALENDAR intrinsics, or HPTIMEF (#1)) needs to be aware of midnight crossing. Thus, I use a helper function in Pascal like:

June 06, 2016

SLEEPER, Awakened for the Masses

The contributed program SLEEPER is probably the software cited most often as proof of the riches of the Contributed Software Library. First created outside of the IT shops of Boeing (according to its first writer Ray Legault) the program was among the most classic of solutions for straightforward jobsteam management. You wouldn't mistake SLEEPER for something professional like Maestro, or even the free MasterOp. But SLEEPER was contributed to the 3000 community, not cast into the free wilds like MasterOp was after its commercial career ended. It was meant to be shared.

The trouble was, SLEEPER disappeared from the community's shelves when Interex died. The CSL tapes (and eventually CDs) went off the grid, another skid-mark left when the user group careened into the void in 2005. It's been 11 years, though, and it's finally time to at least make SLEEPER ready for a wake-up call. We've got the two simple source files to share.

Nobody has liability anymore for HP 3000 contributed software. SLEEPER was never released with support or a license; it was simply part of being an Interex member at a certain level. And let us take a minute to recall that Interex folded owing millions of dollars to members and vendors. After 11 years, it's time to make this software a community resource once more. "Hey, I know a guy who can get you that" has been the means to share the utility over the last decade.

Surprisingly, it's just a well-packed 184K of SPL and FORTRAN code. MPE magic never took up much space. That's one of the reasons it was magic. SLEEPER is also a fine example of how 3000 managers helped one another.

We've thrown caution to the winds (not exactly a new event here) and loaded those two files onto our website servers for distribution. It comes with no warranty, of course. Like always with the CSL gems, using it carries the same risk that any contributed utility or application did.

The FORTRAN file is available. And the SPL code is at hand, too. One well-honored MPE/iX engineer looked over the software programs and said they were extraordinary.

The SPL is some of the better looking old-style SPL that I've seen. The FORTRAN is very nice looking.

The SPL program runs in the batch job. The FORTRAN program maintains a configuration file that is presumably used by the SPL program.

So let the jobs of MPE/iX, including those running on standard HP hardware as well as those on virtualized 3000s, be managed a bit more easily. I await your comments on rousing the talents of this classic tool.

June 01, 2016

Recovering Your Lost Logins

In the world of 2016, losing a password can be a time-bending experience. Apple's ID logins for its iCloud services, iTunes, and App Store only bear five attempts at most before they close out access to the phone. From there, you're on to a lengthy call to Apple. The smartphone is a tiny computer. The larger ones are not as strident about refusing repeated attempts, including the HP 3000.

They are secure, though. In these times when the servers can go dark for awhile as cold-start archive systems, though, a login can get misplaced. We're not supposed to write these things down, after all. What do you do when you have a question like the one below?

I restarted a 3000 9x7 after a few years in mothballs to run an old in-house app. I was able to boot up and login as OPERATOR.SYS, but cannot remember or find the password for MANAGER.SYS. Is there anyway to reset, clear, or overwrite the password file? I know the old machine is a very secure one, but now I am hoping there is a way around it.

What follows are a couple of suggestions to get back into the manager's driver's seat. However, you'll need to at least have a ticket to ride in OPERATOR.SYS.

Gilles Schipper shares what he knows.

There's not an easy way around the security. But since you can log on as OPERATOR.SYS, you should be able to store off the system directory to tape, as follows:

:file t;dev=tape:store command.pub;*t;directory

Now that you have the directory on tape, you should be able to move around with FCOPY (and its ;char;hex options) to find the password for MANAGER.SYS. And that's all assuming you haven't implemented the directory encryption feature available with HP's security product Security Monitor/iX.

There's one other way. It's a good thing, because the alternatives do not include a rescue call to HP. There's nothing like Apple's password recovery support left for the HP 3000, as the vendor has long ago left the field. However, a good support contract for an MPE/iX server can be just the thing, and we know where you can get one.

May 18, 2016

Tape drive changer a powerhouse for MPE

Autochanging tape drives used to be the stuff of science fiction among 3000 managers, but those days passed by before HP cut off making Classic 3000 MPE V systems. Just because an autochanger is a 3000 storage option does not make it automatic to program, however.

A question posed to the community by Ideal Computer Services Ryan Melander reached for help on programmatically controlling such autochangers -- to select a slot, and load the tape and come to ready. "I am trying to configure an old DDS3 auto-changer, one that I don't believe will unload and load the next tape," he said.

Denys Beauchemin mentioned HP's pass-through SCSI driver as a tool to drive the device's robot. The software was built by HP's labs and labeled as "not for the faint of heart" by engineers, but can assert a programmatic control over autochangers. Some third party programs such as Orbit's Software Backup+/iX can also do this work.

If ever there was a theme song for an autochanger at work, it would be a tune called Powerhouse. Children of the Fifties and Sixties will know it as soon as they hear it, if they've ever watched a Warner Brothers cartoon.

Some programming ideas came from Beauchemin, the engineer who developed at HiComp for the HiBack 3000 solution. "For the next tape to be brought online automatically, I seem to remember there had to be a special setting with the dip switches."

As for being able to control the robot itself, you definitely need to have the [HP] SCSI pass-through driver configured and loaded, and then you need a program to actually issue the IOCTL calls to the robot with the properly formatted SCSI commands. There was such a program a long time ago from a vendor, but that's all gone now.

Perhaps the high-test flutes and heavy octane horns of Powerhouse -- used in Duck Dodgers and the 24th and a Half Century -- can be put up on the MP3 player while fitting the driver to MPE. ("Oh drat these computers -- they're so naughty and so complex," says Marvin the Martian in one installment. "I could just pinch them.")

From our archives we found this advice from the late Jack Connor of Abtech, pointing to similar complex answers about controlling DDS changers.

Typically, there's a second SCSI port/address assigned for the transport control which allows the selection of specific tape. For MPE, stacker mode is typically selected, which tells the drive to just mount the next tape in line when requested. I don't know if the DDS autoloaders have a network connection available like the C7145NA DLT autoloaders do; with that device's web interface you can reload any tape, bypass a bad tape, and so on.

Back in 2011, John Pitman checked in to report that a much simpler solution to his changer's control needs popped up. "On re-examining my code for HPDEVCONTROL, I found I had catered for 1- and 2-digit device numbers in the string passed, but I had configured the drive as dev 777. This produced a string dev number of 77, which doesn't exist as a tape drive. Once I fixed this, it works like a treat."

May 06, 2016

Options for STORE You May Not Know

STORE is the default backup tool for every HP 3000, but this bedrock of backup has options which might exceed expectations for a subsystem utility. Gilles Schipper, the support guru of three-plus decades on the 3000, told us about a few STORE sweet spots.

To start, using the :MAXTAPEBUF option can cut a four-hour backup to three hours or less. Schipper says that increasing the buffer size default to 32K, from the usual 16K, speeds up the backup when STORE sees MAXTAPEBUF. "That's a pretty good payback for one option."

Backups don't need to be specified with an @.@.@ command to be complete. "People should really be using the forward slash," he says, "because it's easy to accidentally omit the Posix file structure if you're not careful constructing your fileset backup." The slash is so much better that a backup specified by HP's TurboStore will replace any @.@.@ operation with "./" Combining @.@.@ with exclusions can lead to omitting files which should have been in a backup.

Schipper says that including a directory on a backup is smart, but private volumes in use on a 3000 need more than :directory as an option.

"If you're using private volumes and the directory option, you'll only get the system volume directory — and you will not get the private volume directories," he says. The backup must explicitly specify the volumes through the ONVS= command, using the long name of the private volume.

The partdb option on a STORE command ensure that any 3000 databases which are incomplete will get backed up. Without partdb, if a root file "doesn't have its corresponding data sets, the root file won't get stored. It's silly, but partdb ensures they get stored." A privileged file can also look like a partial database, so partdb brings those files into a STORE backup.

A 3000 with HP's TurboStore, rather than just the default STORE, can take advantage of the :online command. "It will give you zero downtime, if you have that version from HP," Schipper says. But that begins to drift away from the no-cost STORE options available to any HP 3000 administrator or owner.

May 04, 2016

CPR for a Non-Responsive Console

On my HP 3000, after a short power blip, the console is now non-responsive. I can connect to the system's GSP port and the session is connected, but nothing is displayed. Neither <ctrl> A or <ctrl> B works. I type away, but get no response. I can then connect via VT-MGR and take the console :console !hpldevin and I receive all the console messages.

So, the messages are being sent (since I see them on the VT connection), but neither the physical console or the GSP gets any console messages. What can I try?

Gilles Schipper says

I believe a START NORECOVERY reboot is in order here. Since <ctrl> A <ctrl> B do not work, you will need to power-recycle the machine to effect a reboot. Presumably you would want to do this after gracefully stopping all jobs and asking online users to log off, if possible.

Depending upon which patch level your level of MPE is on, the :SHUTDOWN RESTART MPE command may also work from a logged-on session with at least OP capability.

Mark Ranft adds

If you haven't rebooted, I've seen similar issues. From the VT console can you try to do 'abortio 20' until it says no I/O to abort. A WHILE loop may make this easier. I've had luck with this in the past. But since Ctrl-B doesn't work, you may be out of luck.

Robert Thwaites notes

These are the simplest things to try first

<ctrl>Q (x-on)

<return>

Among the commonest issues: forgetting to do an x-on after a <ctrl>S (x-off) to stop output, so you can look at the line you are interested in. One time I saw another issue where someone had pressed <space> on the console and hadn't pressed <return>.

April 22, 2016

How to Transform MPE Spoolfiles to PDFs

HP 3000 data becomes more useful if it can be e-mailed as industry standard report documents. After more than two decades of pushing at it, Adobe has made its PDF the de-facto way to exchange documents, even complex ones.

Which might have prompted this question from HP consultant and Suprtool trainer Jeff Kubler:

Does anyone have a lead on a tool that converts spoolfiles to PDF files? Are there any Contributed Library tools?

It won't be in the Interex Contributed Library (because the programs have gone underground; ask your colleagues if they have a swap tape) but the txt2pdf product works nicely to make this conversion. Even in its most advanced version it's about $1,000, at last glance. Bob McGregor reports as much.

1. Checks a pseudo device we have setup for any spoolfiles that are on the device with an PRI >0

2. If it finds a spoolfile, we convert it to PDF and move it to a server

3 Sends an e-mail to the streamedby variable telling them the PDF doc is ready on the server.

4. Alters the priority to 0 to mark it processed

We've been using it for a couple years, and it works great — of course, once we got the bugs worked out. What's cool is if someone delete the file, we just adjust the priority to something greater than 0 and it gets reprocessed.