/bin/bash-ing MultiValue
One thing about system upgrades: They rarely go as quickly or as simply as we might hope. As more time passes, the risk of things changing independently on the two systems increases until things can get downright ugly.
I've been converting a UniData customer from a near-ancient AIX system to Red Hat Enterprise Linux for a while now. Procedurally, the basics of the conversion are pretty straightforward: Install the current versions of UniData and SB+, setup printers and external connections to the new system, move and convert the data from the old system to the new system, and perform general cleanup as needed. However, nothing ever goes quite to plan. Due to factors outside of my control, several months have passed from the beginning of this project until it went live. And as that time passed, a great deal changed on both systems.
Copying and converting the data from one system to another can be done many different ways. Using rsync, rcp, ftp, sftp — whatever is available — the gigs of data and programming can be moved nearly effortlessly. Moving the global catalog — a completely separate area used by UniData — was an adventure. To understand why we need to dive into the weeds.
UniData actually has three separate catalog spaces. The global catalog holds a copy of compiled code and supports some nifty memory management to ensure that if a hundred people access a particular program, that code is loaded into memory only once. Also, programs in the global catalog can be used by all accounts on the system without compiling the code in each account. For programs which are not in the global catalog, UniData will look at the VOC pointer in the account to determine if the program is in the local catalog, or if the program has been cataloged direct .
Unlike the global version, there is a separate local catalog stored inside each specific account which holds a copy of the compiled code. If a program runs out of the local catalog, when a hundred people access it the object code is loaded a hundred times.
The direct catalog is actually not a catalog space at all; it refers to all the catalog pointers within a VOC. Those pointers are NOT copies of the compiled code; they tell Unidata to run the actual p-code produced by the BASIC command. With the direct catalog, there is no copy of the object anywhere. Once something has been cataloged direct you will never have to catalog it again.
Programs that are cataloged locally or globally must be updated every time the source is recompiled.
With this AIX to RHEL conversion, the version of UniData and SB+ running on the two platforms are different. UniData and SB+ had already updated the global catalog on RHEL with many entries before we moved the AIX programming over. We couldn't simply overlay the global catalog on RHEL with the global catalog on AIX, as many critical things in the new versions of Unidata and SB+ would be lost.
Adding complexity to this mix, the global catalog on Unidata is a multi-dimensional directory structure at $UDTHOME/sys/CTLG. There is a subdirectory for each letter of the alphabet, and an additional "X" directory for those compiled objects that don't start with a letter. The globally cataloged p-code for UniData programs lives under each of these subdirectories.
So as we approached our go-live, I noticed the global catalog from AIX had been updated by several programs over the past months and the global catalog on RHEL had also been updated as we converted programs to that platform. With thousands of items in these subdirectories, how could I tell which things had changed in AIX since the last copy over to RHEL? More importantly, how could I get these answers quickly?
The answer came in the form of a shell script, shown in [Figure 1]. I copied the AIX global catalog over to the /tmp directory on RHEL so that I had both global catalogs on the same system, taking great care to preserve the last modification times on the AIX global catalog when doing the copy. With both catalogs on the same system, I could then compare the two directories to find out what was different. Now, I really don't care about things that are on RHEL that aren't on AIX; my concern was making sure that any globally cataloged programs that changed on AIX since the last copy to RHEL would be updated.
#!/bin/bash
. /etc/profile.d/unidata.sh
for FILE in `find /tmp/CTLG`
do
SBCHECK=`echo $FILE | egrep '_SB543|_SB524|_SB534|_SB538|_SB543'`
if [[ -f "$FILE" && -z "$SBCHECK" ]]
then
BASE=`echo $FILE | cut -c6-999`
if [[ ! -f "/usr/udthome/sys/$BASE" || "$FILE" -nt "/usr/udthome/sys/$BASE" ]]
then
#convcode $FILE
#cp $FILE /usr/udthome/sys/$BASE
echo $FILE
fi
fi
done
Fig. 1
Line three loads a profile called unidata.sh. This sets up the $PATH and other variables for UniData to run properly in the script.
Line five is the golden ticket for this task. This for-loop issues a find command that retrieves all of the file names in the AIX global catalog (that I temporarily stored at /tmp/CTLG on RHEL) and then it iterates through that list. This is a remarkably easy way to navigate through a directory of files regardless of how many files are in that directory tree.
I didn't want to bring over any global catalog stuff from old versions of SB+, so I used this variable SBCHECK to see if the name of the file contains any of the SB+ global catalog prefixes. On line eight, I check to see if the file name given to me by the find command is an actual file (-f = is a file) and if the $SBCHECK variable is empty (-z = zero length string). If both of those are true then we can figure out which file is newer; the one on AIX or the one on RHEL.
On line ten, I'm cutting off the "/tmp/" from the front of the file name and assigning that to a new variable BASE. This will leave "CTLG/ programName " in BASE. From there, I check to see if this file does not exist (! = not, -f = is a file) or if the AIX catalog is newer (-nt) than the version on RHEL. If the file on AIX has a newer last-modification time than the one on RHEL, that file needs to be converted and brought it to RHEL.
In this particular version, I've commented-out the UniData routine that converts the byte-order of the p-code and also commented-out the copy command that installs this into the RHEL global catalog. All I want to do here is display the name of each AIX item that is newer than the RHEL item, which is where I started — and also ended — with this task.
Once I knew the discrepancies, I uncommented those two lines to convert and install the AIX globally cataloged version into the RHEL global catalog.
There are, of course, some variations that can be done with this technique, but this simply illustrates how powerful the for-loop can be in shell scripting. With shell scripts being able to do looping, if-conditions, simple variable assignment, and much more, you might find — as I did — that this can provide the perfect solution for those onerous programs where a BASIC routine just doesn't seem like the right fit.
