For a while now, IBM have diverted their efforts in storage multipathing from SDDPCM to the default AIX PCM . This brings a few advantages, but specifically for me, it means the driver is now updated as part of the AIX system maintenance, and is no longer something I need to maintain separately. All significant functionality that SDDPCM provides, can now be provided by the default AIX PCM driver. Additionally, SDDPCM is not supported on POWER 9 hardware.
For those currently using SDDPCM, removing the driver can be somewhat complicated, and even more so when the boot LUN (rootvg LUN) is being managed by SDDPCM. Buried deep in the Multipath Subsystem Device Driver User’s Guide is a command called manage_disk_drivers. The manage_disk_drivers command can be used to display a list of storage families, and the driver that manages or supports each family. This allows us to easily (with a reboot if you boot from an SDDPCM managed device) switch the driver from SDDPCM to AIXPCM (or vice versa).
I built a basic two-node PowerHA SystemMirror (HACMP) cluster for my team a little while ago to use as a test environment for patch updates. While it wasn’t a true reflection of how the production environment is configured, it was enough to test functionality. As such, I configured a single virtual ethernet adapter in each cluster node, which would house both the boot IP and the service IP of the cluster. After a couple of weeks, I noticed that my weekly NIM mksysb’s on one of the two cluster nodes was always failing. Further investigation found that the NIM mksysb’s would always fail on the cluster node that had the active resource group with the service IP attached to it. If I failed the resource group over to the other cluster node, the NIM mksysb would complete successfully.
Update: Patch links from both IBM  and Perzl  at the bottom of this post.
Below I detail how I patched over 800 AIX LPAR’s that were exposed by CVE-2014-6271  and CVE-2014-7169 , also known as shellshock, using the NIM server.
From everything that I’ve been reading on IBM’s Knowledge Centre, creating an LPP source containing only RPM’s isn’t possible. To patch my AIX environment, I decided to use the “script” resource available to the NIM master, along with the pre-existing NFS mounts that I had configured.
We recently upgraded the firmare on our Power frame, which required shutting down some of our AIX LPAR’s. The firmware upgrade went well, as did starting up all the AIX LPAR’s, except for one. This particular LPAR booted to HMC LED code 2700 and hung there. I restarted the partition to the Open Firmware (OF) prompt, and tried booting again using verbose mode to see where the boot process was hanging.
Having the pleasure of working across many client accounts, it’s funny to see some of the convoluted scripts people have written to receive alerts from the AIX error log daemon. Early in my AIX career, I used to do the exact same thing, and it involved a whole bunch of SSH keys, some text manipulation, crontab, and sendmail. Wouldn’t it be nicer if AIX had some way of doing all of this for us? Well, you know I wouldn’t ask the question if the answer wasn’t yes!
AIX has an Error Notification object class in the Object Data Manager (ODM). By default, there are a number of predefined errnotify entries, and each time an error is logged via errlog, it checks if that error entry matches the criteria of any of the Error Notification objects. What we’re about to do, is add another entry into the errnotify object class to be checked and actioned upon.