CSM Commands (dsh, dping, dcp and dshbak) for AIX

Cluster Systems Management (CSM) software provides a bunch of tools to manage from a single point of control your AIX environment. There are a number of commands available[1] but the main four which I find quite useful are dsh, dping, dcp and dshbak. In this post, I’ll go over each command and some of its uses.

Before we begin, we need to select a server which is going to be the point of control. While this can be any AIX server on the network, it’s mainly common to have your NIM server as the point of control. In the case of dsh and dcp, we’re going to be using SSH, so keys[2] need to be setup between your point of control server and the rest of your environment.

Distributed Shell (dsh) allows you to concurrently run commands on remote nodes. It requires very little setup, but it’s one of those hidden AIX commands that few administrators know about. As you may have already guessed, dsh comes with the CSM client filesets. Specifically, it comes from the csm.dsh fileset which creates a symbolic link – /usr/bin/dsh -> /opt/csm/bin/dsh

root@AIX / > lslpp -w /usr/bin/dsh
  File                                        Fileset               Type
  /usr/bin/dsh                                csm.dsh               Symlink

There are a few environmental variables that we need to setup first.

DSH_NODE_RSH=/usr/bin/ssh – By default, dsh uses rsh to execute remote commands. If you don’t know why using rsh is a bad idea, hand in your sysadmin card now. We need to tell dsh to execute all remote commands via ssh.
DCP_NODE_RCP=/usr/bin/scp – As above, dcp uses rcp to send files remotely, so lets change that to use scp
DSH_NODE_OPTS=-q – This passes the -q option to ssh to suppress the banner message from showing. This just makes the output much easier to read when working with a number of nodes.
DSH_NODE_LIST=/.dsh/model – You can feed a list of nodes to dsh a number of ways, but I find using text files the best way to manage it.

Most places have a typical dev/mod/prod setup which can consist of a number of servers within each environment. I have four node text files in my /.dsh directory.

-rw-rw-r--    1 root     system          577 Jan 11 10:19 all
-rw-r--r--    1 root     system           71 Feb 12 13:06 dev
-rw-r--r--    1 root     system           75 Oct 28 10:38 mod
-rw-r--r--    1 root     system          106 Jan 25 13:00 prod

This way I can run commands on all the nodes in a specific environment, or across all nodes. The easiest way to switch between lists is by using aliases.

In my .kshrc file

alias dshmod='export DSH_NODE_LIST=/.dsh/mod'
alias dshprod='export DSH_NODE_LIST=/.dsh/prod'
alias dshdev='export DSH_NODE_LIST=/.dsh/dev'
alias dshall='export DSH_NODE_LIST=/.dsh/all'

The contents of the node text files is each node on a single line.

root@AIX / > cat /.dsh/mod

That’s it for the configuration. You can run dsh -q to show the current configured variables. Look at the man page for dsh to see further configurable options and the IBM website[3].

Let’s see if it works.

root@AIX / > dshmod
root@AIX / > dsh date
SERVER1: Mon Feb 28 12:59:34 EETDT 2011
SERVER2: Mon Feb 28 12:53:58 EETDT 2011
SERVER3: Mon Feb 28 12:56:51 EETDT 2011

If you only wanted to run the command on a select number of hosts, you can use the -w switch.

root@AIX / > dsh -w SERVER1,SERVER3 "date"
SERVER1: Mon Feb 28 13:01:20 EETDT 2011
SERVER3: Mon Feb 28 12:55:43 EETDT 2011

We can make that output look a bit neater by piping through the dshbak command.

root@AIX / > dsh -w SERVER1,SERVER3 "date" | dshbak
Mon Feb 28 13:17:17 EETDT 2011
Mon Feb 28 13:11:41 EETDT 2011

Just like the dsh command, dcp allows you to send files concurrently to multiple hosts. dcp requires the same environmental setup as dsh. Seeing as that’s already setup, we can use it straight away.

root@AIX / > lmktemp test 1M
root@AIX / > dcp test /
root@AIX / > dsh ls -l /test
SERVER1: -rw-r--r--   1 root     system      1048576 Feb 28 13:32 /test
SERVER2: -rw-r--r--   1 root     system      1048576 Feb 28 13:32 /test
SERVER3: -rw-r--r--   1 root     system      1048576 Feb 28 13:32 /test

What I did above is created a 1M file called test, used dcp to send it to my hosts, then verified the file was on each node using dsh. You can find more about dcp from the man pages and the IBM website[4].

The dping command allows you to ping nodes in parallel (with the option of serially). Now, it doesn’t read the DSH_NODE_LIST variable like dsh/dcp, but you can pass the -f option and tell it to.

root@AIX / > dping -f /.dsh/mod
SERVER1: ping (alive)
SERVER2: ping (alive)
SERVER3: ping (alive)
SERVER4: noping (unreachable)

If you only want to ping a select few hosts, then you can use the -n switch.

root@AIX / > dping -n SERVER1,SERVER3
SERVER1: ping (alive)
SERVER3: ping (alive)

You can find more on dping by reading the man pages and on the IBM website[5]. Like dsh, you can also pipe the output of dping through dshbak.

If you have any questions on the above, use the comment section below.

[1] – http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=/com.ibm.cluster.csm16.cmds.doc/am7cm_csmcomm.html
[2] – http://www.ece.uci.edu/~chou/ssh-key.html
[3] – http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=/com.ibm.cluster.csm16.cmds.doc/am7cm_dsh.html
[4] – http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=/com.ibm.cluster.csm16.cmds.doc/am7cm_dcp.html
[5] – http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=/com.ibm.cluster.csm16.cmds.doc/am7cm_dping.html

5 thoughts on “CSM Commands (dsh, dping, dcp and dshbak) for AIX

Leave a Reply

Your email address will not be published. Required fields are marked *