Re: Automatic IOC configuration loading

[Bruce Hill <bhill@slac.stanford.edu>]

Hi Bruno,
We use a system similar to your solution 1.

We have over 100 diskless linux x86 based host systems
that all boot using DHCP and PXE.     In the diskless images,
(we use a slightly different image for each of our 6
experimental areas), we add an init.d file.

% cat /etc/init.d/ioc
#!/bin/bash

if [ -x /reg/d/iocCommon/hosts/$HOSTNAME/startup.cmd ]; then
  /reg/d/iocCommon/hosts/$HOSTNAME/startup.cmd
  exit 0
fi

There are also 3 soft links under /etc/rc.d to control when /etc/init.d/ioc runs:
% ll /etc/rc.d/rc*/*ioc*
lrwxrwxrwx 1 root root 13 Dec 10  2010 /etc/rc.d/rc3.d/S99ioc -> ../init.d/ioc*
lrwxrwxrwx 1 root root 13 Dec 10  2010 /etc/rc.d/rc4.d/S99ioc -> ../init.d/ioc*
lrwxrwxrwx 1 root root 13 Dec 10  2010 /etc/rc.d/rc5.d/S99ioc -> ../init.d/ioc*


Each soft ioc host machine NFS mounts the /reg/d/iocCommon filesystem,
which is also available on all our development systems.

Thus, when each ioc host machine boots, it looks for and runs
it's ioc host startup script if found.   The host script is run as root,
so that's where we put any special module or driver initialization.

As we have 1 to N soft ioc's for each host, the host startup.cmd
files also typically contain one or more lines like these:

# Run the soft IOC startup.cmd scripts
/reg/d/iocCommon/sioc/ioc-las-fiberOven/startup.cmd
/reg/d/iocCommon/sioc/ioc-las-hist1/startup.cmd

The sioc startup.cmd files run some common environment setup
scripts, change to the appropriate ioc release directory, and use the
linux runuser command to run the ioc st.cmd file as the appropriate
ioc userid using procServ.

When we want to update a soft ioc to a different version, we just edit
it's startup.cmd file to cd to the new directory, kill the prior procServ,
and use sudo to re-run the ioc startup.cmd file.

We use a similar scheme for our VME hard ioc's, but they boot via
BOOTP and TFTP, and use a TFTP startup script to search for and
run the hard ioc's st.cmd file, again found via NFS and the ioc's hostname.

This has worked well for us, making it relatively easy to manage over
500 soft ioc's running on over 100 host machines.    The downside of this
approach is that if your DHCP/NFS/PXE/BOOTP/TFTP server or servers are
down, nothing boots.    As our group supports end stations, and the central
server's are needed to support the control room computers as well, this
has been an acceptable tradeoff for us.

Hope this helps,
- Bruce


On 03/06/2013 11:55 AM, Bruno Seiva Martins wrote:

Hi,

I've been wondering: is there a standard (or common) way of loading IOC configuration data upon power up? If one have a site with, say, 100 computers (x86), each running an IOC, then what would be the best way to load their configuration? I'm thinking basically in two scenarios where this would be necessary:

1. During testing, when configuration (and/or binaries) might change often

2. Diskless systems, which don't have local storage and must load their data when they boot

Suppose there is a DHCP server which gives them their IP addresses. For the diskless systems, I think something like PXE to boot a Operating System would be necessary. 

Then, once booted, I thought about some solutions:

1. Mount remote folders residing in a server, via NFS. Maybe the name of the folder would contain the IP address of the interface, so you could have, for instance, a file tree in the server that would look like:

nfs/

  general/  --> for all IOC's (binaries, maybe)

  10.0.0.2/ --> for each IOC (st.cmd, .db files, etc)

  10.0.0.3/

  10.0.0.120/

Would this solution overload the network? This is the simplest solution, I think.

2. Use the same scheme above, but copy all the necessary data locally, then umount the NFS folder. In a system with local storage, this copy could happen conditionally, only if the local files differ from the remote ones. There would be a lot of shell scripts to be written.

3. Write scripts that run in the server and use scp to copy the files to the computers. The server would have to keep track of the versions of the files in the systems with disk to avoid unnecessary copies.

4. Use git. Since I'm already checking for different versions, why not go full version control? In this situation, there would be a "general" git repository and then one repository for each IP. Once booted, a computer would simply do a git pull (or a git clone if it is a diskless system) and then it would have all the latest files locally.

Of course, this is a naïve approach. So, before going into the trouble of implementing and testing these, I came here to see if any of you could point out a better way of doing this, or if you have advice about any of those four "solutions".

Thanks in advance!

Best regards,

Bruno Martins

-- 
Bruce Hill
Member Technical Staff
SLAC National Accelerator Lab
2575 Sand Hill Road M/S 10
Menlo Park, CA  94025