Experimental Physics and
Industrial Control System

"Johnson, Andrew N. via Tech-talk" <tech-talk at aps.anl.gov> · Tue, 30 Mar 2021 23:06:00 +0000

Hi Adam,

So there is no single right/wrong answer, there are many different factors that will affect what the best solution is for you.

Here at the APS we have a few large server-class Linux machines in a cluster that run many soft IOCs (83 on one VM, 103 on another for example). Our setup predates procServ so all our soft IOCs are running inside a GNU Screen session (so our engineers can connect to the iocsh of any IOC for live maintenance and debugging) which runs a Perl script (for automatically restarting it if the IOC dies) which runs the IOC application.

Using procServ provides an alternative approach to our GNU Screen sessions; it’s really useful to be able to login to an IOC's console to see what it’s actually doing, so I would recommend doing something like this and not running IOCs with no way to get to their console/shell. Some places use a separate terminal window on their PC’s console for each IOC to run in (instead of procServ), and for a small installation that might be the most efficient way to manage them, but once you have tens or hundreds of IOCs it won’t be.

Some other considerations: If all your devices are controlled by one IOC that makes it very easy to stop and start them all at once, but you can't keep some devices running while you’re fixing or adding functionality to others. If you have to restart the single IOC to fix communications problems with one device say, that will bring all of them down which might be a deal-breaker for some uses.

I don’t think you’ll have resource problems running 8 IOCs on a decent desktop machine that are just talking to some DMMs and lower-speed digitizers. The IOC code is actually quite frugal since it was written to run on the embedded VME systems that were common 30 years ago, which were tiny by today’s standards. You can scale it up for running AreaDetector and taking multi-GB images at tens of Hz though, and with that kind of setup you might need to run performance tests of various scenarios to work out how best to set things up.

HTH,

- Andrew

> On Mar 30, 2021, at 5:02 PM, Manoussakis, Adamandios via Tech-talk <tech-talk at aps.anl.gov> wrote:
> 
> Hey Dave,
>  
> Thanks for the information, for platform we will be using Debian flavors of linux as the host for the IOCs and so far have not encountered running multiple IOCs of the epics 3/7.  I was concerned with spawning out so many IOC instances if I did go with the design of one ioc one device (looking at say 30+ IOCs on one host vs combining down to ~5).  I can see the benefit of efficiency when records need to communicate between each other but in this case the digitizers/dmms do not have any dependencies on each other.  
>  
> Couple other thoughts
>  
> I was looking at starting to use procServ but was a little confused from reading about the documentation on its use.  What is the difference between spawning multiple IOCs through running your st.cmd for each and instead spawning them using procServ?  Is the IOC treated differently in linux when using procServ to start them?
>  
> Will I run into resource issues (on a highend desktop) spawning so many IOCs (~30) on one host (I did implement Ralphs script already to solve the multiple ioc ca issue)?
>  
> Are there concerns when running say 8 devices using streamdevice in a single IOC, I would assume there might be some delay when streamdevice has to make multiple calls in the same IOC?
>  
>  
>  
>  
>  
>  
>  
> From: D Willimoto <dwilliams at kiesbar.com> 
> Sent: Tuesday, March 30, 2021 1:52 PM
> To: Manoussakis, Adamandios <manoussakis1 at llnl.gov>
> Subject: Re: Question on Multiple Instruments in one IOC or Multiple IOCs for each
>  
> Hi Adam,
>  
> The platform is one dependency.  Some platforms have trouble instantiating multiple IOCs of certain EPICS versions.  It's difficult to do on a RaspPi for instance.
>  
> I use RaspiPi to connect to a vacuum and power supply system and I combine them into one IOC for this reason.
>  
> From a design perspective, there are two contrary forces.  First, encapsulation of device as an object and address.  
>  
> Second is the power of records to be actions and perform modifications.  For instance a record can be a calculation from two other records. If so, you want those two source records in the same IOC to eliminate constant communication lag.  This factor also exists with other database systems like Elastic, Splunk, etc.
>  
> There are other perspectives but these are the main practical ones in my view.
>  
> Best,
> Dave
>  
>  
> On Wed, Mar 31, 2021 at 5:44 AM Manoussakis, Adamandios via Tech-talk <tech-talk at aps.anl.gov> wrote:
> Hello,
>  
> From a design perspective, is it better to have one device per IOC or would it be more efficient to group same devices into one IOC?  I have multiple same instruments (digitizers, DMMs) and was thinking to create one IOC with say 8 Digitizers or 8 DMMs but wasn’t sure if this was the way to go about this.  If its better to go the other direction 1 device 1 IOC, is there a better way to spawn out the IOCs (I would guess a bash script with variables for any of the values that need to be changed eg. IP address for each)?
>  
> Thanks,
> Adam

-- 
Complexity comes for free, simplicity you have to work for.

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System