Experimental Physics and
Industrial Control System

Michael Davidsaver via Tech-talk <tech-talk at aps.anl.gov> · Fri, 17 Jul 2020 11:49:52 -0700

On 7/17/20 10:39 AM, Matt Newville wrote:
> 
> 
> On Fri, Jul 17, 2020 at 11:56 AM Michael Davidsaver via Tech-talk <tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>> wrote:
> 
>     On 7/17/20 9:33 AM, Michael Davidsaver wrote:
>     > The python subprocess module shouldn't let this happen though since it does
>     > fork() and then execv(), and the situation which I describe is only possible
>     > if fork() is not followed by execv().
> 
>     Oh wait.  CAProcess (which I've not seen previously) is using
>     the multiprocessing module, which can do fork() without exec().
>     In fact, this is the default on *NIX.  So this is likely or
>     problem.  I think you'll need to use the 'spawn' or 'forkserver'
>     method.
> 
> 
> Well, CAProcess deletes the CA context and clears any cached CA data before the forked process runs so that the process will create and use its own CA context.

Unfortunately, this is not sufficient to reset all of the mutable shared state.
Like a great many C libraries, libca and libCom have not been written to remain
safely usable from a fork()'d child process.

> The problem actually reported happens when trying to create 25,000 unused CAProcesses, none of which did any real CA work. 

This is one way to provoke race conditions.

> But, sure, creating many CAProcesses from a CAThread to do work that might want to connect (each with their own CA context) to PVs might not be the wisest approach.  One could have separate worker processes running, ready to react to messages (perhaps from CA) to start some long-running work.   

There are all kinds of ways to scale up a python script.  I'm only pointing out
one way not to: libca + python multiprocessing with the 'fork' method.

I can't say for certain whether the 'forkserver' method would work, it depends
on when libca is loaded.  I'm certain that the 'spawn' method will.  Of
course this adds complexity.  (part of the reason I personally avoid using
the multiprocessing module)

> What I find most confusing is the fact that the program creates a CAThread from pycaspy write().  Some value in a python program changed and you want to send that to the outside world via CA (and are using pycaspy because you were too lazy to create a PV in a softIoc or this is all too dynamic to capture in a softIoc?) *and* you want to notify another thread in the same program that the value changed, and you decide that CA is the right way to send that message.   For communicating within a single process, you don't need CA at all.  

I haven't gone through the whole github ticket in detail, and don't quite follow
you here.  The situation as I understand it doesn't seem limited to the example
given in the initial report.

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System