Re: Weird channel access problem within Python softioc process

["Cobb, Tom \(DLSLtd,RAL,LSCI\) via Tech-talk" <tech-talk at aps.anl.gov>]

Hi Paul,

Import the pyepics and softioc libraries.

Something to be aware of, the default in pythonSoftIOC is to use cothread as its concurrency framework. It uses lightweight co-routines instead of OS threads. This means that you must be careful not to block the thread you call softioc.iocInit() in, otherwise the callbacks from caput and myrecord.set() will end up not being processed. Functions like softioc.interactive_ioc() will co-operatively block, allowing cothread to service callbacks in the background.

I can suggest a couple of things to try:

• Either continue using cothread for concurrency, and use cothread.catools instead of pyepics for channel access. This will co-operatively block, allowing cothread to keep servicing the record callbacks
• Or don't use cothread for concurrency. The only other supported concurrency framework at the moment is asyncio, but you could use that example to make a background thread dispatcher that did the same kind of thing as AsyncioDispatcher, but with a single background OS thread

Please could you put responses on the pythonSoftIOC issue that Michael created to track this.

Thanks,

Tom

---

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Paul Richards via Tech-talk <tech-talk at aps.anl.gov>
Sent: 11 August 2021 01:05
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: Weird channel access problem within Python softioc process

 

All,

 

At Keck, we have been using the EPICS Python softioc with great success.  The recent 3.1 release has made it even more popular and easier to use.

 

That said, I have found a very odd behavior that I can replicate, relating to the consumption of keywords inside the same process that runs the softioc.  Sometimes it makes sense to do it this way, instead of finding a reference to the channel in the IOC and manipulating that.

 

A simplified example of the problem:

1. Import the pyepics and softioc libraries.
2. Start a softioc with a channel (call it ‘prefix:long’ in my example).
3. Start a thread that periodically updates the channel with a new value.
4. Start another thread that tries to access this channel, either with epics.PV() or epics.caget().
5. Channel access in thread 2 always times out, forever.

 

Today I found a “solution” to this that does not make sense: if I start the softioc (the call to softioc.iotInit()) in the first thread, and not in the main process, everything works as expected.

 

Our engineers are unable to explain this behavior.  I hope there’s someone here on the list that might know why!

 

 

Example code that demonstrates the failure:

--------------------

 

import os

import time

import threading

 

os.environ['EPICS_CA_SERVER_PORT'] = '5100'

os.environ['EPICS_CA_ADDR_LIST'] = 'localhost:5100'

os.environ['EPICS_CA_AUTO_ADDR_LIST'] = 'NO'

 

try:

    import epicscorelibs.path.pyepics

except ImportError:

    pass

import epics

 

from softioc import softioc, builder

 

# Create a long record for this demo

long = builder.longOut('prefix:long', initial_value=1)

builder.LoadDatabase()

softioc.iocInit()

 

def thread1():

    while True:

        val = long.get()

        long.set(val + 1)

        print(f'thread1: {val}')

        time.sleep(1)

 

def thread2():

    pv = epics.PV('prefix:long')

 

    while True:

        val = pv.get()

        print(f'thread2: {val}')

        time.sleep(1)

 

t1 = threading.Thread(target=thread1, daemon=True)

t2 = threading.Thread(target=thread2, daemon=True)

 

t1.start()

t2.start()

 

time.sleep(10)

 

 

 

Example that works; note the only change is the movement of the .iocInit() line down into thread1:

--------------------

 

import os

import time

import threading

 

os.environ['EPICS_CA_SERVER_PORT'] = '5100'

os.environ['EPICS_CA_ADDR_LIST'] = 'localhost:5100'

os.environ['EPICS_CA_AUTO_ADDR_LIST'] = 'NO'

 

try:

    import epicscorelibs.path.pyepics

except ImportError:

    pass

import epics

 

from softioc import softioc, builder

 

# Create a long record for this demo

long = builder.longOut('prefix:long', initial_value=1)

builder.LoadDatabase()

 

def thread1():

    softioc.iocInit()

 

    while True:

        val = long.get()

        long.set(val + 1)

        print(f'thread1: {val}')

        time.sleep(1)

 

def thread2():

   pv = epics.PV('prefix:long')

 

    while True:

        val = pv.get()

        print(f'thread2: {val}')

        time.sleep(1)

 

t1 = threading.Thread(target=thread1, daemon=True)

t2 = threading.Thread(target=thread2, daemon=True)

 

t1.start()

t2.start()

 

time.sleep(10)

 

 

The output when it fails:

 

$ python3 ./demoFailingDaemon.py

Starting iocInit

############################################################################

## EPICS 7.0.6.0

## Rev. 7.0.6.99.1.0

############################################################################

iocRun: All initialization complete

thread1: 1

thread1: 2

thread1: 3

thread2: None

thread1: 4

thread1: 5

thread2: None

thread1: 6

thread1: 7

thread1: 8

thread2: None

thread1: 9

thread1: 10

thread1: 11

 

 

The output when it works:

 

$ python3 ./demoWorkingDaemon.py

Starting iocInit

############################################################################

## EPICS 7.0.6.0

## Rev. 7.0.6.99.1.0

############################################################################

iocRun: All initialization complete

thread1: 1

thread2: 2

thread1: 2

thread2: 3

thread1: 3

thread2: 4

thread1: 4

thread2: 5

thread1: 5

thread2: 6

thread1: 6

thread2: 7

thread1: 7

thread2: 8

thread1: 8

thread2: 9

thread1: 9

thread2: 10

thread1: 10

thread2: 11

 

 

 

 

 

Paul

 

 

Paul Richards Software Engineer W. M. Keck Observatory 65-1120 Mamalahoa Hwy. Kamuela, Hawai'i 96743

Main: (808) 885-7887 Direct: (808) 881-3537 Fax: (808) 885-3535

 

 

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom