RE: Incompatibility of pyepics and pvaPy 4.0.0 on Windows

[Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>]

Hi Sinisa,

 

Ø  This is not quite the same, as in your test application you first import pvaccess, while in tomoscan this seems to happen later.

 

I tried changing the order of importing epics and pvaccess, but it worked with either order.

 

Ø  Also, in this code you are only using epics module.

 

But in tomoscan if I comment out the 2 files in __init__.py then that is also only using epics, it does not use pvaccess at all.  But tomoscan fails with only the epics module if I set PYEPICS_LIBCA to point to the pvaccess version of ca.dll.  But the test program does not fail even with PYEPICS_LIBCA set to the pvaccess version of ca.dll

 

Mark

 

 

 

From: Veseli, Sinisa <sveseli at anl.gov>
Sent: Thursday, September 9, 2021 3:03 PM
To: 'tech-talk at aps.anl.gov' <tech-talk at aps.anl.gov>; Mark Rivers <rivers at cars.uchicago.edu>
Cc: De Carlo, Francesco <decarlo at anl.gov>; Nikitin, Viktor <vnikitin at anl.gov>
Subject: Re: Incompatibility of pyepics and pvaPy 4.0.0 on Windows

 

Ho Mark,

 

This is not quite the same, as in your test application you first import pvaccess, while in tomoscan this seems to happen later. Also, in this code you are only using epics module.

 

Like I mentioned earlier, there were some threading-related changes in pvapy 4.0.0 that might be causing this.

 

 

Sinisa

 

 

--

Siniša Veseli

Scientific Software Engineering & Data Management

Advanced Photon Source

Argonne National Laboratory

sveseli at anl.gov

(630)252-9182

---

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>
Sent: Thursday, September 9, 2021 2:51 PM
To: 'tech-talk at aps.anl.gov' <tech-talk at aps.anl.gov>
Cc: De Carlo, Francesco <decarlo at anl.gov>; Veseli, Sinisa <sveseli at anl.gov>; Nikitin, Viktor <vnikitin at anl.gov>
Subject: RE: Incompatibility of pyepics and pvaPy 4.0.0 on Windows

 

I wrote a very simple test program which I would have expected to fail, since it implements the same logic as the startup error in tomoscan.

 

corvette:~/support/tomoscan/tomoscan>more test_ca.py

import pvaccess

import epics

import threading

import time

 

class test_ca():

    def __init__(self):

        thread = threading.Thread(target=self.test_put, args=(), daemon=True)

        thread.start()

 

    def test_put(self):

         pv = epics.PV('13BMDPG1:cam1:AcquireTime')

         while True:

            print("Setting PV to 0.1")

            pv.put(0.1)

            time.sleep(3)

 

However, I cannot make it fail, even by setting PYEPICS_LIBCA to point to the pvaccess version of ca.dll.

 

Mark

 

 

From: Mark Rivers
Sent: Thursday, September 9, 2021 2:05 PM
To: tech-talk at aps.anl.gov
Cc: 'Veseli, Sinisa' <sveseli at anl.gov>; Matt Newville <newville at cars.uchicago.edu>; Francesco De Carlo <decarlo at aps.anl.gov>; Nikitin, Viktor <vnikitin at anl.gov>
Subject: Incompatibility of pyepics and pvaPy 4.0.0 on Windows

 

Folks,

 

I believe there is an incompatibility of the Python pyepics and pvapy packages on Windows.  I have a Python application called tomoscan that uses both packages.

https://github.com/tomography/tomoscan

 

This works fine on Linux.  It used to work OK on Windows.  However, using the latest versions of pyepics (3.5.0) and pvapy (4.0.0) it crashes as follows on startup:

 

(tomoscan) J:\epics\support\tomoscan\iocBoot\iocTomoScan_13BM_PSO>python -i start_tomoscan.py

Exception in thread Thread-1:

Traceback (most recent call last):

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\threading.py", line 932, in _bootstrap_inner

    self.run()

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\threading.py", line 870, in run

    self._target(*self._args, **self._kwargs)

  File "\\corvette\people_rw\epics\support\tomoscan\tomoscan\tomoscan.py", line 240, in reset_watchdog

    self.epics_pvs['Watchdog'].put(5)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\pv.py", line 54, in wrapped

    return func(self, *args, **kwargs)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\pv.py", line 652, in put

    return ca.put(self.chid, value,

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 610, in wrapper

    return fcn(*args, **kwds)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 1717, in put

    poll()

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 547, in wrapper

    return fcn(*args, **kwds)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 923, in poll

    pend_event(evt)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 910, in pend_event

    ret = libca.ca_pend_event(timeout)

OSError: exception: access violation reading 0x0000000000000048

 

Note that it is crashing with an access violation in the ca_pend_event() in the C library.  This happens when epics.PV.put() is called in a different Python thread.

 

I suspect the problem is that the version of ca.dll that comes with pvapy is somehow incompatible with pyepics.

 

This suspicion comes from the fact that I can modify the tomoscan package so that it does not use pvaccess by commenting out the 2 lines shown below in __init__py:

 

corvette:~/support/tomoscan/tomoscan>more __init__.py

__version__ = '0.1'

 

from tomoscan.tomoscan import *

from tomoscan.tomoscan_pso import *

from tomoscan.tomoscan_step import *

#from tomoscan.tomoscan_stream_pso import *

from tomoscan.tomoscan_13bm_pso import *

from tomoscan.tomoscan_13bm_mcs import *

from tomoscan.tomoscan_2bm import *

from tomoscan.tomoscan_2bm_step import *

from tomoscan.tomoscan_6bm_step import *

from tomoscan.tomoscan_32id import *

from tomoscan.tomoscan_32id_step import *

#from tomoscan.tomoscan_stream_2bm import *

from tomoscan.tomoscan_7bm import *

 

.tomoscan_stream_pso and .tomoscan_stream_2bm are the only files that use pvaccess.

 

With those 2 lines commented out tomoscan starts fine:

(tomoscan) J:\epics\support\tomoscan\iocBoot\iocTomoScan_13BM_PSO>python -i start_tomoscan.py

>>> 

 

However, I can cause it to crash again by telling pyepics to use the version of ca.dll that is part of pvapy using the environment variable PYEPICS_LIBCA.

 

>set PYEPICS_LIBCA=C:\Users\rivers\Anaconda3\envs\tomoscan\Lib\site-packages\pvaccess\ca.dll

 

(tomoscan) J:\epics\support\tomoscan\iocBoot\iocTomoScan_13BM_PSO>python -i start_tomoscan.py

Exception in thread Thread-1:

Traceback (most recent call last):

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\threading.py", line 932, in _bootstrap_inner

    self.run()

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\threading.py", line 870, in run

    self._target(*self._args, **self._kwargs)

  File "\\corvette\people_rw\epics\support\tomoscan\tomoscan\tomoscan.py", line 240, in reset_watchdog

    self.epics_pvs['Watchdog'].put(5)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\pv.py", line 54, in wrapped

    return func(self, *args, **kwargs)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\pv.py", line 652, in put

    return ca.put(self.chid, value,

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 610, in wrapper

    return fcn(*args, **kwds)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 1717, in put

    poll()

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 547, in wrapper

    return fcn(*args, **kwds)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 923, in poll

    pend_event(evt)

  File "C:\Users\rivers\Anaconda3\envs\tomoscan\lib\site-packages\epics\ca.py", line 910, in pend_event

    ret = libca.ca_pend_event(timeout)

OSError: exception: access violation reading 0x0000000000000048

 

This is pretty good evidence that the problem is due to some problem with the version of ca.dll that comes with pvapy 4.0.0.

 

If I downgrade pvapy from 4.0.0 to the previous version, 3.1.0 it works fine, using the original __init__.py that enables pvaccess.  This is true whether or not PYEPICS_LIBCA is defined to point to the pvaccess version of ca.dll.  It seems like there is some problem with the ca.dll that comes with pvapy 4.0.0.

 

Mark