Experimental Physics and
Industrial Control System

You don't often get email from newville at cars.uchicago.edu. Learn why this is important

Hi Erico,

 

Sorry for the trouble.  That does sound like a strange problem.   

 

How long are these int and char waveforms?   As you may recall, waveforms with more than 64k elements will not be automatically monitored by default in pyepics, and that would need to be enabled when creating the PV.  But if using character waveforms for “long strings”, that’s a pretty long string. 

Before triggering the bug, the PV gets monitor events just fine, and the waveforms are much shorter than the pyEPICS limit as well (864 elements for the integer waveform).

 

I guess one thing to try would be to take and widget system out of the picture.

In the email I describe tests I performed with pyEPICS directly and with Sequencer to narrow it down, and the issue was always happening. Michael's email suggested that the issue could be in the IOC, and my testing confirmed it.

Thank you for the suggestions!

 

--Matt

 

 

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Érico Nogueira Rolim via Tech-talk <tech-talk at aps.anl.gov>
Date: Thursday, April 4, 2024 at 3:28 PM
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: caPutLog breaks monitoring of PV when the same channel is used for putting

Hi!


We are investigating a bug where PyDM widgets used for waveform records
(both integer and char, for long strings) don't display the value that
was put into them correctly. I'm sending it here because it's unclear
what specifically is failing and where I'd even report a bug.


This seems to happen only when caPutLog is initialized in the IOC (and
keeps happening if it's turned off), both the traditional one and the
JSON version. This has happened both with an asyn record and with a
record with no device support module. It also seems relevant that puts
to these waveforms don't show up in the resulting log.


The interface starts up with the current value shown in the widget. If
the PV receives a write from somewhere else, the interface updates
itself normally. However, if the write is performed from the interface,
the bug starts happening and doesn't restore itself unless the interface
is restarted.


We have seen this manifest itself in two ways:

- the last value in the widget remains there even after the user presses
enter. The readback value shows the updated value, however, so we know
the write went through

- for the integer array case, we have seen the interface instead display
a random *floating point* value, usually just one element


Since we are using PyDM with pyEPICS, I figured I'd test pyEPICS
directly. If I simply create a PV object and add a callback to it, and
put to the PV from somewhere else, the callback is called. However, once
I do a single PV.put(), no more callbacks are called, and PV.value isn't
updated anymore (I have checked that auto_monitor is still True). Of
course, PV.get(use_monitor=False) gets me a new value in cache, but it
doesn't fix callbacks.


To try and narrow it down between pyEPICS and libCA itself, I tried
implementing a sequencer client to test things. It is attached, and its
output is: the original value of the PV, the new value of the PV that
it's going to try and write, the new value of the PV after it's been
processed (completely zeroed out). So, I'm not sure what conclusion to
draw... The fact that I'm using the event flag would indicate that the
IOC sent a monitor event, however it contained no information? And if
libCA got an event, why wouldn't it be forwarded to pyEPICS? Are my
conclusions correct?


For some reason I didn't manage to catch this happening in a network
capture, but I will attempt it again when able.


I saw a similar-ish issue in the caPutLog bugtracker [1], but it was
specifically about QSRV1.


Our versions of things:

In the IOCs and sequencer client:

- epics-base: 7.0.7

- caPutLog: R4.0

- Sequencer: R2-2-9


One of the PyDM interfaces (via conda):

- PyDM: 1.10.3

- pyEPICS: 3.5.0


Other PyDM interface report (via conda):

- PyDM: 1.22.0 or 1.19.1

- pyEPICS: 3.5.1 (from pip)


Cheers,

Érico


[1] https://urldefense.us/v3/__https://github.com/epics-modules/caPutLog/issues/22__;!!G_uCfscf7eWS!ejmmpa48B5Lg-cSSwalI96o2gTPgfS-fdi3OeNW5T6si95vrsnEEIouNzK6cet7e6zcOuYs5JytQ8Xs9sGnrzYO4c64$

Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente.

Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately.

References:

caPutLog breaks monitoring of PV when the same channel is used for putting Érico Nogueira Rolim via Tech-talk

Re: caPutLog breaks monitoring of PV when the same channel is used for putting Matthew Newville via Tech-talk

Navigate by Date:

Prev: RE: iocStats Restart IOC PV does not work Zhang, Tong via Tech-talk

Next: Alarm Handler Limitations Srinivas, Dhruv via Tech-talk

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  <2024> 

Navigate by Thread:

Prev: Re: caPutLog breaks monitoring of PV when the same channel is used for putting Matthew Newville via Tech-talk

Next: Alarm Handler Limitations Srinivas, Dhruv via Tech-talk

Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  <2024>