EPICS Home EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

Format page
for printing


Search Tech-talk
1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <2022>  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <2022>  2023  2024 
<== Date ==> <== Thread ==>
Subject: Re: odd ASYN behavior
From: Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>
To: "Runchey, John" <jrunchey at anl.gov>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Date: Fri, 18 Mar 2022 21:10:56 +0000
The problem has been solved.

The problem was that the EPICS callback queue was overflowing.  John was getting a lot of these errors:

callbackRequest: cbLow ring buffer full

By adding this command just before iocInit() the problem was fixed:

callbackSetQueueSize(10000)

This is normally not needed if the IOC is controlling a single TetrAMM.   However, John is controlling several TetrAMM devices from the same IOC, and that led to more callbacks than the default queue size could handle.

Mark


From: Runchey, John <jrunchey at anl.gov>
Sent: Friday, March 18, 2022 1:39 PM
To: Mark Rivers <rivers at cars.uchicago.edu>; tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: Re: odd ASYN behavior
 
Hi Mark,

  This is with ASYN 4.42, epics base 7.06, and quadem 9.4, built on RHEL 8

The weird thing is, this is new behavior - my prior build with epics base 7.04, asyn 4.40, build on RHEL 7, and the same quadem doesn't exhibit this.

  What I really​ don't get is how the problem goes away after I manually do a caput to the .PROC for the _RBV record.
 
  I'll enable all the debugging and see what I can glean from that.

John
From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Friday, March 18, 2022 1:16 PM
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>; Runchey, John <jrunchey at anl.gov>
Subject: Re: odd ASYN behavior
 
Hi John,

I just tested with my TetrAmm and I cannot reproduce the behavior you are seeing.  As soon as I change AcquireMode then AcquireMode_RBV also changes.  Actually AcquireMode_RBV changes about 10-20 microseconds before AcquireMode.  Here is the output from camonitor:

corvette:~>camonitor 13IDA:QE1:AcquireMode 13IDA:QE1:AcquireMode_RBV
13IDA:QE1:AcquireMode          2022-02-01 10:07:24.187031 Continuous WRITE INVALID
13IDA:QE1:AcquireMode_RBV      2022-02-01 10:07:24.187028 Continuous
13IDA:QE1:AcquireMode_RBV      2022-03-18 13:06:48.532124 Single
13IDA:QE1:AcquireMode          2022-03-18 13:06:48.582639 Single
13IDA:QE1:AcquireMode_RBV      2022-03-18 13:06:53.401958 Multiple
13IDA:QE1:AcquireMode          2022-03-18 13:06:53.401969 Multiple
13IDA:QE1:AcquireMode_RBV      2022-03-18 13:06:58.066979 Continuous
13IDA:QE1:AcquireMode          2022-03-18 13:06:58.066999 Continuous
13IDA:QE1:AcquireMode_RBV      2022-03-18 13:07:12.250304 Single
13IDA:QE1:AcquireMode          2022-03-18 13:07:12.278290 Single
13IDA:QE1:AcquireMode_RBV      2022-03-18 13:07:22.147686 Multiple
13IDA:QE1:AcquireMode          2022-03-18 13:07:22.147697 Multiple
13IDA:QE1:AcquireMode_RBV      2022-03-18 13:07:27.644870 Continuous
13IDA:QE1:AcquireMode          2022-03-18 13:07:27.644886 Continuous

Please send a screen shot of your quadEM.adl screen when you have the issue and we can see if there is some difference in your settings from mine that could be causing the problem.

Mark

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Friday, March 18, 2022 12:45 PM
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>; Runchey, John <jrunchey at anl.gov>
Subject: Re: odd ASYN behavior
 
HI John,

I just looked at the code in drvQuadEM::writeInt32 and drvTetrAMM::setAcquireMode and it looks OK.  It sets the new value in the parameter library at the beginning and unconditionally does callParamCallbacks at the end.  That should result in the camonitor value being correct.

I am on vacation but will test this when I return.  Meanwhile you could enable all of the asynTrace debugging to see what that shows.

Mark

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Runchey, John via Tech-talk <tech-talk at aps.anl.gov>
Sent: Monday, March 14, 2022 11:14 AM
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: odd ASYN behavior
 
I'm working with a new build of QUADEM for the tetrAMM and seeing a weird asyn issue.

Summary: When writing to the AcquireMode PV, the associated _RBV PV is 'one behind' in reporting the new value, consistently. But if I manually do a caput to the _RBV.PROC field and force it to process, the readback is then consistently correct.

camonitor of the issue:
S28IDFE-XBPM:CM3ds:AcquireMode 03/11/22 09:00:39.158459244 Single
S28IDFE-XBPM:CM3ds:AcquireMode_RBV 03/11/22 09:00:39.158436082 Multiple

S28IDFE-XBPM:CM3ds:AcquireMode_RBV 03/11/22 09:01:23.572226245 Single
S28IDFE-XBPM:CM3ds:AcquireMode 03/11/22 09:01:23.572245721 Continuous

S28IDFE-XBPM:CM3ds:AcquireMode_RBV 03/11/22 09:01:34.876944669 Continuous
S28IDFE-XBPM:CM3ds:AcquireMode 03/11/22 09:01:34.876973666 Multiple

after caput AcquireMode_RBV.PROC 1:
S28IDFE-XBPM:CM3ds:AcquireMode_RBV 03/11/22 09:06:33.097901171 Single
// caput caused _RBV to update to the (correct) value of Continuous
S28IDFE-XBPM:CM3ds:AcquireMode_RBV 03/11/22 09:06:42.683848695 Continuous
S28IDFE-XBPM:CM3ds:AcquireMode 03/11/22 09:06:42.683874661 Continuous

S28IDFE-XBPM:CM3ds:AcquireMode_RBV 03/11/22 09:06:44.284941724 Multiple
S28IDFE-XBPM:CM3ds:AcquireMode 03/11/22 09:06:44.284974097 Multiple

S28IDFE-XBPM:CM3ds:AcquireMode_RBV 03/11/22 09:06:45.876836770 Single
S28IDFE-XBPM:CM3ds:AcquireMode 03/11/22 09:06:45.876854823 Single

This is the first time I've built this on a RHEL 8 machine for what that's worth.

Any thoughts?
Thanks,
John
Replies:
Re: odd ASYN behavior Ralph Lange via Tech-talk
References:
odd ASYN behavior Runchey, John via Tech-talk
Re: odd ASYN behavior Mark Rivers via Tech-talk
Re: odd ASYN behavior Mark Rivers via Tech-talk
Re: odd ASYN behavior Runchey, John via Tech-talk
Navigate by Date:
Prev: Re: StripTool license file Thomas, Patrick via Tech-talk
Next: Re: odd ASYN behavior Ralph Lange via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <2022>  2023  2024 
Navigate by Thread:
Prev: Re: odd ASYN behavior Runchey, John via Tech-talk
Next: Re: odd ASYN behavior Ralph Lange via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <2022>  2023  2024 
ANJ, 14 Sep 2022 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·