| 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 <2020> 2021 2022 2023 2024 | Index | 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 <2020> 2021 2022 2023 2024 |
| <== Date ==> | <== Thread ==> |
|---|
| Subject: | RE: Problems with scaler record and base 7.0.3.1 |
| From: | Mark Rivers via Core-talk <core-talk at aps.anl.gov> |
| To: | "'Mooney, Tim M.'" <mooney at anl.gov>, "'Johnson, Andrew N.'" <anj at anl.gov> |
| Cc: | "core-talk at aps.anl.gov" <core-talk at aps.anl.gov>, Joanne Stubbs <stubbs at cars.uchicago.edu>, "'tech-talk at aps.anl.gov'" <tech-talk at aps.anl.gov>, Dongzhou Zhang <dzzhang at cars.uchicago.edu>, Peter Eng <eng at cars.uchicago.edu> |
| Date: | Sat, 1 Feb 2020 15:04:55 +0000 |
|
I just rebuilt everything with 7.0.3, rather than 7.0.3.1. That fixed the problem. I even tested 2 scalers in the same IOC, a Joerger and an SIS3820, both running at
60 Hz at the same time. It worked fine. Something is broken in 7.0.3.1. You never find these problems until you test on a bunch of real-world IOCs
J Mark From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of
Mark Rivers via Tech-talk Tim and Andrew, We have discovered a serious problem with the scaler record running under the following configuration. Base 7.0.3.1 vxWorks 6.9.4.1 std master mca master vme master asyn master The problem is the following: -
If the scaler display update rate (.RATE field) is 59 (Hz) or less it works fine. The cbHigh task is using less than 2% of the CPU as shown by spy. -
If .RATE=60 then the following happens:
o
The cbHigh task uses >50% of the CPU
o
The timerTask uses >20% of the CPU
o
There is 0% IDLE time in the CPU
o
The crate becomes unresponsive and loses CA connections
o
Typing ‘dbpf “13LAB:scaler1.RATE”,”59”’ fixes the problem immediately, the crate becomes responsive and CA connections are restorerd. -
We observe this problem on both the Joerger scaler and the SIS3820 scaler. -
The problem also happens if Autocount is enabled and .RAT1=60. We are certain that this is a new problem, because we have autosave files from 2 vxWorks IOCs going all the way back to 2014 and .RATE was always 60 for those scalers. This includes the last run in December 2019.
We first observed the failures yesterday (the first day of the run, naturally!) Nothing has changed in the scaler record, or in the device support for the Joerger scaler and the SIS3820 scaler since 2018. We have been running the master branch of these all the time, so the fact that it was working all of 2019 means
the problem is unlikely to be in those modules. The Joerger scaler does not use asyn, so the problem cannot be in asyn. The main thing that has changed in this run is that we have updated from base 7.0.3 to 7.0.3.1. Is there anything that could have changed in base 7.0.3.1 that might cause this behavior? We can work around the problem for now by setting .RATE less than 60, but others are likely to be hit by the same problem. Thanks, Mark |