Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
<== Date ==> <== Thread ==>

Subject: Re: Odd problems with RHEL6 diskless machines
From: Andrew Johnson <anj@aps.anl.gov>
To: tom.cobb@diamond.ac.uk
Cc: tech-talk@aps.anl.gov
Date: Wed, 24 Jul 2013 10:33:15 -0500
Hi Tom,

On 07/24/2013 09:45 AM, tom.cobb@diamond.ac.uk wrote:
When running an IOC with just devIocStats in it, all of the scan
tasks seem to pause for up to 300 seconds (about half of the pauses
are between 299 and 301 seconds), then resume. During this time an
existing ssh connection to the server will continue to work, but
channel access monitors may time out. R3.14.8.2, R3.14.11 and
R3.14.12.3 IOCs are all affected. dmesg and /var/log/messages show
nothing suspicious, and cat /proc/sys/kernel/random/entropy_avail
returns 3968.

Interesting problem...

Does this have anything to do with devIocStats and the fact that you're presumably running the IOC as a real-time process? I don't know the internals of that module running on Linux, but the old VxStats module on vxWorks has a lowest-priority idle thread that every so often busy-waits for a specific period of time to measure how much work the CPU does in the higher priority threads. I could see a problem if devIocStats is trying to use the same technique on Linux because IIRC all real-time (SCHED_FIFO) threads still have a higher priority than regular threads, and this could cause the effect you're seeing. Why it only seems to affect diskless systems is still a bit of a question, but I could think up some reasons for that.

Take a look at the internals of devIocStats, and if it has such an idle task try disabling it (it should be asking the OS instead of trying to measure it anyway). Alternatively, see if not running the IOC as a real-time process stops the problem.

HTH,

- Andrew
--
Advertising may be described as the science of arresting the human
intelligence long enough to get money from it. -- Stephen Leacock

Replies:
RE: Odd problems with RHEL6 diskless machines tom.cobb
Re: Odd problems with RHEL6 diskless machines Ralph Lange
References:
Odd problems with RHEL6 diskless machines tom.cobb

Navigate by Date:
Prev: RE: fsmRecord Initial State Steiner, Mathias
Next: RE: Odd problems with RHEL6 diskless machines tom.cobb
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
Navigate by Thread:
Prev: Odd problems with RHEL6 diskless machines tom.cobb
Next: RE: Odd problems with RHEL6 diskless machines tom.cobb
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
ANJ, 20 Apr 2015 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·