Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
<== Date ==> <== Thread ==>

Subject: RE: Odd problems with RHEL6 diskless machines
From: <tom.cobb@diamond.ac.uk>
To: <anj@aps.anl.gov>
Cc: tech-talk@aps.anl.gov
Date: Wed, 24 Jul 2013 15:57:02 +0000
I wrote a tiny C program that prints the time taken to do a usleep in a loop, and that also displays the behaviour, so I guess this has nothing to do with EPICS! I'll get onto red hat and see if they have any ideas...

Thanks,
Tom Cobb

> -----Original Message-----
> From: Andrew Johnson [mailto:anj@aps.anl.gov]
> Sent: 24 July 2013 16:33
> To: Cobb, Tom (DLSLtd,RAL,DIA)
> Cc: tech-talk@aps.anl.gov
> Subject: Re: Odd problems with RHEL6 diskless machines
> 
> Hi Tom,
> 
> On 07/24/2013 09:45 AM, tom.cobb@diamond.ac.uk wrote:
> > When running an IOC with just devIocStats in it, all of the scan
> > tasks seem to pause for up to 300 seconds (about half of the pauses
> > are between 299 and 301 seconds), then resume. During this time an
> > existing ssh connection to the server will continue to work, but
> > channel access monitors may time out. R3.14.8.2, R3.14.11 and
> > R3.14.12.3 IOCs are all affected. dmesg and /var/log/messages show
> > nothing suspicious, and cat /proc/sys/kernel/random/entropy_avail
> > returns 3968.
> 
> Interesting problem...
> 
> Does this have anything to do with devIocStats and the fact that you're
> presumably running the IOC as a real-time process?  I don't know the
> internals of that module running on Linux, but the old VxStats module
> on
> vxWorks has a lowest-priority idle thread that every so often busy-
> waits
> for a specific period of time to measure how much work the CPU does in
> the higher priority threads.  I could see a problem if devIocStats is
> trying to use the same technique on Linux because IIRC all real-time
> (SCHED_FIFO) threads still have a higher priority than regular threads,
> and this could cause the effect you're seeing.  Why it only seems to
> affect diskless systems is still a bit of a question, but I could think
> up some reasons for that.
> 
> Take a look at the internals of devIocStats, and if it has such an idle
> task try disabling it (it should be asking the OS instead of trying to
> measure it anyway).  Alternatively, see if not running the IOC as a
> real-time process stops the problem.
> 
> HTH,
> 
> - Andrew
> --
> Advertising may be described as the science of arresting the human
> intelligence long enough to get money from it. -- Stephen Leacock

-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 





References:
Odd problems with RHEL6 diskless machines tom.cobb
Re: Odd problems with RHEL6 diskless machines Andrew Johnson

Navigate by Date:
Prev: Re: Odd problems with RHEL6 diskless machines Andrew Johnson
Next: Re: Odd problems with RHEL6 diskless machines Ralph Lange
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
Navigate by Thread:
Prev: Re: Odd problems with RHEL6 diskless machines Andrew Johnson
Next: Re: Odd problems with RHEL6 diskless machines Ralph Lange
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
ANJ, 20 Apr 2015 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·