Experimental Physics and
Industrial Control System

<[email protected]> · Wed, 24 Jul 2013 15:57:02 +0000

I wrote a tiny C program that prints the time taken to do a usleep in a loop, and that also displays the behaviour, so I guess this has nothing to do with EPICS! I'll get onto red hat and see if they have any ideas...

Thanks,
Tom Cobb

> -----Original Message-----
> From: Andrew Johnson [mailto:[email protected]]
> Sent: 24 July 2013 16:33
> To: Cobb, Tom (DLSLtd,RAL,DIA)
> Cc: [email protected]
> Subject: Re: Odd problems with RHEL6 diskless machines
> 
> Hi Tom,
> 
> On 07/24/2013 09:45 AM, [email protected] wrote:
> > When running an IOC with just devIocStats in it, all of the scan
> > tasks seem to pause for up to 300 seconds (about half of the pauses
> > are between 299 and 301 seconds), then resume. During this time an
> > existing ssh connection to the server will continue to work, but
> > channel access monitors may time out. R3.14.8.2, R3.14.11 and
> > R3.14.12.3 IOCs are all affected. dmesg and /var/log/messages show
> > nothing suspicious, and cat /proc/sys/kernel/random/entropy_avail
> > returns 3968.
> 
> Interesting problem...
> 
> Does this have anything to do with devIocStats and the fact that you're
> presumably running the IOC as a real-time process?  I don't know the
> internals of that module running on Linux, but the old VxStats module
> on
> vxWorks has a lowest-priority idle thread that every so often busy-
> waits
> for a specific period of time to measure how much work the CPU does in
> the higher priority threads.  I could see a problem if devIocStats is
> trying to use the same technique on Linux because IIRC all real-time
> (SCHED_FIFO) threads still have a higher priority than regular threads,
> and this could cause the effect you're seeing.  Why it only seems to
> affect diskless systems is still a bit of a question, but I could think
> up some reasons for that.
> 
> Take a look at the internals of devIocStats, and if it has such an idle
> task try disabling it (it should be asking the OS instead of trying to
> measure it anyway).  Alternatively, see if not running the IOC as a
> real-time process stops the problem.
> 
> HTH,
> 
> - Andrew
> --
> Advertising may be described as the science of arresting the human
> intelligence long enough to get money from it. -- Stephen Leacock

-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System