Re: Increasing scan rate to 10 kHz

[Evgeniy <tikh@triumf.ca>]

In reply to my initial post Andrew wrote:

".. if the IOC's processing of all the records that have SCAN=.01 second 
takes longer than 0.01 second to execute, the scan rate will drop to 
10Hz automatically."
And I really observed this behavior for compress record/circular buffer. 
Another slow processing is for subArray, when it fetches an element from 
waveform ~ 0.1 second (which I measure by EPICS client program). At the 
same time channel access CA write/read takes only ~0.01 sec.
The question is: if Mark got processing time for bi record 150 
microseconds 20 years ago should I have 1.5 microseconds for the same 
record on 100 times faster CPU ?
Or this arithmetic doesn't work? Is it necessary to configure 
"something" to make it working?
The same arithmetic question for scan. If its minimum period was 0.01 
seconds 20 years ago should I expect minimum 0.0001 seconds nowadays?
Thanks,
Evgeniy







-------- Original Message --------
> I just set up an areaDetector simDetector to run with a very small ROI and short acquire time and acquire period.  It is computing about 1200 new images per second.
>
> This is a Linux IOC that is processing many records per image.  Here is the output of camonitor on the image counter PV:
>
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.561426) 78635
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.561745) 78636
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.562320) 78637
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.562860) 78638
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.563468) 78639
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.563998) 78640
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.564548) 78641
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.565135) 78642
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.565706) 78643
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.566242) 78644
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.566817) 78645
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.567354) 78646
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.567846) 78647
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.568436) 78648
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.569005) 78649
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.569513) 78650
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.570096) 78651
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.570678) 78652
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.571235) 78653
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.571846) 78654
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.572373) 78655
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.572909) 78656
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.573487) 78657
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.574075) 78658
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.574565) 78659
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.575156) 78660
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.575704) 78661
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.576291) 78662
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.576888) 78663
> 13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.577468) 78664
>
> So you can see that it is only about 600-700 microseconds per callback, and this is by no means limited by the rate at which records can process.
>
> Mark
>
>
> -----Original Message-----
> From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Mark Rivers
> Sent: Wednesday, December 18, 2013 1:16 PM
> To: 'Evgeniy'; Eric Norum
> Cc: EPICS Tech Talk
> Subject: RE: Increasing scan rate to 10 kHz
>
> What EPICS record, and how did you measure that?  Something is very wrong.
>
> Here is a paper with performance measurements on an ancient (~20 years ago) 68040 that shows only 150 microseconds was required for a bi record, including posting a CA monitor.
>
> http://www.aps.anl.gov/epics/EpicsDocumentation/EpicsGeneral/epicsX5Farch-1.html#HEADING1-23
>
> Mark
>
> -----Original Message-----
> From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Evgeniy
> Sent: Wednesday, December 18, 2013 1:07 PM
> To: Eric Norum
> Cc: EPICS Tech Talk
> Subject: Re: Increasing scan rate to 10 kHz
>
> Processing of an EPICS record takes ~2 milliseconds on my SL5.3/DELL
> 2.27 GHz. It is unbelievably slow!
> Is it possible to shorten this time without rebuilding Linux kernel?
>
>
> -------- Original Message --------
> > On Nov 27, 2013, at 11:29 AM, Evgeniy <tikh@triumf.ca> wrote:
> >
> > > Thanks, Andrew and Emmanuel for your replies.
> > >
> > > I am running Soft IOC on Dell server (64 bit) with eight  Intel Xeon CPUs, each  2.27GHz.
> > > It seems that syntax for dbd file is correct. I found that minimum scan period for my system is 0.001 seconds. For shorter periods it falls down to 0.1 second, as Andrew mentioned.
> > > The difference in one CPU usage is only 5% if I run records with scan 0.1 second or 0.001 second. It looks like it is limitation of EPICS, not hardware.
> > Or perhaps the kernel timer implementation has a resolution of 1 ms?
> > >    What is the bottleneck for processing of a record on GHz CPUs?
> > > Is it possible to improve it?
> > >
> > > Evgeniy