Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
<== Date ==> <== Thread ==>

Subject: Re: Increasing scan rate to 10 kHz
From: Evgeniy <tikh@triumf.ca>
Cc: EPICS Tech Talk <tech-talk@aps.anl.gov>
Date: Mon, 23 Dec 2013 15:46:22 -0800
In reply to my initial post Andrew wrote:

".. if the IOC's processing of all the records that have SCAN=.01 second takes longer than 0.01 second to execute, the scan rate will drop to 10Hz automatically."

And I really observed this behavior for compress record/circular buffer. Another slow processing is for subArray, when it fetches an element from waveform ~ 0.1 second (which I measure by EPICS client program). At the same time channel access CA write/read takes only ~0.01 sec.

The question is: if Mark got processing time for bi record 150 microseconds 20 years ago should I have 1.5 microseconds for the same record on 100 times faster CPU ? Or this arithmetic doesn't work? Is it necessary to configure "something" to make it working?

The same arithmetic question for scan. If its minimum period was 0.01 seconds 20 years ago should I expect minimum 0.0001 seconds nowadays?

Thanks,
Evgeniy







-------- Original Message --------
I just set up an areaDetector simDetector to run with a very small ROI and short acquire time and acquire period.  It is computing about 1200 new images per second.

This is a Linux IOC that is processing many records per image.  Here is the output of camonitor on the image counter PV:

13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.561426) 78635
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.561745) 78636
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.562320) 78637
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.562860) 78638
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.563468) 78639
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.563998) 78640
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.564548) 78641
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.565135) 78642
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.565706) 78643
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.566242) 78644
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.566817) 78645
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.567354) 78646
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.567846) 78647
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.568436) 78648
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.569005) 78649
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.569513) 78650
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.570096) 78651
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.570678) 78652
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.571235) 78653
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.571846) 78654
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.572373) 78655
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.572909) 78656
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.573487) 78657
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.574075) 78658
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.574565) 78659
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.575156) 78660
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.575704) 78661
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.576291) 78662
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.576888) 78663
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.577468) 78664

So you can see that it is only about 600-700 microseconds per callback, and this is by no means limited by the rate at which records can process.

Mark


-----Original Message-----
From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Mark Rivers
Sent: Wednesday, December 18, 2013 1:16 PM
To: 'Evgeniy'; Eric Norum
Cc: EPICS Tech Talk
Subject: RE: Increasing scan rate to 10 kHz

What EPICS record, and how did you measure that?  Something is very wrong.

Here is a paper with performance measurements on an ancient (~20 years ago) 68040 that shows only 150 microseconds was required for a bi record, including posting a CA monitor.

http://www.aps.anl.gov/epics/EpicsDocumentation/EpicsGeneral/epicsX5Farch-1.html#HEADING1-23

Mark

-----Original Message-----
From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Evgeniy
Sent: Wednesday, December 18, 2013 1:07 PM
To: Eric Norum
Cc: EPICS Tech Talk
Subject: Re: Increasing scan rate to 10 kHz

Processing of an EPICS record takes ~2 milliseconds on my SL5.3/DELL
2.27 GHz. It is unbelievably slow!
Is it possible to shorten this time without rebuilding Linux kernel?


-------- Original Message --------
On Nov 27, 2013, at 11:29 AM, Evgeniy <tikh@triumf.ca> wrote:

Thanks, Andrew and Emmanuel for your replies.

I am running Soft IOC on Dell server (64 bit) with eight  Intel Xeon CPUs, each  2.27GHz.
It seems that syntax for dbd file is correct. I found that minimum scan period for my system is 0.001 seconds. For shorter periods it falls down to 0.1 second, as Andrew mentioned.
The difference in one CPU usage is only 5% if I run records with scan 0.1 second or 0.001 second. It looks like it is limitation of EPICS, not hardware.
Or perhaps the kernel timer implementation has a resolution of 1 ms?
   What is the bottleneck for processing of a record on GHz CPUs?
Is it possible to improve it?

Evgeniy




References:
Increasing scan rate to 10 kHz Evgeniy
Re: Increasing scan rate to 10 kHz Andrew Johnson
Re: Increasing scan rate to 10 kHz Evgeniy
Re: Increasing scan rate to 10 kHz Eric Norum
Re: Increasing scan rate to 10 kHz Evgeniy
RE: Increasing scan rate to 10 kHz Mark Rivers
RE: Increasing scan rate to 10 kHz Mark Rivers

Navigate by Date:
Prev: RE: Mclennan PM600 motor controller Mark Rivers
Next: Sequencer Installation error Ganesh Jangir
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
Navigate by Thread:
Prev: RE: Increasing scan rate to 10 kHz Mark Rivers
Next: Re: Increasing scan rate to 10 kHz Andrew Johnson
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
ANJ, 20 Apr 2015 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·