EPICS RE: Increasing scan rate to 10 kHz

Experimental Physics and Industrial Control System

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 <2013> 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024	Index	1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 <2013> 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024
<== Date ==>		<== Thread ==>

Subject:	RE: Increasing scan rate to 10 kHz
From:	Mark Rivers <[email protected]>
To:	"'Evgeniy'" <[email protected]>, Eric Norum <[email protected]>
Cc:	EPICS Tech Talk <[email protected]>
Date:	Wed, 18 Dec 2013 19:25:32 +0000

I just set up an areaDetector simDetector to run with a very small ROI and short acquire time and acquire period.  It is computing about 1200 new images per second.

This is a Linux IOC that is processing many records per image.  Here is the output of camonitor on the image counter PV:

13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.561426) 78635  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.561745) 78636  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.562320) 78637  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.562860) 78638  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.563468) 78639  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.563998) 78640  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.564548) 78641  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.565135) 78642  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.565706) 78643  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.566242) 78644  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.566817) 78645  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.567354) 78646  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.567846) 78647  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.568436) 78648  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.569005) 78649  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.569513) 78650  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.570096) 78651  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.570678) 78652  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.571235) 78653  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.571846) 78654  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.572373) 78655  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.572909) 78656  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.573487) 78657  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.574075) 78658  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.574565) 78659  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.575156) 78660  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.575704) 78661  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.576291) 78662  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.576888) 78663  
13SIM1:cam1:ArrayCounter_RBV   (2013-12-18 13:19:53.577468) 78664  

So you can see that it is only about 600-700 microseconds per callback, and this is by no means limited by the rate at which records can process.

Mark


-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Mark Rivers
Sent: Wednesday, December 18, 2013 1:16 PM
To: 'Evgeniy'; Eric Norum
Cc: EPICS Tech Talk
Subject: RE: Increasing scan rate to 10 kHz

What EPICS record, and how did you measure that?  Something is very wrong.

Here is a paper with performance measurements on an ancient (~20 years ago) 68040 that shows only 150 microseconds was required for a bi record, including posting a CA monitor.

http://www.aps.anl.gov/epics/EpicsDocumentation/EpicsGeneral/epicsX5Farch-1.html#HEADING1-23

Mark

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Evgeniy
Sent: Wednesday, December 18, 2013 1:07 PM
To: Eric Norum
Cc: EPICS Tech Talk
Subject: Re: Increasing scan rate to 10 kHz

Processing of an EPICS record takes ~2 milliseconds on my SL5.3/DELL 
2.27 GHz. It is unbelievably slow!
Is it possible to shorten this time without rebuilding Linux kernel?


-------- Original Message --------
> On Nov 27, 2013, at 11:29 AM, Evgeniy <[email protected]> wrote:
>
>> Thanks, Andrew and Emmanuel for your replies.
>>
>> I am running Soft IOC on Dell server (64 bit) with eight  Intel Xeon CPUs, each  2.27GHz.
>> It seems that syntax for dbd file is correct. I found that minimum scan period for my system is 0.001 seconds. For shorter periods it falls down to 0.1 second, as Andrew mentioned.
>> The difference in one CPU usage is only 5% if I run records with scan 0.1 second or 0.001 second. It looks like it is limitation of EPICS, not hardware.
> Or perhaps the kernel timer implementation has a resolution of 1 ms?
>>   What is the bottleneck for processing of a record on GHz CPUs?
>> Is it possible to improve it?
>>
>> Evgeniy

Replies:: Re: Increasing scan rate to 10 kHz Evgeniy

References:: Increasing scan rate to 10 kHz Evgeniy; Re: Increasing scan rate to 10 kHz Andrew Johnson; Re: Increasing scan rate to 10 kHz Evgeniy; Re: Increasing scan rate to 10 kHz Eric Norum; Re: Increasing scan rate to 10 kHz Evgeniy; RE: Increasing scan rate to 10 kHz Mark Rivers

Navigate by Date:: Prev: RE: Increasing scan rate to 10 kHz Mark Rivers; Next: Re: Increasing scan rate to 10 kHz Andrew Johnson; Index: 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 <2013> 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024
Navigate by Thread:: Prev: RE: Increasing scan rate to 10 kHz Mark Rivers; Next: Re: Increasing scan rate to 10 kHz Evgeniy; Index: 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 <2013> 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024