Experimental Physics and
Industrial Control System

Is it possible that some device that should be asynchronous is actually synchronous?  If this is an asyn device then perhaps it did not set ASYN_CANBLOCK when it should have.  If that happens then that device will slow down the scan-1 thread.

asynReport

will show you whether you have any asyn drivers with canBlock:Yes

Mark


-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Konrad, Martin via Core-talk
Sent: Wednesday, April 3, 2019 11:59 AM
To: EPICS Core Talk <[email protected]>
Subject: Excessive scan times for periodic scans

Hi,
On one of our IOCs I'm seeing the following warning:

dbScan warning from '1 second' scan thread:
        Scan processing averages 1.405 seconds (1.028 .. 2.559).
        Over-runs have now happened 10 times in a row.
        To fix this, move some records to a slower scan rate.

The IOC is talking to 132 devices using StreamDevice. "scanppl" reports
11408 records on the '1 second' scan list. The VM hosting the IOC is running happily with 20-30% CPU load on each of its 8 cores. "top -H"
reports 10-15% for the "scan-1" thread.

Naively, I would expect the "scan-1" thread to use 100% of one core (at least most of the time) when the warning above gets printed. However, the records in question are being scanned asynchronously which might put part of the burden on other threads. Nevertheless it's unclear to me why the "scan-1" thread can't keep up and if there is anything I can do to improve performance.

I'm also wondering if the work of scanning can be spread out over multiple threads as suggested by Ralph in [1] - it's unclear to me if this ever got implemented.

Thanks a lot,

Martin

[1] http://accelconf.web.cern.ch/AccelConf/ICALEPCS2013/papers/moppc124.pdf

--
Martin Konrad
High Performance Controls Team Leader
Facility for Rare Isotope Beams
Michigan State University
640 South Shaw Lane
East Lansing, MI 48824-1321, USA
Tel. 517-908-7253
Email: [email protected]

References:

Excessive scan times for periodic scans Konrad, Martin via Core-talk

Navigate by Date:

Prev: Excessive scan times for periodic scans Konrad, Martin via Core-talk

Next: Re: EPICS 7 incompatibility in errMdef.h Johnson, Andrew N. via Core-talk

Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <2019>  2020  2021  2022  2023  2024 

Navigate by Thread:

Prev: Excessive scan times for periodic scans Konrad, Martin via Core-talk

Next: Re: Excessive scan times for periodic scans Ralph Lange via Core-talk

Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <2019>  2020  2021  2022  2023  2024