EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  <20232024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  <20232024 
<== Date ==> <== Thread ==>

Subject: Re: Scan rate '.01 second' not achievable
From: Andrew Johnson via Tech-talk <tech-talk at aps.anl.gov>
To: Ricardo Cardenes <ricardo.cardenes at noirlab.edu>, Érico Nogueira Rolim <erico.rolim at lnls.br>
Cc: "Barrett \(US\), Patrick E" <patrick.e.barrett at boeing.com>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Date: Fri, 6 Jan 2023 11:24:48 -0600
The EPICS core developers have long been aware that there are problems with the use and Posix implementation of epicsThreadSleepQuantum() on Linux, but they haven't risen to the level of itchiness that we have scratched them ourselves — patches or pull requests welcome! Implementing the suggestions here might be one possible route.

This email provides some additional background information.

The text in the old Application Developers guide that described the function says:

epicsThreadSleepQuantum

This function returns the minimum slumber interval obtainable with epicsThreadSleep() in seconds. On most OS there is a system scheduler interrupt interval which determines the value of this parameter. Knowledge of this parameter is used by the various components of EPICS to improve scheduling of software tasks in time when the reduction of average time scheduling errors is important. If this parameter is unknown or is unpredictable for a particular OS then it is safe to return zero.

I obviously forgot that last sentence when I wrote the code in dbScan.c which divides by the quantum, so that should get fixed although nobody has actually reported being hit by it in over 10 years.

EPICS Base provides a performance measurement program that you can run to measure/estimate what the epicsThreadSleepQuantum() result should be on your system. On EPICS 7.0.x it can be run like this:
$ ./modules/libcom/test/O.linux-x86_64/epicsThreadPerform
epicsThreadSetPriority called by non epics thread
epicsThreadSetPriority called by non epics thread
testPriority main error expected from epicsThreadSetPriority
    id 0x7ea450 old 0 new 0
testPriority thread
    id 0x7ecd50 old 50 new 99
Estimating sleep quantum..........done
Estimating sleep quantum..........done
The epicsThreadSleepQuantum() call returns 0.010000 sec.
This doesnt match the quantum estimate of 0.000163 sec within 10%.
It takes 0.059363 micro sec to call epicsThreadGetIdSelf ()
epicsThreadPrivateGet() takes 0.023940 microseconds
Adjust O.linux-x86_64 in the path for your particular host architecture. The above was on a RHEL-7 system, RHEL-8 gives me effectively the same result. On my Apple M1 MacBook Pro though the output looks like this:
$ ./modules/libcom/test/O.darwin-aarch64/epicsThreadPerform
epicsThreadSetPriority called by non epics thread
epicsThreadSetPriority called by non epics thread
testPriority main error expected from epicsThreadSetPriority
    id 0x600003f3c0b0 old 0 new 0
testPriority thread
    id 0x600003d3c000 old 50 new 99
Estimating sleep quantum..........done
Estimating sleep quantum..........done
The epicsThreadSleepQuantum() call returns 0.010000 sec.
This doesnt match the quantum estimate of 0.009273 sec within 1%.
It takes 0.011298 micro sec to call epicsThreadGetIdSelf ()
epicsThreadPrivateGet() takes 0.005903 microseconds
In EPICS Base-3.15 the path to use is ./src/libCom/test/O.linux-x86_64/epicsThreadPerform but the output is identical. I don't particularly like the "doesn't match the quantum estimate ... within n%" language in the output, but those of you with a statistical background might recognize an attempt to be mathematically correct.

Thus on a Mac the value returned by epicsThreadSleepQuantum() is reasonably accurate, while on a fairly recent and fast Linux box the "minimum slumber interval" may be up to about 2 orders of magnitude smaller.

The concept of the quantum is important for systems like RTEMS and VxWorks that schedule thread delays using a fixed "tick" interval, so the IOC code does need to account for that and we can't just remove it completely. The issues #106 and LP: #1861612 are related to that.

- Andrew


On 1/6/23 9:14 AM, Ricardo Cardenes via Tech-talk wrote:
Hi Érico,



On Fri, Jan 6, 2023 at 11:48 AM Érico Nogueira Rolim via Tech-talk <tech-talk at aps.anl.gov> wrote:
On Linux, that function uses essentially '1.0 / sysconf ( _SC_CLK_TCK )'. The sysconf value is usually 100 (Hz), and it would explain why EPICS thinks it won't achieve the desired scan rate. I don't think a modern kernel will actually be limited by its tick rate, so I don't know how accurate that heuristic is.

sysconf provides information about the system to the user. Last time I checked (but this was many years ago) the tick rate for Linux is configured at the kernel's build time: if sysconf(_SC_CLK_TCK) is returning 100, that means the kernel has been built with a 100 Hz tick rate, and that's about it. Years ago there was the option of building the kernel with higher rates (for more responsive systems), from top of my head I believe the alternative values were 350Hz (maybe 250? can't remember) and 1000Hz, plus an additional "NO_HZ" mode where you build a "tickless kernel" but this is of no use to us, I believe.

Googling a bit I see a "divider" kernel parameter so that a 1000Hz compiled value can be divided a boot time to the one we want to use (eg. divider=10 -> 100Hz), but I'm not sure if this is the case for Barret. At best the kernel might be built with a 1000Hz, with a "divider=10" by default in the boot options, and it would be just a matter of changing that. At worst default kernel is simply built configured to use 100Hz, and then a different pre-compiled one could be used (distros often include one with PREEMPT built in, maybe at 1000Hz), and failing everything else a custom kernel could be built.

Cheers,
Ricardo


Either way, the 'period' value isn't touched after this verification, so I think the IOC will still attempt to use it. You could try running a simple counter in your SCAN and checking with camonitor if the 100Hz rate is actually achieved.


I think scanppl() might have trouble detecting overruns with a period of 0.01, though:


        if (period > 0.0 &&
            (fabs(period - ppsl->period) > 0.05))
            continue;


By the way, epicsThreadSleepQuantum() can return 0.0 if the sysconf errors out, which I believe would result in a SIGFPE in initPeriodic() when dividing by quantum.

Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente.

Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately.


-- 
Complexity comes for free, Simplicity you have to work for.

Replies:
Re: Scan rate '.01 second' not achievable Michael Davidsaver via Tech-talk
Re: Scan rate '.01 second' not achievable Michael Davidsaver via Tech-talk
References:
Scan rate '.01 second' not achievable Barrett (US), Patrick E via Tech-talk
Re: Scan rate '.01 second' not achievable Érico Nogueira Rolim via Tech-talk
Re: Scan rate '.01 second' not achievable Ricardo Cardenes via Tech-talk

Navigate by Date:
Prev: Re: Scan rate '.01 second' not achievable Ricardo Cardenes via Tech-talk
Next: Re: Scan rate '.01 second' not achievable Hu, Yong via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  <20232024 
Navigate by Thread:
Prev: Re: Scan rate '.01 second' not achievable Ricardo Cardenes via Tech-talk
Next: Re: Scan rate '.01 second' not achievable Michael Davidsaver via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  <20232024 
ANJ, 09 Jan 2023 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·