Chip,
A problem exists with elevating the CA TCP task priority that has caused
us to not run in that configuration for some time now. It has happened
that an EPICS processing task (such as a scan or sequence task) becomes
stuck in an infinite loop polling something that never responds. You can
argue that this is due to poor implementation, but never the less, it happens.
When this does happen, under normal circumstances (CA TCP task at priority 181),
the CA tasks sending notification to clients that the IOC is alive become
starved and the client eventually disconnects. The client then tries to
reconnect, but is blocked because the CA TCP task is also starved. At
this point someone usually notices, logs into the IOC, and finds the offending
task starving all tasks with lower priority. This is fairly easy to diagnose.
When the CA TCP task has an elevated priority of say 54 (as was the case at
Jefferson Lab for some time), these reconnects are processed into new
CA client and event tasks which only die on the vine themselves because
of CPU starvation. New connections consume file descriptors. This
disconnect/reconnect process repeats until all the IOC file descriptors
are consumed and the IOC locks up and will not even allow telnet to connect.
This becomes a harder problem to diagnose as the IOC appears to hang for an
unknown reason and no diagnostic can be performed. With the use of our
CA name server, we now have acceptable connection times without elevating
priorities.
--
Richard Dickson
Thomas Jefferson National Accelerator Facility
[email protected]
(757)269-5082
- Replies:
- Re: IOC Performance numbers Marty Kraimer
- References:
- IOC Performance numbers Dave Reid
- Re: IOC Performance numbers Chip Watson
- Navigate by Date:
- Prev:
Re: Compiling EPICS Matthieu Bec
- Next:
Re: INP field for generic GPIB record Ned Arnold
- Index:
1994
1995
1996
1997
1998
1999
<2000>
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
priority inversions with scanning and channel access leo dalesio
- Next:
Re: IOC Performance numbers Marty Kraimer
- Index:
1994
1995
1996
1997
1998
1999
<2000>
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|