Experimental Physics and Industrial Control System
|
The GeSys application is configured for
an unlimited amount of semaphores and in
case of beatnik/mvme6100 15MB of CONFIGURE_EXECUTIVE_RAM_SIZE.
( https://github.com/till-s/rtems-gesys/blob/master/config.c, for
beatnik MEMORY_HUGE is defined).
If the rtems monitor is loaded (make sure the 'monitor.obj' module
is loaded before you
run into the problem as the monitor creates at least one semaphore
for its own use; IIRC
the standard startup scripts at SLAC do in fact load the monitor)
then you can invoke the monitor from the cexp shell and dump info
about semaphores
cexpsh> rtemsMonitor()
monitor> sema
(note that the monitor has its own interpreter with syntax that
differs from cexpsh's; leave
the monitor by typing 'exit')
monitor> exit
cepxsh>
Strange - in my many years at SLAC I have never seen this
problem...
HTH
- Till
On 5/10/22 04:14, Joel Sherrill via Tech-talk wrote:
On
5/9/22 16:42, Joel Sherrill wrote:
>
>
> On Mon, May 9, 2022, 5:45 PM Michael Davidsaver via
Tech-talk <tech-talk at aps.anl.gov
<mailto:tech-talk at aps.anl.gov>>
wrote:
>
> On 5/9/22 15:08, Luchini, Kristi L. via Tech-talk
wrote:
> > Hello,
> >
> > I have a MVME6100 ioc running RTEMS that
appears to be having a resource issue. Does anyone
recognize these error messages below and happen to know
function call that I can add to the st.cmd to increase
that resource? Do I need to increase the number file
descriptors?
> >
> > If I reboot the ioc the message will go
away and the iocs works fine, but after some period of
time, a month or so, this message reappears and you can’t
access pvs over CA, which I assume is because the iocs
seems to be too busy spewing error messages.
> >
> > Any help would be appreciated.
>
> Sounds like there is a resource leak somewhere,
at least of RTEMS semaphores.
> I'm not sure if the mention of "file descriptor"
is real, or a case of
> overloading the meaning of an errno code.
>
> Which drivers and other code is being loaded?
>
> Are any of these creating worker threads after
iocInit() ?
>
> Are there any earlier error messages being
logged?
>
> You could try running "epicsThreadShowAll()" to
see if some driver threads are
> hanging around when they shouldn't.
>
> Unfortunately, there is no "epicsMutexShowAll"
implemented for RTEMS 4.X, and
> I don't know if your RTEMS builds include the
RTEMS shell and its associated
> diagnostics functions.
>
>
> I don't know what RTEMS configure options are used
but there is an option for unlimited and another for
unified workspace. If they both are configured, you have
one pool of memory to allocate everything from. This
includes RTEMS objects and things you allocate via
malloc() and new. In this configuration, any leak would be
bad and could lead to this. It would just be a matter of
which create or allocate failed. Malloc() will return NULL
in this situation and that often isn't error checked.
@joel, I'm 90% certain that Kristi is using Till's GESYS
application,
which has a different RTEMS configuration. So the
following isn't
relevant for @SLAC.
Still. fyi. The current 7.0 default RTEMS configurations
for EPICS applications are:
for RTEMS <= 4.x which uses separate pools.
https://github.com/epics-base/epics-base/blob/cbae8d37b3da486ac8a68ad6ef9d2028cd98cca0/modules/libcom/RTEMS/score/rtems_config.c#L29-L38
I think that for 4.10, the cpp logic is setting
unified workspace. If I remember correctly that's the first
version with that feature.
This uses 64 as the maximum file descriptors
while the previous had a much larger number. But 64 is still a
lot. You shouldn't run out. :)
Kristi.. what are the configure settings for
maximums/unlimited and unified workspace.
I am pretty sure unified workspace was first an
option around 4.10.
--joel
> --joel
>
>
>
> > Thanks,
> >
> > Kristi
> >
> > Message from RTEMS ioc:
> >
> > Can't create mutex semaphore: too many
> >
> > Can't create mutex semaphore: too many
> >
> > Can't create mutex semaphore: too many
> >
> > Can't create mutex semaphore: too many
> >
> > Cexp@ioc-in10-mp01>
> >
> > Cexp@ioc-in10-mp01>Error on UDP RX 172.27.72.43:47591
<http://172.27.72.43:47591>
-> 172.27.75.255:5076
<http://172.27.75.255:5076>
at 46 : Can't create mutex semaphore: too many
> >
> > epicsMutex::mutexCreateFailed()
> >
> > 0x00 ca024000 27000000 d5b37962 00000000
..@. '... ..yb ....
> >
> > 0x10 c84f9922 00740000 00000000 00000000
.O." .t.. .... ....
> >
> > 0x20 0000ffff 00000000 d3130374 6370ff
.... .... ...t cp.
> >
> > CAS: Client accept error: Too many open
files in system (16)
> >
> > Error on UDP RX 172.27.72.99:37311
<http://172.27.72.99:37311>
-> 172.27.75.255:5076
<http://172.27.75.255:5076>
at 46 : Can't create mutex semaphore: too many
> >
> > epicsMutex::mutexCreateFailed()
> >
> > 0x00 ca024000 27000000 a5a77862 00000000
..@. '... ..xb ....
> >
> > 0x10 ea850c38 004f0000 00000000 00000000
...8 .O.. .... ....
> >
> > 0x20 0000ffff 00000000 d3130374 6370ff
.... .... ...t cp.
> >
> > * Kristi
> >
>
- References:
- RE: Can't create mutex semaphore: too many Luchini, Kristi L. via Tech-talk
- Re: Can't create mutex semaphore: too many Michael Davidsaver via Tech-talk
- Re: Can't create mutex semaphore: too many Joel Sherrill via Tech-talk
- Re: Can't create mutex semaphore: too many Michael Davidsaver via Tech-talk
- Re: Can't create mutex semaphore: too many Joel Sherrill via Tech-talk
- Navigate by Date:
- Prev:
Re: Can't create mutex semaphore: too many Joel Sherrill via Tech-talk
- Next:
Re: Device support of asyn port driver for system commands. Hu, Yong via Tech-talk
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
<2022>
2023
2024
- Navigate by Thread:
- Prev:
Re: Can't create mutex semaphore: too many Joel Sherrill via Tech-talk
- Next:
Re: Can't create mutex semaphore: too many Till Straumann via Tech-talk
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
<2022>
2023
2024
|
ANJ, 14 Sep 2022 |
·
Home
·
News
·
About
·
Base
·
Modules
·
Extensions
·
Distributions
·
Download
·
·
Search
·
EPICS V4
·
IRMIS
·
Talk
·
Bugs
·
Documents
·
Links
·
Licensing
·
|