Subject: |
EPICS on RTEMS crashing on CA access, due to GNU readline?!? |
From: |
Angus Gratton <[email protected]> |
To: |
tech-talk <[email protected]> |
Date: |
Tue, 03 May 2011 16:07:28 +1000 |
I have a _very_ unusual bug that I'm currently working through.
I'm running RTEMS 4.10 on PowerPC (MVME-3100) w/ EPICS base 3.14.11.
A short time ago I noticed my IOCs had started occasionally hanging
following a 'caput' - no console output, no iocsh, no ping or CA
responses, required a hard reboot. Strange.
Turns out I can reproduce it very easily from a command line client:
while [ 1 ]; do caput test 0; caput test 1; done
Will hang my crate within 5-60 seconds of it running. It seems to
require the individual caput channel/socket init/teardowns to to run,
none of my long-running UI processes seem to cause it to trigger.
I went back back and bisected through my changes, and found that it was
when I recently enabled GNU Readline (!!) in Base that this started
happening.
Sure enough - if I just change COMMANDLINE_LIBRARY from READLINE back to
EPICS, I can run my test script for hours on end without hanging.
One better, if I edit src/libCom/osi/os/default/epicsReadline.c and add
a forced
#define EPICS_COMMANDLINE_LIBRARY EPICS_COMMANDLINE_LIBRARY_EPICS
... the problem goes away.
The problem doesn't seem to have any other mitigating factor that I can
find (I've disabled all my custom libraries, dbd includes, etc. and it
still happens.) Also, the use of readline in epicsReadline.c looks
innocuous - nothing there that jumps out at me as a red flag problem.
So my questions are:
1) Can anyone with a similar setup, running w/ readline, reproduce the
same problem if they run my test script?
2) Does anyone have any ideas on how I might continue to debug this? I'm
assuming it's some kind of memory corruption problem, may or may not
actually have anything to do with readline (it may just happen readline
triggers an access pattern that causes previously corrupted memory to
hang, maybe something w/ stdin/stdout on RTEMS.)
Any ideas?
Regards,
Angus
- Replies:
- Re: EPICS on RTEMS crashing on CA access, due to GNU readline?!? Angus Gratton
- Navigate by Date:
- Prev:
Re: EPICS task watch dog Till Straumann
- Next:
Re: EPICS on RTEMS crashing on CA access, due to GNU readline?!? Angus Gratton
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
<2011>
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
mca R7-0 Mark Rivers
- Next:
Re: EPICS on RTEMS crashing on CA access, due to GNU readline?!? Angus Gratton
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
<2011>
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|