Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020 
<== Date ==> <== Thread ==>

Subject: Re: Problems with stream device 2.8.8
From: "Zimoch Dirk \(PSI\) via Tech-talk" <tech-talk@aps.anl.gov>
To: "anj@anl.gov" <anj@anl.gov>, "rivers@cars.uchicago.edu" <rivers@cars.uchicago.edu>
Cc: "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Tue, 9 Jul 2019 07:31:47 +0000
I have created a bug report here: 
https://github.com/paulscherrerinstitute/StreamDevice/issues/41
Dirk


On Mon, 2019-07-08 at 17:42 +0000, Mark Rivers via Tech-talk wrote:
> I have been communicating with Abdalla about this offline.  He sent the attached gdb output when trying to exit.
>  
> This is from my last message to him:
>  
> It looks like the problem is these 2 threads:
>  
> Thread 5 (Thread 0x7fffedfe9700 (LWP 30806)):
> #0  0x00007ffff5be38ed in connect () from /lib64/libc.so.6
> #1  0x00007ffff76dddb4 in connectIt (pasynUser=0x713308, drvPvt=0x71def0) at ../../asyn/drvAsynSerial/drvAsynIPPort.c:476
> #2  asynCommonConnect (drvPvt=0x71def0, pasynUser=0x713308) at ../../asyn/drvAsynSerial/drvAsynIPPort.c:520
> #3  0x00007ffff76d08fb in portConnectProcessCallback (pasynUser=0x713308) at ../../asyn/asynDriver/asynManager.c:3076
> #4  0x00007ffff76d39c7 in portThread (pport=0x711ed0) at ../../asyn/asynDriver/asynManager.c:820
> #5  0x00007ffff67128ec in start_routine (arg=0x712f10) at ../../../src/libCom/osi/os/posix/osdThread.c:403
> #6  0x00007ffff5698e25 in start_thread () from /lib64/libpthread.so.0
> #7  0x00007ffff5be2bad in clone () from /lib64/libc.so.6
>  
> Thread 1 (Thread 0x7ffff7fd9740 (LWP 30798)):
> #0  0x00007ffff569f51d in __lll_lock_wait () from /lib64/libpthread.so.0
> #1  0x00007ffff569ae36 in _L_lock_870 () from /lib64/libpthread.so.0
> #2  0x00007ffff569ad2f in pthread_mutex_lock () from /lib64/libpthread.so.0
> #3  0x00007ffff6714b66 in mutexLock (id=0x712150) at ../../../src/libCom/osi/os/posix/osdMutex.c:46
> #4  epicsMutexOsdLock (pmutex=0x712150) at ../../../src/libCom/osi/os/posix/osdMutex.c:130
> #5  0x00007ffff76cf38b in lockPort (pasynUser=0x714d08) at ../../asyn/asynDriver/asynManager.c:1741
> #6  0x00007ffff76dcd66 in cleanup (arg=0x71def0) at ../../asyn/drvAsynSerial/drvAsynIPPort.c:246
> #7  0x00007ffff6708dd3 in epicsExitCallAtExitsPvt (pep=<optimized out>) at ../../../src/libCom/misc/epicsExit.c:95
> #8  epicsExitCallAtExits () at ../../../src/libCom/misc/epicsExit.c:113
> #9  0x00007ffff6709178 in epicsExit (status=0) at ../../../src/libCom/misc/epicsExit.c:181
> #10 0x0000000000405d0d in main (argc=<optimized out>, argv=<optimized out>) at ../srMain.cpp:21
>  
> Thread 1 is drvAsynIPPort trying to close the socket.  It is hanging on the call to pasynManager->lockPort(), which tries to lock pport->synchronousLock.  The reason is that thread 5 is the asyn port thread trying to connect to the device, and it has the mutex pport->synchronousLock locked.  I suspect that thread 5 is trying to connect frequently because many records are talking to the device, and the device has Autoconnect=Yes.  If that thread never unlocks the mutex for long then this problem will occur.
>  
> I need to try to reproduce this and see if there is a solution.
>  
> Next time it happens before you type "exit" please type "epicsMutexShowAll 1".
>  
> Mark
>  
>  
>  
> From: tech-talk-bounces@aps.anl.gov <tech-talk-bounces@aps.anl.gov> On Behalf Of Johnson, Andrew N. via Tech-talk
> Sent: Monday, July 8, 2019 11:50 AM
> To: tech-talk@aps.anl.gov
> Subject: Re: Problems with stream device 2.8.8
>  
> Hi Abdalla,
> 
> On 7/7/19 5:46 AM, Abdalla Ahmad via Tech-talk wrote:
> > Exit command renders the IOC shell frozen, i.e. the IOC does not exit.
> >  
> > We faced #2 before, but I want to ask is it asyn or stream device problem? Because I want to try different versions but I’m not sure whether asyn or stream to change.
> 
> We can't answer that, but you might get some more hints by running the command
>     var atExitDebug 1
> on the IOC shell before exiting. It causes a message to be printed before each of the registered epicsAtExit() routines gets run, so you should be able to see which routine is hanging.
> 
> - Andrew
> 
> -- 
> Complexity comes for free, Simplicity you have to work for.

References:
Problems with stream device 2.8.8 Abdalla Ahmad via Tech-talk
Re: Problems with stream device 2.8.8 Johnson, Andrew N. via Tech-talk
RE: Problems with stream device 2.8.8 Mark Rivers via Tech-talk

Navigate by Date:
Prev: Re: Fwd: cross-compiling (kind of) with Buildroot Pierrick M Hanlet via Tech-talk
Next: RE: updating ADAndor from RHEL6 to RHEL7 tom.cobb--- via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020 
Navigate by Thread:
Prev: RE: Problems with stream device 2.8.8 Mark Rivers via Tech-talk
Next: RE: Problems with stream device 2.8.8 Abdalla Ahmad via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  <20192020 
ANJ, 10 Jul 2019 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·