Experimental Physics and
Industrial Control System

1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 <2020> 2021 2022 2023 2024	Index	1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 <2020> 2021 2022 2023 2024
<== Date ==>		<== Thread ==>

Subject:	Re: CAS-client thread issues in areaDetector IOC
From:	"Wlodek, Jakub via Tech-talk" <tech-talk at aps.anl.gov>
To:	Mark Rivers <rivers at cars.uchicago.edu>, "Wlodek, Jakub via Tech-talk" <tech-talk at aps.anl.gov>, Michael Davidsaver <mdavidsaver at gmail.com>
Date:	Wed, 5 Feb 2020 15:35:28 +0000

Hi all,

After some testing, I think I am able to reproduce the problem. Essentially, I had a CS-Studio client opened for both the simDetector IOC and the ADUVC IOC, with a misconfigured

CA_MAX_ARRAY_BYTES (it had too small a value). I hadn't noticed because I use ImageJ for testing the camera IOCs.

Closing the CSS window and running the IOC with a simple python script to set values etc. I did not see the error despite running overnight.

I tried running again after fixing the CSS issue, and now I don't see the problem. I wonder how the client is able to have such an effect on the IOC process, and why I only saw the issue running with

base 7.0.3.1?

I also tried the following to make sure the issue was tied to this: I started the IOC without the misconfigured CSS open, and ran the script to control it (this gave no errors). Then, while the IOC was running,

I opened the CSS window, and promptly saw the error message display in the IOC shell several times, followed by a Segmentation Fault, as before.

Thanks for the help with this problem,

Jakub

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Friday, January 31, 2020 7:22 PM
To: Wlodek, Jakub <jwlodek at bnl.gov>; Wlodek, Jakub via Tech-talk <tech-talk at aps.anl.gov>; Michael Davidsaver <mdavidsaver at gmail.com>
Subject: RE: CAS-client thread issues in areaDetector IOC

Hi Jakub,

The error seems to be consistently in the CAS-client. This suggests perhaps there is some CA client that is causing it. Can you try shutting down all CA clients that are accessing this IOC? You can use “casr 1” to determine what clients are connected.

Mark

From: Wlodek, Jakub <jwlodek at bnl.gov>
Sent: Friday, January 31, 2020 8:22 AM
To: Wlodek, Jakub via Tech-talk <tech-talk at aps.anl.gov>; Mark Rivers <rivers at cars.uchicago.edu>; Michael Davidsaver <mdavidsaver at gmail.com>
Subject: Re: CAS-client thread issues in areaDetector IOC

Hi all,

After updating my machine and rebooting it, then recompiling all of EPICS + modules, I ran a test by starting a simDetector IOC and letting it sit overnight.

I didn't activate any plugins or press any buttons, and when I checked on the IOC this morning, the error message was displayed several times, along with

a Segmentation Fault crash:

epics>

epics> epicsEventTrigger: pthread_mutex_lock failed: Invalid argument

epicsEventMustTriggerThread CAS-client (0x7fd6e8042940) can't proceed, suspending.

Dumping a stack trace of thread 'CAS-client':

[ 0x56549c7567e3]: /epics/src/support/areaDetector/ADSimDetector/iocs/simDetectorIOC/bin/linux-x86_64/simDetectorApp(epicsStackTrace+0x73)

[ 0x56549c747215]: /epics/src/support/areaDetector/ADSimDetector/iocs/simDetectorIOC/bin/linux-x86_64/simDetectorApp(cantProceed+0xc5)

[ 0x56549c6d0403]: /epics/src/support/areaDetector/ADSimDetector/iocs/simDetectorIOC/bin/linux-x86_64/simDetectorApp(db_close_events+0x33)

[ 0x56549c6f7d2f]: /epics/src/support/areaDetector/ADSimDetector/iocs/simDetectorIOC/bin/linux-x86_64/simDetectorApp(destroy_tcp_client+0x8f)

[ 0x56549c6f894d]: /epics/src/support/areaDetector/ADSimDetector/iocs/simDetectorIOC/bin/linux-x86_64/simDetectorApp(camsgtask+0x13d)

[ 0x56549c751c18]: /epics/src/support/areaDetector/ADSimDetector/iocs/simDetectorIOC/bin/linux-x86_64/simDetectorApp(start_routine+0xf8)

[ 0x7fd82d7016db]: /lib/x86_64-linux-gnu/libpthread.so.0(start_thread+0xdb)

[ 0x7fd82c49688f]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)

Thread CAS-client (0x7fd6e8042940) suspended

epicsEventTrigger: pthread_mutex_lock failed: Invalid argument

epicsEventMustTriggerThread CAS-client (0x7fd6e8042bf0) can't proceed, suspending.