EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019  2020  2021  2022  Index 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019  2020  2021  2022 
<== Date ==> <== Thread ==>

Subject: Re: 3.15.0.2: Shutdown issues
From: Benjamin Franksen <[email protected]>
To: <[email protected]>
Date: Tue, 18 Nov 2014 00:24:03 +0100
Hi Michael

Am Montag, 17. November 2014, 16:06:17 schrieb Michael Davidsaver:
> It looks like you have a thread exit handler which is calling epicsExit()?

It calls exit(EXIT_SUCCESS) from the C standard library (but that may in turn
call epicsExit() by some mechanism in base I am not familiar with).

> Since you are running the latest, can you arrange to set the global
> variable atExitDebug=1 (defined in epicsExit.c)?  This should give some
> clue about where in the exit sequence things are crashing.

It ends now with:

...as before...
    Results
    =======
       Tests: 24
      Passed:  24 = 100.00%
Instance 0 of sequencer program "bittypesTest" terminated
atExit 0x4055f0((nil))
atExit dbProcessNotifyExit((nil))



A call to 'assert(ellCount(&pnotifyGlobal->freeList)==0)'
    by thread 'bittypesTest' failed in ../../../src/ioc/db/dbNotify.c line
304.
epics> Dumping a stack trace of thread 'bittypesTest':
[    0x7fed3459847b]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsStackTrace+0x4b)
[    0x7fed345921aa]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsAssert+0x4a)
[    0x7fed347f070c]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libdbCore.so.3.15.0(dbProcessNotifyExit+0x2c)
[    0x7fed34589d1c]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsExitCallAtExits+0x9c)
[    0x7fed3458a0a8]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsExit+0x8)
[    0x7fed34589e34]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsExitCallAtThreadExits+0x74)
[    0x7fed34593191]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(start_routine+0x101)
[    0x7fed33801182]: /lib/x86_64-linux-gnu/libpthread.so.0(start_thread+0xc2)
[    0x7fed34285fbd]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)
EPICS Release EPICS R3.15.0.2-DEV $$Date$$.
Local time is 2014-11-18 00:22:21.864598739 CET
Please E-mail this message to the author or to [email protected]
Calling epicsThreadSuspendSelf()

>
> Michael
>
> On 11/17/2014 01:57 PM, Benjamin Franksen wrote:
> > Am Freitag, 14. November 2014, 10:32:45 schrieb Michael Davidsaver:
> >> On 11/14/2014 04:16 AM, Ralph Lange wrote:
> >> > ...
> >> > Any ideas on what to do about rsrv shutdown?
> >>
> >> Yesterday I made a commit which I suspect will hide this issue (and many
> >> other shutdown issues).
> >>
> >> http://bazaar.launchpad.net/~epics-core/epics-base/3.15/revision/12587
> >>
> >> This restores the DB shutdown sequence to what it was before the new DB
> >> unittesting code was merged.  This will be necessary until RSRV (and may
> >> other components) learn how to shutdown cleanly.
> >
> > Funny, I have a problem that seems to related to shutdown, too. It may be
> > related or not. This came up when I tested the sequencer (version 2.2, for
> > which I promised Mark to make a new release ASAP). I updated my clone of
> > the base repo immediately before the test, so I should have Michael's
> > latest patch.
> >
> > You can repat this by downloading
> > http://www-csr.bessy.de/control/SoftDist/sequencer/releases/seq-2-2-snaps
> > hot-latest.tar.gz, building it against the head of base/3.15, and then
> > running:
> >
> > [email protected][1]: .../seq/branch-2-2 > cd test/validate/O.linux-x86_64
> > [email protected][1]: .../validate/O.linux-x86_64 > ./bittypes -t -d
> > ../bittypes.db
> > Starting iocInit
> > ##########################################################################
> > ## ## EPICS R3.15.0.2-DEV $$Date$$
> > ## EPICS Base built Nov 17 2014
> > ##########################################################################
> > ## iocRun: All initialization complete
> > Sequencer release 2.2.0.3, compiled Mon Nov 17 19:16:07 2014
> > Spawning sequencer program "bittypesTest", thread 0x143ce60:
> > "bittypesTest"
> > bittypesTest[0]: all channels connected & received 1st monitor
> > 1..24
> > ok  1 - sizeof((i8)) == 1
> > ok  2 - sizeof((ui8)) == 1
> > ok  3 - sizeof((i16)) == 2
> > ok  4 - sizeof((ui16)) == 2
> > ok  5 - sizeof((i32)) == 4
> > ok  6 - sizeof((ui32)) == 4
> > ok  7 - pvstat == pvStatOK
> > ok  8 - pvstat == pvStatOK
> > ok  9 - i8 == seqg_vars_test.seqg_vars_conversion.xi8
> > ok 10 - pvstat == pvStatOK
> > ok 11 - pvstat == pvStatOK
> > ok 12 - ui8 == seqg_vars_test.seqg_vars_conversion.xui8
> > ok 13 - pvstat == pvStatOK
> > ok 14 - pvstat == pvStatOK
> > ok 15 - i16 == seqg_vars_test.seqg_vars_conversion.xi16
> > ok 16 - pvstat == pvStatOK
> > ok 17 - pvstat == pvStatOK
> > ok 18 - ui16 == seqg_vars_test.seqg_vars_conversion.xui16
> > ok 19 - pvstat == pvStatOK
> > ok 20 - pvstat == pvStatOK
> > ok 21 - i32 != seqg_vars_test.seqg_vars_conversion.xi32
> > ok 22 - pvstat == pvStatOK
> > ok 23 - pvstat == pvStatOK
> > ok 24 - ui32 != seqg_vars_test.seqg_vars_conversion.xui32
> >
> >     Results
> >     =======
> >
> >        Tests: 24
> >
> >       Passed:  24 = 100.00%
> >
> > Instance 0 of sequencer program "bittypesTest" terminated
> >
> >
> >
> > A call to 'assert(ellCount(&pnotifyGlobal->freeList)==0)'
> >
> >     by thread 'bittypesTest' failed in ../../../src/ioc/db/dbNotify.c line
> >
> > 304.
> > Dumping a stack trace of thread 'bittypesTest':
> > [    0x7f589713447b]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
> > x86_64/libCom.so.3.15.0(epicsStackTrace+0x4b)
> > [    0x7f589712e1aa]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
> > x86_64/libCom.so.3.15.0(epicsAssert+0x4a)
> > [    0x7f589738c70c]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
> > x86_64/libdbCore.so.3.15.0(dbProcessNotifyExit+0x2c)
> > [    0x7f5897125d1c]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
> > x86_64/libCom.so.3.15.0(epicsExitCallAtExits+0x9c)
> > [    0x7f58971260a8]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
> > x86_64/libCom.so.3.15.0(epicsExit+0x8)
> > [    0x7f5897125e34]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
> > x86_64/libCom.so.3.15.0(epicsExitCallAtThreadExits+0x74)
> > [    0x7f589712f191]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
> > x86_64/libCom.so.3.15.0(start_routine+0x101)
> > [    0x7f589639d182]:
> > /lib/x86_64-linux-gnu/libpthread.so.0(start_thread+0xc2) [
> > 0x7f5896e21fbd]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)
> > EPICS Release EPICS R3.15.0.2-DEV $$Date$$.
> > Local time is 2014-11-17 19:18:35.822216163 CET
> > Please E-mail this message to the author or to [email protected]
> > Calling epicsThreadSuspendSelf()
> >
> >
> > I am using epicsAtThreadExit (in test/validate/testSupport.c) just before
> > the state set thread executes the SNL program shutdown. When the thread
> > exits, my atExit-callback does exit(EXIT_SUCCESS) in order to kill the
> > whole process (the IOC). It is done in this apparently backward manner so
> > that I can detect problems during the shutdown of the sequencer program,
> > which I could not if I called exit directly.
> >
> > Cheers
> > Ben
> > --
> > "Make it so they have to reboot after every typo." -- Scott Adams
> >
> >
> > ________________________________
> >
> > Helmholtz-Zentrum Berlin für Materialien und Energie GmbH
> >
> > Mitglied der Hermann von Helmholtz-Gemeinschaft Deutscher
> > Forschungszentren e.V.
> >
> > Aufsichtsrat: Vorsitzender Prof. Dr. Dr. h.c. mult. Joachim Treusch, stv.
> > Vorsitzende Dr. Beatrix Vierkorn-Rudolph Geschäftsführung: Prof. Dr. Anke
> > Rita Kaysser-Pyzalla, Thomas Frederking
> >
> > Sitz Berlin, AG Charlottenburg, 89 HRB 5583
> >
> > Postadresse:
> > Hahn-Meitner-Platz 1
> > D-14109 Berlin
> >
> > http://www.helmholtz-berlin.de

--
"Make it so they have to reboot after every typo." -- Scott Adams


________________________________

Helmholtz-Zentrum Berlin für Materialien und Energie GmbH

Mitglied der Hermann von Helmholtz-Gemeinschaft Deutscher Forschungszentren e.V.

Aufsichtsrat: Vorsitzender Prof. Dr. Dr. h.c. mult. Joachim Treusch, stv. Vorsitzende Dr. Beatrix Vierkorn-Rudolph
Geschäftsführung: Prof. Dr. Anke Rita Kaysser-Pyzalla, Thomas Frederking

Sitz Berlin, AG Charlottenburg, 89 HRB 5583

Postadresse:
Hahn-Meitner-Platz 1
D-14109 Berlin

http://www.helmholtz-berlin.de


Replies:
Re: 3.15.0.2: Shutdown issues Michael Davidsaver
Re: 3.15.0.2: Shutdown issues Benjamin Franksen
References:
3.15.0.2: Shutdown issues Ralph Lange
Re: 3.15.0.2: Shutdown issues Benjamin Franksen
Re: 3.15.0.2: Shutdown issues Michael Davidsaver

Navigate by Date:
Prev: Build failed in Jenkins: epics-base-3.15-win32s #97 APS Jenkins
Next: Jenkins build is back to normal : epics-base-3.15-vx55 #98 APS Jenkins
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019  2020  2021  2022 
Navigate by Thread:
Prev: Re: 3.15.0.2: Shutdown issues Michael Davidsaver
Next: Re: 3.15.0.2: Shutdown issues Michael Davidsaver
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  <20142015  2016  2017  2018  2019  2020  2021  2022 
ANJ, 18 Nov 2014 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·