Re: 3.15.0.2: Shutdown issues

[Benjamin Franksen <benjamin.franksen@helmholtz-berlin.de>]

Am Freitag, 14. November 2014, 10:32:45 schrieb Michael Davidsaver:
> On 11/14/2014 04:16 AM, Ralph Lange wrote:
> > ...
> > Any ideas on what to do about rsrv shutdown?
>
> Yesterday I made a commit which I suspect will hide this issue (and many
> other shutdown issues).
>
> http://bazaar.launchpad.net/~epics-core/epics-base/3.15/revision/12587
>
> This restores the DB shutdown sequence to what it was before the new DB
> unittesting code was merged.  This will be necessary until RSRV (and may
> other components) learn how to shutdown cleanly.

Funny, I have a problem that seems to related to shutdown, too. It may be
related or not. This came up when I tested the sequencer (version 2.2, for
which I promised Mark to make a new release ASAP). I updated my clone of the
base repo immediately before the test, so I should have Michael's latest
patch.

You can repat this by downloading http://www-csr.bessy.de/control/SoftDist/sequencer/releases/seq-2-2-snapshot-latest.tar.gz, building it against the head of base/3.15, and then running:

ben@sarun[1]: .../seq/branch-2-2 > cd test/validate/O.linux-x86_64
ben@sarun[1]: .../validate/O.linux-x86_64 > ./bittypes -t -d ../bittypes.db
Starting iocInit
############################################################################
## EPICS R3.15.0.2-DEV $$Date$$
## EPICS Base built Nov 17 2014
############################################################################
iocRun: All initialization complete
Sequencer release 2.2.0.3, compiled Mon Nov 17 19:16:07 2014
Spawning sequencer program "bittypesTest", thread 0x143ce60: "bittypesTest"
bittypesTest[0]: all channels connected & received 1st monitor
1..24
ok  1 - sizeof((i8)) == 1
ok  2 - sizeof((ui8)) == 1
ok  3 - sizeof((i16)) == 2
ok  4 - sizeof((ui16)) == 2
ok  5 - sizeof((i32)) == 4
ok  6 - sizeof((ui32)) == 4
ok  7 - pvstat == pvStatOK
ok  8 - pvstat == pvStatOK
ok  9 - i8 == seqg_vars_test.seqg_vars_conversion.xi8
ok 10 - pvstat == pvStatOK
ok 11 - pvstat == pvStatOK
ok 12 - ui8 == seqg_vars_test.seqg_vars_conversion.xui8
ok 13 - pvstat == pvStatOK
ok 14 - pvstat == pvStatOK
ok 15 - i16 == seqg_vars_test.seqg_vars_conversion.xi16
ok 16 - pvstat == pvStatOK
ok 17 - pvstat == pvStatOK
ok 18 - ui16 == seqg_vars_test.seqg_vars_conversion.xui16
ok 19 - pvstat == pvStatOK
ok 20 - pvstat == pvStatOK
ok 21 - i32 != seqg_vars_test.seqg_vars_conversion.xi32
ok 22 - pvstat == pvStatOK
ok 23 - pvstat == pvStatOK
ok 24 - ui32 != seqg_vars_test.seqg_vars_conversion.xui32

    Results
    =======
       Tests: 24
      Passed:  24 = 100.00%
Instance 0 of sequencer program "bittypesTest" terminated



A call to 'assert(ellCount(&pnotifyGlobal->freeList)==0)'
    by thread 'bittypesTest' failed in ../../../src/ioc/db/dbNotify.c line
304.
Dumping a stack trace of thread 'bittypesTest':
[    0x7f589713447b]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsStackTrace+0x4b)
[    0x7f589712e1aa]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsAssert+0x4a)
[    0x7f589738c70c]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libdbCore.so.3.15.0(dbProcessNotifyExit+0x2c)
[    0x7f5897125d1c]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsExitCallAtExits+0x9c)
[    0x7f58971260a8]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsExit+0x8)
[    0x7f5897125e34]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(epicsExitCallAtThreadExits+0x74)
[    0x7f589712f191]: /mnt/extra/ben/Epics/R3.15/base/head/lib/linux-
x86_64/libCom.so.3.15.0(start_routine+0x101)
[    0x7f589639d182]: /lib/x86_64-linux-gnu/libpthread.so.0(start_thread+0xc2)
[    0x7f5896e21fbd]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)
EPICS Release EPICS R3.15.0.2-DEV $$Date$$.
Local time is 2014-11-17 19:18:35.822216163 CET
Please E-mail this message to the author or to tech-talk@aps.anl.gov
Calling epicsThreadSuspendSelf()


I am using epicsAtThreadExit (in test/validate/testSupport.c) just before the
state set thread executes the SNL program shutdown. When the thread exits, my
atExit-callback does exit(EXIT_SUCCESS) in order to kill the whole process
(the IOC). It is done in this apparently backward manner so that I can detect
problems during the shutdown of the sequencer program, which I could not if I
called exit directly.

Cheers
Ben
--
"Make it so they have to reboot after every typo." -- Scott Adams


________________________________

Helmholtz-Zentrum Berlin für Materialien und Energie GmbH

Mitglied der Hermann von Helmholtz-Gemeinschaft Deutscher Forschungszentren e.V.

Aufsichtsrat: Vorsitzender Prof. Dr. Dr. h.c. mult. Joachim Treusch, stv. Vorsitzende Dr. Beatrix Vierkorn-Rudolph
Geschäftsführung: Prof. Dr. Anke Rita Kaysser-Pyzalla, Thomas Frederking

Sitz Berlin, AG Charlottenburg, 89 HRB 5583

Postadresse:
Hahn-Meitner-Platz 1
D-14109 Berlin

http://www.helmholtz-berlin.de