2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 <2019> 2020 2021 2022 2023 2024 | Index | 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 <2019> 2020 2021 2022 2023 2024 |
<== Date ==> | <== Thread ==> |
---|
Subject: | Beacon handling |
From: | Timo Korhonen via Core-talk <[email protected]> |
To: | EPICS Core Talk <[email protected]> |
Date: | Fri, 8 Nov 2019 16:06:41 +0000 |
Colleagues, We have recently been investigating reports of IOCs crashing. We are using base 7.0.3 and the problem seems to me somehow to be related to beacon handling in PVA code. I attached a stack trace that I got from our colleagues working on an IOC, trying to integrate a CAEN power supply using StreamDevice.
The crash is probably not related to StreamDevice in any way; it seems only happen when we have PVA support activated.
Does the attached stack trace tell you anything? Can you give ideas about what to do to give better troubleshooting information? Thanks, Timo |
[root@ipc-detector-01 bm-caen]# [root@ipc-detector-01 bm-caen]# gdb /epics/base-7.0.3/bin/linux-x86_64/softIocPVA GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-115.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /epics/base-7.0.3/bin/linux-x86_64/softIocPVA...done. (gdb) (gdb) (gdb) core core.381813 [New LWP 381836] [New LWP 381838] [New LWP 381839] # (...) [New LWP 381827] [New LWP 381817] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/epics/base-7.0.3/bin/linux-x86_64/softIocPVA -D /epics/base-7.0.3/dbd/softIocP'. Program terminated with signal 6, Aborted. #0 0x00007fe37a2f5337 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55 55 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); (gdb) (gdb) (gdb) backtrace full #0 0x00007fe37a2f5337 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55 resultvar = 0 pid = 381813 selftid = 381836 #1 0x00007fe37a2f6a28 in __GI_abort () at abort.c:90 save_stage = 2 act = {__sigaction_handler = {sa_handler = 0x2d, sa_sigaction = 0x2d}, sa_mask = {__val = {45, 140614994131280, 140614984506032, 45, 140613202745480, 140614996503872, 140614987968960, 45, 1, 45, 140614994131280, 140614987953024, 140614984460034, 0, 140613269851440, 140613202742576}}, sa_flags = 1960852144, sa_restorer = 0x7fe314000d30} sigs = {__val = {32, 0 <repeats 15 times>}} #2 0x00007fe37ac057d5 in __gnu_cxx::__verbose_terminate_handler () at ../../../../libstdc++-v3/libsupc++/vterminate.cc:95 terminating = true t = <optimized out> #3 0x00007fe37ac03746 in __cxxabiv1::__terminate (handler=<optimized out>) at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:38 No locals. #4 0x00007fe37ac03773 in std::terminate () at ../../../../libstdc++-v3/libsupc++/eh_terminate.cc:48 No locals. #5 0x00007fe37ac042df in __cxxabiv1::__cxa_pure_virtual () at ../../../../libstdc++-v3/libsupc++/pure.cc:50 No locals. #6 0x00007fe37b9554d9 in epics::pvData::compare (a=..., b=...) at ../../src/factory/Compare.cpp:86 nflds = <optimized out> #7 0x00007fe37b913c4a in epics::pvData::FieldCreate::Helper::cache<epics::pvData::Structure> (create=create@entry=0xce02b0, ent=std::tr1::shared_ptr (count 1, weak 1) 0x7fe310000d30) at ../../src/factory/FieldCreateFactory.cpp:63 cent = 0x7fe314000d30 centx = <optimized out> hash = 2882912583 G = {mutexPtr = @0xce0300, locked = true} itp = {first = {first = 2882912583, second = }, second = {first = 2914433719, second = }} #8 0x00007fe37b90ceda in epics::pvData::FieldCreate::createStructure (this=this@entry=0xce02b0, fieldNames=std::vector of length 6, capacity 6 = {...}, fields=std::vector of length 6, capacity 6 = {...}) at ../../src/factory/FieldCreateFactory.cpp:1305 sp = std::tr1::shared_ptr (count 1, weak 1) 0x7fe310000d30 #9 0x00007fe37b9106d9 in deserializeStructureField (control=0x10ae170, buffer=0x10ae280, fieldCreate=0xce02b0) at ../../src/factory/FieldCreateFactory.cpp:251 id = "" size = 6 fields = std::vector of length 6, capacity 6 = {std::tr1::shared_ptr (count 72, weak 1) 0xce0af0, std::tr1::shared_ptr (count 18, weak 1) 0xce0e30, std::tr1::shared_ptr (count 18, weak 1) 0xce0e30, std::tr1::shared_ptr (count 72, weak 1) 0xce0af0, std::tr1::shared_ptr (count 72, weak 1) 0xce0af0, std::tr1::shared_ptr (count 16, weak 1) 0xce19b0} fieldNames = std::vector of length 6, capacity 6 = {"connections", "allocatedMemory", "freeMemory", "threads", "deadlocks", "averageSystemLoad"} #10 epics::pvData::FieldCreate::deserialize (this=0xce02b0, buffer=buffer@entry=0x10ae280, control=control@entry=0x10ae170) at ../../src/factory/FieldCreateFactory.cpp:1468 typeCode = <optimized out> scalarOrArray = <optimized out> notArray = <optimized out> #11 0x00007fe37bcc4229 in (anonymous namespace)::BeaconResponseHandler::handleResponse (this=<optimized out>, responseFrom=0x7fe374e03d90, transport= std::tr1::shared_ptr (count 2, weak 1) 0x10ae170, version=<optimized out>, command=<optimized out>, payloadSize=<optimized out>, payloadBuffer=0x10ae280) at ../../src/remoteClient/clientContextImpl.cpp:2761 timestamp = {secondsPastEpoch = 1572541624, nanoseconds = 907433738, userTag = 0} field = std::tr1::shared_ptr (empty) 0x0 guid = {value = "gs\222s\217cD\230\206\346\224", <incomplete sequence \321>} serverAddress = {ia = {sin_family = 2, sin_port = 33179, sin_addr = {s_addr = 16914348}, sin_zero = "\000\000\000\000\000\000\000"}, sa = {sa_family = 2, sa_data = "\233\201\254\027\002\001\000\000\000\000\000\000\000"}} protocol = "tcp" data = std::tr1::shared_ptr (empty) 0x0 context = std::tr1::shared_ptr (count 5, weak 57) 0xfe8e90 beaconHandler = std::tr1::shared_ptr (count 2, weak 0) 0x7fe310000a20 #12 0x00007fe37bcbecba in (anonymous namespace)::ClientResponseHandler::handleResponse (this=<optimized out>, responseFrom=<optimized out>, transport=..., version=<optimized out>, command=<optimized out>, payloadSize=<optimized out>, payloadBuffer=0x10ae280) at ../../src/remoteClient/clientContextImpl.cpp:3008 payloadSize = <optimized out> command = 0 '\000' transport = <optimized out> this = <optimized out> payloadBuffer = 0x10ae280 version = <optimized out> responseFrom = <optimized out> #13 0x00007fe37bc9b60f in epics::pvAccess::BlockingUDPTransport::processBuffer (this=this@entry=0x10ae170, transport=std::tr1::shared_ptr (count 2, weak 1) 0x10ae170, fromAddress=..., receiveBuffer=receiveBuffer@entry=0x10ae280) at ../../src/remote/blockingUDPTransport.cpp:420 payloadSize = <optimized out> nextRequestPosition = 190 #14 0x00007fe37bc9c658 in epics::pvAccess::BlockingUDPTransport::run (this=0x10ae170) at ../../src/remote/blockingUDPTransport.cpp:269 ignore = false bytesRead = <optimized out> recvfrom_buffer_start = 0x10ae388 "\312\001\300" recvfrom_buffer_len = 65487 fromAddress = {ia = {sin_family = 2, sin_port = 54291, sin_addr = {s_addr = 16914348}, sin_zero = "\000\000\000\000\000\000\000"}, sa = {sa_family = 2, sa_data = "\023\324\254\027\002\001\000\000\000\000\000\000\000"}} addrStructSize = 16 thisTransport = std::tr1::shared_ptr (count 2, weak 1) 0x10ae170 #15 0x00007fe37aee6829 in epicsThreadCallEntryPoint (pPvt=0x10ce480) at ../../src/osi/epicsThread.cpp:83 pThread = 0x10ce480 threadDestroyed = false #16 0x00007fe37aeec7bc in start_routine (arg=0x10ceb90) at ../../src/osi/os/posix/osdThread.c:403 pthreadInfo = 0x10ceb90 status = <optimized out> blockAllSig = {__val = {18446744067267100671, 18446744073709551615 <repeats 15 times>}} #17 0x00007fe37a0aae65 in start_thread (arg=0x7fe374e04700) at pthread_create.c:307 __res = <optimized out> pd = 0x7fe374e04700 now = <optimized out> unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140614895159040, 9139648661074023024, 0, 2101248, 0, 140614895159040, -9146696730313420176, -9146676574535609744}, mask_was_saved = 0}}, priv = { pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = <optimized out> pagesize_m1 = <optimized out> sp = <optimized out> freesize = <optimized out> #18 0x00007fe37a3bd88d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 No locals. (gdb)