Re: Error that seems to suspend all .st programs

[Michael Davidsaver via Tech-talk <tech-talk@aps.anl.gov>]

On 12/3/25 10:26 AM, McClanahan, Ryan via Tech-talk wrote:

Hi,

I get the following error from time to time in a soft IOC that only hosts a few .st programs, and each time I get the message all of the .st programs lock up.  This must be something deep in epics because I don't have a seq_ca.c in my app, and it asks me to email teck-talk.  What can I do about this?

A call to 'assert(dbch)'

    by thread 'CAC-TCP-recv' failed in ../seq_ca.c line 378.

This assertion comes from:

https ://github.com/epics-modules/sequencer/blob/f313d4c2ef47dec28448f125ed68639c5afbcfed/src/seq/seq_ca.c#L378

Do you see any earlier errors or warnings?

This might be a situation where some previous error was not handled correctly, resulting in this later logic error.

Dumping a stack trace of thread 'CAC-TCP-recv':

[          0x777ac3]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(epicsStackTrace+0x53)

[          0x77207e]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(epicsAssert+0x3e)

[          0x533a0b]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(seq_camonitor+0x14b)

[          0x533bbb]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(seq_conn_handler+0x17b)

[          0x73b4df]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(_ZN16oldChannelNotify13connectNotifyER10epicsGuardI10epicsMutexE+0x3f)

[          0x73fdc5]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(_ZN3cac23createChannelRespActionER15callbackManagerR6tcpiiuRK9epicsTimeRK15caHdrLargeA

rrayPv+0xc5)  

[          0x74cad3]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(_ZN6tcpiiu15processIncomingERK9epicsTimeR15callbackManager+0x1e3)

[          0x74ecd7]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(_ZN13tcpRecvThread3runEv+0x267)

[          0x76dc9b]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(epicsThreadCallEntryPoint+0x2b)

[          0x7731b2]: /afs/slac/g/lcls/epics/iocCommon/sioc-sys0-ts06/iocSpecificRelease/bin/rhel9-x86_64/tpgSupport(start_routine+0xc2)

[    0x7fb4a8c89d22]: /lib64/libc.so.6(start_thread+0x2d2)

[    0x7fb4a8d0ed40]: /lib64/libc.so.6(__clone3+0x30)

EPICS Release EPICS R7.0.3.1-1.0.6.

Local time is 2025-11-26 17:23:41.758220473 PST

Please E-mail this message to benjamin.franksen@helmholtz-berlin.de or to tech-talk@aps.anl.gov

Calling epicsThreadSuspendSelf()

sioc-sys0-ts06>seqShow bsaControl23

State Program: "bsaControl"

  thread priority = 50

  number of state sets = 1

  number of syncQ queues = 0

  number of channels = 20016

  number of channels assigned = 852

  number of channels connected = 192

  number of channels monitored = 4

  options: async=0, debug=1, newef=1, reent=1, conn=1

  user variables: address = 0x7fb45987a010, length = 960336

  State Set: "bsaControlss"

  thread name = bsaControl23;  Thread id = 0x58ee1f0

  First state = "new_root_names"

  Current state = "checking"

  Previous state = "assigning"

  Elapsed time since state was entered = 578695.89 seconds

  Wake up delay = inf seconds

  Get in progress = [000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]

  Put in progress = [000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000]

Kind Regards,

Ryan McClanahan

AD-EED Controls Software

Desk Ext. 7493

SLAC National Laboratory
Bldg 034, Rm 206

2575 Sand Hill Rd., MS50
Menlo Park, CA. 94025

rpm@slac.stanford.edu