EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: 'assert (pca->pgetNative)' failed in ../dbCa.c
From: Andrew Johnson <[email protected]>
To: "[email protected]" <[email protected]>, EPICS tech-talk <[email protected]>
Date: Fri, 29 Jun 2018 11:33:46 -0500
Hi Michael,

On 06/29/2018 01:43 AM, [email protected] wrote:
> From: [email protected]
>> So by this point you're probably hoping that the attached patch
>> fixes the issue; well congratulations for reading this far, I tried
>> out my suspicion above and the attached patch does seem to work for
>> me on the 3.14 branch version, which should be close enough to
>> yours to be able to apply one way or the other. Please test and let
>> me know so I can apply it to the Base-3.14 branch and merge up.
> 
> I've just noticed that the patch doesn't address the data type
> mismatch directly, only through the separate connection callback.  Is
> this going to be enough to avoid hitting those asserts even in the
> presence of an IOC server breaking the rules?

Without the patch I can trigger this assertion using the attached pair
of databases. Start by booting both in separate IOCs, then switch the
anj:val alias to point to a different record type and reboot that IOC.
This causes the other IOC to die with:

> epics> DB CA Link Exception: "Virtual circuit disconnect", context "tux.aps.anl.gov:44710"
> 
> 
> 
> A call to 'assert(pca->pgetNative)'
>     by thread 'CAC-TCP-recv' failed in ../dbCa.c line 686.
> EPICS Release EPICS R3.14.12.7-DEV.
> Local time is 2018-06-27 14:45:03.260025965 CDT
> Please E-mail this message to the author or to [email protected]
> Calling epicsThreadSuspendSelf()

After applying that patch I can see all the INP links reconnecting and
getting new values from the alias without any problems. It takes a
couple of scan periods for everything to reconnect and sync up in this
example, but there were no more crashes in my testing.

> After all, an assert is a confident statement that the invariant in
> question is never going to be broken, because all of the elements of
> the invariant are under *our* control; but in this case aren't we
> still in the situation where the ca_field_type() is not as expected?
> Or are you saying (implicitly) that here connectionCallback() is
> *guaranteed* to be called before any change of ca_field_type()?

Au contraire, ca_field_type() should have been updated before the call
to connectionCallback(). However the assertion that fails is in
eventCallback() which isn't called until after connectionCallback(). By
clearing the original monitor subscription inside connectionCallback()
we are stopping the calls to eventCallback() with the old native data type.

> I have a feeling that in our case when we saw the failure, it wasn't
> so much that the restarting server changed its record type, but that
> there was something rather more bumpy about its restart (my colleague
> working on it was trying to migrate between different EPICS and Linux
> versions with some unexpected failures).  However the only concrete
> evidence we have is the error message and a large coincidence.

If you can find another way to trigger this assertion failure with the
patch applied please try to replicate it so someone else can trigger it.

- Andrew


-- 
Arguing for surveillance because you have nothing to hide is no
different than making the claim, "I don't care about freedom of
speech because I have nothing to say." -- Edward Snowdon
record(ai, "$(user):ai") {
    field(INP, "$(user):val")
    field(SCAN, "10 second")
}
record(bi, "$(user):bi") {
    field(INP, "$(user):val")
    field(SCAN, "10 second")
}
record(longin, "$(user):li") {
    field(INP, "$(user):val")
    field(SCAN, "10 second")
}
record(stringin, "$(user):si") {
    field(INP, "$(user):val")
    field(SCAN, "10 second")
}
record(bi, "$(user):v1") {
    field(ZNAM, "Zero")
    field(ONAM, "One")
    field(VAL, 1)
}
record(ai, "$(user):v2") {
    field(VAL, 2)
}
record(stringin, "$(user):v3") {
    field(VAL, "3")
}
alias("$(user):v3", "$(user):val")

Replies:
RE: 'assert (pca->pgetNative)' failed in ../dbCa.c [email protected]
References:
'assert (pca->pgetNative)' failed in ../dbCa.c [email protected]
Re: 'assert (pca->pgetNative)' failed in ../dbCa.c Andrew Johnson
RE: 'assert (pca->pgetNative)' failed in ../dbCa.c [email protected]

Navigate by Date:
Prev: RE: 'assert (pca->pgetNative)' failed in ../dbCa.c [email protected]
Next: Setting items property for Combo widget in Display Builder Gregory, Ray
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: RE: 'assert (pca->pgetNative)' failed in ../dbCa.c [email protected]
Next: RE: 'assert (pca->pgetNative)' failed in ../dbCa.c [email protected]
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020  2021  2022  2023  2024 
ANJ, 09 Jul 2018 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·