I don't think my IOC uses any of the records that synApps adds except the asynRecord in asyn. I will strip down the application to only the modules actually needed, doing it one at a time and see if I can figure out what the offending module is.
Thanks,
Mark
-----Original Message-----
From: Michael Davidsaver <mdavidsaver at gmail.com>
Sent: Friday, May 27, 2022 2:29 AM
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: tech-talk at aps.anl.gov
Subject: Re: Bus errors accessing VME with base 7.0.6.1 and latest synApps modules
On 5/26/22 20:47, Mark Rivers wrote:
Hi Michael,
Thanks for the instructions on git bisect. It seems to have worked. I only had to do 1 skip because base gave an error building on vxWorks, all of the rest built OK and clearly failed (in just a few seconds) or clearly worked (ran for 10+ minutes).
Here is the final result:
corvette:local/epics-devel/base-7.0.6>git bisect bad
892a361de7002bf2b3f375f24bc5bf61858de2e5 is the first bad commit
commit 892a361de7002bf2b3f375f24bc5bf61858de2e5
Author: Michael Davidsaver <mdavidsaver at gmail.com>
Date: Mon Mar 30 13:56:13 2020 -0700
add alarm message field
:040000 040000 e339a59018c9e2f28c3b4efd555b675a803e84f7 a7fd036f53c20f6e7fd673d7191ffb3b28770a79 M modules
https://github.com/epics-base/epics-base/commit/892a361de7002bf2b3f375f24bc5bf61858de2e5
Well, that points the finger squarely at me...
From the earlier references to dbEvent.c, the obvious place to start is is if you remove the db_post_events() call added by this commit.
diff --git a/modules/database/src/ioc/db/recGbl.c
b/modules/database/src/ioc/db/recGbl.c
index 95387f5de..971716fa5 100644
--- a/modules/database/src/ioc/db/recGbl.c
+++ b/modules/database/src/ioc/db/recGbl.c
@@ -204,7 +204,7 @@ unsigned short recGblResetAlarms(void *precord)
}
if (stat_mask) {
db_post_events(pdbc, &pdbc->stat, stat_mask);
- db_post_events(pdbc, &pdbc->amsg, stat_mask);
+ //db_post_events(pdbc, &pdbc->amsg, stat_mask);
val_mask = DBE_ALARM;
if (!pdbc->ackt || new_sevr >= pdbc->acks) {
Mark
-----Original Message-----
From: Michael Davidsaver <mdavidsaver at gmail.com>
Sent: Thursday, May 26, 2022 5:23 PM
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: tech-talk at aps.anl.gov
Subject: Re: Bus errors accessing VME with base 7.0.6.1 and latest
synApps modules
On 5/26/22 14:48, Mark Rivers wrote:
Other ideas of how to debug it?
If we accept that the stack trace is correct, then memory corruption at some earlier point seems likely. It would be interesting to inspect the stack of the faulting thread. However, I don't know if this is possible with your setup, and wouldn't be able to talk you through it anyway.
At this point, maybe bite the bullet and go for git-bisect? This will likely take some time to work through, although perhaps less than what has already been spent.
According to git 2.30.2 on my laptop, bisecting 7.0.5 through 7.0.6.1 would take 7 iterations assuming no skips (eg. build failures).
Begin with:
git bisect start R7.0.6.1 R7.0.5
This will update your checkout to a revision for testing. Build and run. If if doesn't build, or the result is otherwise inconclusive, then 'skip':
git bisect skip
If the problem appears:
git bisect bad
If the problem does not appear:
git bisect good
Each time the checkout will be updated to another revision. Repeat until complete. Ideally this will narrow down to a single revision.
If there are skips, it will give a range of revisions (hopefully not including a 3.15 merge commit...).