EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
<== Date ==> <== Thread ==>

Subject: Re: Bus errors accessing VME with base 7.0.6.1 and latest synApps modules
From: Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>
To: Jörn Dreyer <j.dreyer at hzdr.de>
Cc: "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Date: Thu, 2 Jun 2022 12:49:16 +0000
Hi Jorn,

I don’t think valgrind or electric fence are available for our version of vxWorks, but I will check with Andrew Johnson when he returns from vacation.

Michael D. did run valgrind on Linux with same soft database I was testing and he did not find a problem.

Thanks,
Mark


Sent from my iPhone

> On Jun 2, 2022, at 3:49 AM, Jörn Dreyer via Tech-talk <tech-talk at aps.anl.gov> wrote:
> 
> Hi,
> 
> from the first message on, this problem to me looked like a array out of bound 
> problem somewhere. Did you consider running the IOC for a test using valgrind?
> Or compile it with the electric fence library? 
> 
> Regards,
> 
> Jörn 
> 
> Am Donnerstag, 2. Juni 2022, 06:06:53 CEST schrieb Mark Rivers via Tech-talk:
>> We need to explain why adding 2 lines of code to examine the contents of pfl
>> eliminates the failures completely.
> 
>> It’s like a new type of quantum computer - making an observation changes the
>> outcome!
> 
>> Mark
>> 
>> 
>> Sent from my iPhone
>> 
>> 
>>> On Jun 1, 2022, at 10:24 PM, Michael Davidsaver <mdavidsaver at gmail.com>
>>> wrote:
> 
>>> On 6/1/22 14:50, Mark Rivers wrote:
>>> 
>>>> I checked out R7.0.6.1 and made the following change to dbEvent.c  It
>>>> checks whether pfl or pfl->u.r.dtor are in the VME A32 address space. 
>>>> If they are it prints an error and returns.
>> 
>>> 
>>> So, these addresses appear in pfl->u.r.dtor or eg.
>>> pfl->u.v.field.dbf_float
> 
>>> My question is then which part(s) of the db_field_log struct have
>>> incorrect values?
> (eg. "\xbf\x3e\x00\x1e" as float32 is -0.74)
>>> 
>>> One straightforward, though a bit tedious, think to do would be to print
>>> a
>>> dump of the contents of the db_field_log.  eg. I'd be interested to know
>>> if db_field_log::time appears valid.
>>> 
>>> ...
>>> 
>>>> After making that change then one time when I started the IOC I got these
>>>> messages right after iocInit, which is not when it usually fails.
>> 
>>> 
>>> This makes me wonder if these "addresses" are eg. somehow valid ADC
>>> values,
> which just happen to track through the VME window sometimes. 
>>> Though if you've only loading soft records, I don't see how this could
>>> be.
>>> 
>>> Further, I still don't see how these values aren't appearing in the
>>> register dump.
> I don't think PPC has any instructions which do a double
>>> indirection. 
>>> 
>>> Another odd ball idea which occurs to me is that maybe the compiler is
>>> mishandling
> the db_field_log::type bit field.  The dbEvent.o which you
>>> send to Till and I has: 
>>> 
>>>>     4c:       80 1f 00 00     lwz     r0,0(r31)
>>>>     50:       2f 80 00 00     cmpwi   cr7,r0,0
>>> 
>>> 
>>> If I read things correctly, I think this is meant to correspond to
>>> 'pfl->u.r.type==dbfl_type_val==0", however it is testing the whole word,
>>> and would
> mis-trigger because 'pfl->u.r.ctx == dbfl_context_event == 1`
>>> for a monitor update. 
>>> When I look at the result of a local RTEMS build, I think the
>>> corresponding is:
> 
>>> 
>>>>  8:   89 24 00 00     lbz     r9,0(r4)
>>>>  c:   71 29 00 80     andi.   r9,r9,128
>>> 
>>> 
>>> Which is testing the high bit as it should.
>>> 
>>> 
>>> 
>>> For completeness.  Disassembly of db_delete_field_log() from dbEvent.o.
>>> Note that this is object code, with incomplete relocations.
>>> 
>>> 
>>>> 00000034 <db_delete_field_log>:
>>>> 
>>>>     34:       94 21 ff f0     stwu    r1,-16(r1)
>>>>     38:       7c 08 02 a6     mflr    r0
>>>>     3c:       93 e1 00 0c     stw     r31,12(r1)
>>>>     40:       7c 7f 1b 79     mr.     r31,r3
>>>>     44:       90 01 00 14     stw     r0,20(r1)
>>>>     48:       41 82 00 40     beq     88 <db_delete_field_log+0x54>
>>>>     4c:       80 1f 00 00     lwz     r0,0(r31)
>>>>     50:       2f 80 00 00     cmpwi   cr7,r0,0
>>>>     54:       40 9c 00 18     bge     cr7,6c <db_delete_field_log+0x38>
>>>>     58:       80 1f 00 18     lwz     r0,24(r31)
>>>>     5c:       2f 80 00 00     cmpwi   cr7,r0,0
>>>>     60:       41 9e 00 0c     beq     cr7,6c <db_delete_field_log+0x38>
>>>>     64:       7c 09 03 a6     mtctr   r0
>>>>     68:       4e 80 04 21     bctrl
>>>>     6c:       3d 20 00 00     lis     r9,0
>>>>     70:       7f e4 fb 78     mr      r4,r31
>>>>     74:       80 69 00 00     lwz     r3,0(r9)
>>>>     78:       3d 20 00 00     lis     r9,0
>>>>     7c:       39 29 00 00     addi    r9,r9,0
>>>>     80:       7d 29 03 a6     mtctr   r9
>>>>     84:       4e 80 04 21     bctrl
>>>>     88:       80 01 00 14     lwz     r0,20(r1)
>>>>     8c:       83 e1 00 0c     lwz     r31,12(r1)
>>>>     90:       38 21 00 10     addi    r1,r1,16
>>>>     94:       7c 08 03 a6     mtlr    r0
>>>>     98:       4e 80 00 20     blr
>>> 
>>> 
>>> 
>>> From my local build for RTEMS-mvme3100
>>> 
>>> 
>>>> void db_delete_field_log (db_field_log *pfl)
>>>> {
>>>> 
>>>>   if (pfl) {
>>>> 
>>>>  0:   7c 64 1b 79     mr.     r4,r3
>>>>  4:   4d 82 00 20     beqlr
>>>> 
>>>>       /* Free field if reference type field log and dtor is set */
>>>>       if (pfl->type == dbfl_type_ref && pfl->u.r.dtor)
>>>>       pfl->u.r.dtor(pfl);
>>>> 
>>>>  8:   89 24 00 00     lbz     r9,0(r4)
>>>>  c:   71 29 00 80     andi.   r9,r9,128
>>>> 
>>>> 10:   40 82 00 10     bne     20 <db_delete_field_log+0x20>
>>>> 
>>>>       /* Free the field log chunk */
>>>>       freeListFree(dbevFieldLogFreeList, pfl);
>>>> 
>>>> 14:   3d 20 00 00     lis     r9,0
>>>> 18:   80 69 00 00     lwz     r3,0(r9)
>>>> 1c:   48 00 00 00     b       1c <db_delete_field_log+0x1c>
>>>> 
>>>>       if (pfl->type == dbfl_type_ref && pfl->u.r.dtor)
>>>>       pfl->u.r.dtor(pfl);
>>>> 
>>>> 20:   81 24 00 50     lwz     r9,80(r4)
>>>> 24:   2f 89 00 00     cmpwi   cr7,r9,0
>>>> 28:   41 be ff ec     beq     cr7,14 <db_delete_field_log+0x14>
>>>> 
>>>> {
>>>> 
>>>> 2c:   94 21 ff f0     stwu    r1,-16(r1)
>>>> 30:   7c 08 02 a6     mflr    r0
>>>> 
>>>>       if (pfl->type == dbfl_type_ref && pfl->u.r.dtor)
>>>>       pfl->u.r.dtor(pfl);
>>>> 
>>>> 34:   7d 29 03 a6     mtctr   r9
>>>> 
>>>> {
>>>> 
>>>> 38:   90 01 00 14     stw     r0,20(r1)
>>>> 3c:   90 81 00 08     stw     r4,8(r1)
>>>> 
>>>>       if (pfl->type == dbfl_type_ref && pfl->u.r.dtor)
>>>>       pfl->u.r.dtor(pfl);
>>>> 
>>>> 40:   4e 80 04 21     bctrl
>>>> 
>>>>   }
>>>> 
>>>> }
>>>> 
>>>> 44:   80 01 00 14     lwz     r0,20(r1)
>>>> 
>>>>       freeListFree(dbevFieldLogFreeList, pfl);
>>>> 
>>>> 48:   3d 20 00 00     lis     r9,0
>>>> 
>>>>       if (pfl->type == dbfl_type_ref && pfl->u.r.dtor)
>>>>       pfl->u.r.dtor(pfl);
>>>> 
>>>> 4c:   80 81 00 08     lwz     r4,8(r1)
>>>> 
>>>>       freeListFree(dbevFieldLogFreeList, pfl);
>>>> 
>>>> 50:   80 69 00 00     lwz     r3,0(r9)
>>>> 
>>>> }
>>>> 
>>>> 54:   7c 08 03 a6     mtlr    r0
>>>> 58:   38 21 00 10     addi    r1,r1,16
>>>> 
>>>>       freeListFree(dbevFieldLogFreeList, pfl);
>>>> 
>>>> 5c:   48 00 00 00     b       5c <db_delete_field_log+0x5c>
> 
> 
> 

References:
Re: Bus errors accessing VME with base 7.0.6.1 and latest synApps modules Michael Davidsaver via Tech-talk
Re: Bus errors accessing VME with base 7.0.6.1 and latest synApps modules Mark Rivers via Tech-talk
Re: Bus errors accessing VME with base 7.0.6.1 and latest synApps modules Jörn Dreyer via Tech-talk

Navigate by Date:
Prev: Re: Strange memory leak with ADAravis Mark Rivers via Tech-talk
Next: Re: Bus errors accessing VME with base 7.0.6.1 and latest synApps modules Mark Rivers via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
Navigate by Thread:
Prev: Re: Bus errors accessing VME with base 7.0.6.1 and latest synApps modules Maren Purves via Tech-talk
Next: Re: Bus errors accessing VME with base 7.0.6.1 and latest synApps modules Mark Rivers via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
ANJ, 14 Sep 2022 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·