Re: Problem with huge waveforms in EPICS 7

["Zimoch Dirk \(PSI\) via Core-talk" <core-talk at aps.anl.gov>]

Confirmed with my test setup.
The real cameras on Windows will be tested soon but I am confident.

Dirk

On Mon, 2022-06-27 at 20:53 +0000, Mark Rivers wrote:
> I tested Michael's patch and it fixes the problem when I run the ADSimDetector test.
>  
> (base) J:\epics\devel\asyn\iocBoot\ioctestAsynPortDriver>python monitor_test_ad.py
> 2022-06-27 15:50:43.877772: Adding BIGARRAY callback function
> 2022-06-27 15:50:43.878761: Sleeping for 5 seconds
> 2022-06-27 15:50:45.787905: pv_callback counter=1, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=99
> 2022-06-27 15:50:48.880187: Starting acquisition
> 2022-06-27 15:50:48.901216: Waiting for acquisition to complete
> 2022-06-27 15:50:50.290341: pv_callback counter=2, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=100
> 2022-06-27 15:50:51.699526: pv_callback counter=3, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=112
> 2022-06-27 15:50:52.709903: pv_callback counter=4, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=126
> 2022-06-27 15:50:53.127619: pv_callback counter=5, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=136
> 2022-06-27 15:50:53.545305: pv_callback counter=6, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=140
> 2022-06-27 15:50:53.956352: pv_callback counter=7, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=144
> 2022-06-27 15:50:54.371361: pv_callback counter=8, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=148
> 2022-06-27 15:50:55.201308: pv_callback counter=9, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=152
> 2022-06-27 15:50:55.626239: pv_callback counter=10, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=161
> 2022-06-27 15:50:56.240083: pv_callback counter=11, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=165
> 2022-06-27 15:50:56.654982: pv_callback counter=12, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=171
> 2022-06-27 15:50:57.071780: pv_callback counter=13, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=175
> 2022-06-27 15:50:57.495975: pv_callback counter=14, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=179
> 2022-06-27 15:50:57.909298: pv_callback counter=15, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=183
> 2022-06-27 15:50:58.536483: pv_callback counter=16, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=188
> 2022-06-27 15:50:59.158556: pv_callback counter=17, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=194
> 2022-06-27 15:50:59.574255: pv_callback counter=18, pvName=13SIM1:image1:ArrayData, size=20000000, value=[0]=199
> 2022-06-27 15:50:59.576321: Acquisition complete
> 2022-06-27 15:50:59.576321: Sleeping for 60 seconds
> 2022-06-27 15:51:59.579580: Exiting
>  
> There are no callbacks after acquisition is complete.
>  
> Mark
>  
>  
> -----Original Message-----
> From: Core-talk <core-talk-bounces at aps.anl.gov> On Behalf Of Michael Davidsaver via Core-talk
> Sent: Monday, June 27, 2022 9:51 AM
> To: Zimoch Dirk (PSI) <dirk.zimoch at psi.ch>
> Cc: core-talk at aps.anl.gov
> Subject: Re: Problem with huge waveforms in EPICS 7
>  
> On 6/27/22 06:56, Zimoch Dirk (PSI) via Core-talk wrote:
> > On Mon, 2022-06-27 at 15:12 +0200, Zimoch Dirk wrote:
> >> The problem appears between commit 4ab98081 and 56f05d72. (The two
> >> commits in between do not produce stable code.)
> >> 
> >
> > Ben did some code refactoring of db_field_log which changed array handling.
> >
> > In particular in dbEvent.c the following snipped disappeared from
> > db_queue_event_log()
> >
> >      /*
> >       * if we have an event on the queue and both the last
> >       * event on the queue and the current event are emtpy
> >       * (i.e. of type dbfl_type_rec), simply ignore duplicate
> >       * events (saving empty events serves no purpose)
> >       */
> >      if (pevent->npend > 0u &&
> >          (*pevent->pLastLog)->type == dbfl_type_rec &&
> >          pLog->type == dbfl_type_rec) {
> >          db_delete_field_log(pLog);
> >          UNLOCKEVQUE (ev_que);
> >          return;
> >      }
> >
> > I have added a debug printf in this code and see that it triggers
> > whenever a frame is dropped. That is about 3 out of 4 frames in my test case.
> >
> > It is not trivial to put back the code because dbfl_type_rec disappeared from enum dbfl_type too.
>  
> I think the (untested) equivalent would now be:
>  
> > diff --git a/modules/database/src/ioc/db/dbEvent.c
> > b/modules/database/src/ioc/db/dbEvent.c
> > index 0164f55b9..480784d1e 100644
> > --- a/modules/database/src/ioc/db/dbEvent.c
> > +++ b/modules/database/src/ioc/db/dbEvent.c
> > @@ -789,6 +789,18 @@ static void db_queue_event_log (evSubscrip
> > *pevent, db_field_log *pLog)
> > 
> >      LOCKEVQUE (ev_que);
> > 
> > +    /* if we have an event on the queue and both the last
> > +     * event on the queue and the current event are empty,
> > +     * simply ignore duplicate events.
> > +     */
> > +    if (pevent->npend > 0u
> > +            && !dbfl_has_copy(*pevent->pLastLog)
> > +            && !dbfl_has_copy(pLog)) {
> > +        db_delete_field_log(pLog);
> > +        UNLOCKEVQUE (ev_que);
> > +        return;
> > +    }
> > +
> >      /*
> >       * add to task local event que
> >       */