Re: soft IOC string and array records

[Tim Mooney <mooney@aps.anl.gov>]

Kurt Biery wrote:
> At Fermilab, several of us are working on the data acquisition for NOvA, 
> a future neutrino experiment.  We would like to use process variables in 
> soft IOCs to pass string messages between processes in the system, and 
> we are trying to understand the best way to accomplish this.
>
> With EPICS Base R3.14.8.2, we have experimented with string and waveform 
> records and have noticed that messages are not always delivered to 
> monitor processes when the messages are quickly written to the PV.  I'll 
> include some of the details of our tests below, but the central issue 
> seems to be illustrated by the following snippet of code from 
> db_post_single_event_private in dbEvent.c:
>
>    /*
>     * if we have an event on the queue and we are
>     * not saving the current value (because this is a
>     * string or an array) then ignore duplicate
>     * events (saving them without the current value
>     * serves no purpose)
>     */
>    if (!event->valque && event->npend>0u) {
>        UNLOCKEVQUE(ev_que)
>        return;
>    }
>
> I don't understand all of the ramifications of this code comment, but it 
> certainly sounds like fast updates to string and array records are not 
> all expected to be delivered.  So, the question becomes:  if we want 
> updates to a string or waveform record (or something similar) to 
> *always* be delivered to monitors, what are our options?  We have 
> casually used CA servers when first learning EPICS, but have not yet 
> investigated whether they have this same feature.  Are CA servers an 
> option?  Or are there established ways to select the delivery of all 
> updates in a soft IOC?
>
> The tests that we ran used a soft IOC created with 'makeBaseApp.pl -t 
> ioc' and database records like the following:
>
> record(stringout, "test1/sampleString") {
>    field(DTYP, "Soft Channel")
> }
>
> record(waveform,"test1/sampleWaveform") {
>    field(DTYP,"Soft Channel")
>    field(NELM,"32")
>    field(FTVL,"UCHAR")
> }
>
> To update the PV value at a reasonably high rate, we simply made a copy 
> of caput.c and added a loop around the writing of the data as shown 
> below (the modification of the first character in the sbuf/pbuf array 
> allowed us to check for missing or duplicate records when observing the 
> updates with a camonitor client):
>
>    for (idx = 0; idx < 100; idx++) {
>        sprintf(sbuf[0], "%d", idx);
>        result = ca_array_put (dbrType, count, pvs[0].chid, pbuf);
>        result = ca_pend_io(caTimeout);
>        if (result == ECA_TIMEOUT) {
>            fprintf(stderr, "Write operation timed out: Data was not 
> written.\n");
>            return 1;
>        }
>    }
>
> For the waveform record test, we started the soft IOC, ran 'camonitor 
> test1/sampleWaveform', and then ran the modified caput several times 
> with 'caput -a test1/sampleWaveform 5 2 4 6 8 10'.  Some of the times 
> that we ran caput, we saw all 100 messages arrive at the camonitor.  
> Other times, only one or two of them did.
>
> Thanks in advance for any help that folks can provide with this,
> Kurt Biery

I think db_post_events() cannot do what you want.  Even if there is enough
time (network access, etc.) for all queued events to be dispatched, channel
access will not flush it's queue until there's enough information queued to
make a reasonable sized packet (or some time limit has passed).  If a new
string or array event is db_post'ed before the queue is flushed, the old
event will remain queued, and when it's time to send data, whatever's in
the actual string or array field at send time will be sent out to clients.

What you'd probably like is to know if an earlier posting is still queued,
and to be able to flush the queue if it is.  I think neither of these
things are possible at present. (You could, of course, arrange to db_post
enough drivel to ensure that the queue gets flushed, whatever it's state,
but that's probably too awful for any purpose other than testing.)

I think Kay's solution is about as good as you can get if you're
constrained by record-processing/posting rules, and have no control
over the data generator.  But if you can constrain the data generator,
you can get solutions that are more bulletproof.

I think a channel access client can be made to do what you want, but
the simplest solutions I can think of are still pretty arcane.  The
implementation would depend on whether message passing is driven by
the source or the receiver, and whether the source and receiver are
records or CA clients -- stuff like that.

--
Tim Mooney (mooney@aps.anl.gov) (630)252-5417
Beamline Controls & Data Acquisition Group
Advanced Photon Source, Argonne National Lab