Experimental Physics and
Industrial Control System

Andrew Johnson <[email protected]> · Mon, 2 Feb 2009 16:01:03 -0600

Hi Ben,

Thanks for clarifying your collective thoughts on this.

On Monday 02 February 2009 08:05:12 Benjamin Franksen wrote:
>
> If I want to get the last time the record has been processed, I access the
> VAL field. This is directly supported by it being the default if no field
> name is given. Accessing any other field means I care for this field
> specifically, not the record in general.

This is logical, although not necessarily complete.  Complex record types may 
want to have synchronized timestamps for different sets of fields without 
having to read the time whenever it updates a field.  Reading the current 
time can be a fairly expensive operation, it involves taking at least one 
mutex and often two, depending on the active time provider, and probably a 
system call to the OS.  Thus something like the Motor record may want to read 
the time just once whenever it updates a whole series of fields for a new 
motor position say.

> It certainly makes sense to insist on well-defined semantics for timestamps
> (as for anything else, in fact). The current meaning ("time when record was
> last processed") is well-defined, clear and simple. But so would be "time
> when IOC was last booted". The question is: how useful are these meanings?
> Obviously, the former is more useful than the latter, since it gives you
> more detailed and more relevant information. So there are good reasons to
> chose the latter over the former, if it can be implemented at a reasonable
> cost (in memory, run-time, and development effort). I argue that "time when
> field was last updated" is an even more (really much more) useful way to
> define what the timestamp means and it is equally well-defined.

In general I agree with your last sentence, but with some caveats.

What do a record's TSE and TSEL fields mean under this regime?  They are 
currently used to read the record's timestamp from another record, to read 
the timestamp from a particular hardware event system event number, or to 
allow the device support to set the record's TIME field.  Currently I can 
arrange for a particular record to use a completely different source of time  
for its timestamps (which may have little or no connection with wall-clock 
time), and there are labs using this extensively.  Thus I think we'd have to 
change the TSE/TSEL mechanism because it isn't flexible enough to support 
your Field Timestamps proposal in these circumstances.

This change in timestamp behavior should only be implemented in a 2nd-level 
version number upgrade, I wouldn't want it to go into a R3.14.x release.

> We had some discussion here between Ralph, Götz, and me, and we think that
> it is possible to implement this in base at very reasonable cost, providing
> complete backward compatibility (by default), and requiring no changes to
> existing record support code (under moderate assumptions about what record
> and device support do).

Unfortunately I don't think those assumptions are right (see below)...

> We propose to introduce a switch, one per IOC (one per record instance
> would be possible, too, but probably overkill), which lets you select
> between
>
>  (a) old behaviour (timestamp = last record process time)
>  (b) new behaviour (timestamp = last update to field)
>  (c) the maximum of (a) and (b) (timestamp = last update to field or
>      last record process time if this has occurred later)
>
> for all read accesses (ultimately calls to dbGet and CA monitors).

Ned Arnold would like another setting too, on a per-record basis: timestamp = 
last time the value was actually different, so an old timestamp would survive 
through record processing if the result was the same as before.

> The default would be (a), so that compatible behaviour is guaranteed. The
> default could be changed to (b) for future major releases. Whether (c) is
> really useful is debatable, but I wanted to mention it as it would be cheap
> to realize.
>
> Our idea for an efficient implementation is based on the observation that
> in a typical EPICS database there are only very few fields per record that
> actually receive dbPuts (or get updated directly from within base). Only
> for these fields is it necessary to provide storage for the time of the
> last update. This storage will be allocated on demand; each record must
> maintain a (typically very short) list of (field index, timestamp) pairs;
> let us call it the 'field update list' or short 'update list'. When reading
> the timestamp for a field, first search the update list and use the
> contained stamp if one is found, otherwise use the TIME field of the
> record. Memory management for the list nodes can and should use a freelist,
> all nodes are of the same type (and thus size).

That part seems reasonable, modulo Till's remarks.

> Each time the EPICS base updates a record field that can be read from the
> outside it must also update the corresponding timestamp for the field in
> this list, adding a new node if none is found. Each time the record's
> process routine updates one of its fields it must /remove/ the field from
> the list, so that the record's timestamp is used when the field's timestamp
> is requested via dbGet. To avoid having to change existing record supports
> this could be done by db_post_events, where we assume that record support
> calls db_post_events whenever record or device support has changed a field
> that can be read from the outside.
>
> The only problem is that record support changes fields (and calls
> db_post_events) not only in process but also in special. But special gets
> called only directly from base, so base must set a flag in the record to
> indicate to db_post_events that special processing is active (and not
> regular processing), so that it can add timestamp to the update list.
>
> And that's it, nothing overly complex, AFAICS.

The db_post_events() routine is used inside dbAccess.c and recGbl.c as the 
standard mechanism to indicate to CA that a value has been updated.  It can 
also be used by device support that may set other record fields directly; for 
example, it is not uncommon now for an MBBI/O device support to set the 
string fields to the set of choices that its hardware can give or accept.  
This sort of thing could even happen outside of the process() or special() 
routines in the device support's background thread, as long as it locks the 
record first.

There's also the issue that I believe [but haven't traced through to check] 
that special() can be called while a record is still executing its process() 
routine; process() calls dbPutLink() which directly or indirectly modifies a 
field in the same record that is marked special.  It's even possible that 
special could call dbPutLink() and thus end up recursing back to itself.  
Your flag would need to take account of that.

I like your mechanism for recording the updated timestamps, but I don't think 
it would be possible to use db_post_events() in the way you want to, it gets 
called from so many places and the complexity of understanding all of these 
is somewhat daunting.

We could add a new get_timestamp() routine to the RSET without breaking 
anything, and use this to allow record types to provide additional timestamps 
for use by specific (groups of) fields; I suspect the Motor record might want 
to do that for example.  However this is to some extent orthogonal to the 
issue you're wanting to fix.

Keep up the discussion, this is worth talking about.

- Andrew
-- 
The best FOSS code is written to be read by other humans -- Harold Welte

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System