Re: DBR_VFIELD and complex field types

[Timo Korhonen via Core-talk <core-talk at aps.anl.gov>]

Hi everybody,

Since this work has been done on a contract from ESS, and there were some reasons why were interested on having this done,
I think it would be a good idea to explain in the next EPICS core telecon what the contract was about and what is contained.

Could that be put on the agenda of the next meeting?

Timo

On 09/06/20 16:46, "Core-talk on behalf of Michael Davidsaver via Core-talk" <core-talk-bounces at aps.anl.gov on behalf of core-talk at aps.anl.gov> wrote:

    On 6/9/20 12:33 AM, Ralph Lange via Core-talk wrote:
    > Hi Michael,
    > 
    > Before looking closer at the implementation, I would like to understand what you are trying to achieve.
    > So: What are you trying to achieve?
    
    With this goal, a good question to ask ;)
    
    I'll start with two examples.  One in detail, and another in passing.
    
    First, I want to avoid copying array data into and out of record fields.
    The performance is poor, and the API is awkward (NORD vs. NELM, NOVA vs. NEVA).
    
    In the context of the PVA modules, array data is handled via the shared_vector<>
    container.  This c++ class is meant to behave somewhat like std::vector but
    with reference counting instead of copying (the "shared" part of the name).
    
    So I want to create a way for my driver to allocate and fill a buffer, then
    pass it through the database layer without copying.  I especially want to
    avoid any per-client copying.
    
    This led me to create svectorinRecord, which is analogous to aaiRecord
    or waveformRecord.  So where waveformRecord has
    
    >     field(VAL,DBF_NOACCESS) {
    >         ...
    >         special(SPC_DBADDR)
    >         extra("void *		val")
    >     }
    
    svectorinRecord has
    
    >     field(VAL,DBF_NOACCESS) {
    >         ...
    >         special(SPC_DBADDR)
    >         extra("::epics::pvData::shared_vector<const void> val")
    >     }
    
    https://github.com/mdavidsaver/pva2pva/blob/fcdc0e53bd19b5b1bfb51620b6792b1d77082bfa/pdbApp/svectorinRecord.dbd#L33-L39
    
    So a device support for this new record type will (somehow) acquire
    a shared_vector instance pointing to a (const) buffer and assign
    this reference to the VAL field.
    
    So far, I have the ability to then move this reference through a
    DB_LINK and/or move it to a PVA server.
    
    > record(svectorin, "rec:a") {
    >     field(FLNK, "rec:b")
    >     ...
    > }
    > record(svectorin, "rec:b") {
    >     field(INP, "rec:a") # array not copied!
    >     ...
    > }
    
    
    
    The second example I'll only mention in passing because it is the same idea,
    with similar mechanics and a different VAL.  Instead of a single array, make
    VAL a structure containing multiple arrays.  eg. multiple channels of an ADC,
    or multiple columns of a table.
    
    So far I've only explored this idea with the awkwardly general (and generally
    awkward) pvstructinRecord.  Here the VAL field is a PVStructurePtr, which is
    a reference to a PVA data structure.  It is left to device support to
    fill in a specific structure definition during init_record().
    
    
    > Thanks a lot,
    > ~Ralph
    > 
    > 
    > On Tue, 9 Jun 2020 at 01:19, Michael Davidsaver via Core-talk <core-talk at aps.anl.gov <mailto:core-talk at aps.anl.gov>> wrote:
    > 
    >     I'd like to ask for some review of an idea I've been trying out
    >     (funding courtesy of ESS) to be able to usefully represent
    >     complex types as database fields.  My immediate application is
    >     to the PVA data containers shared_vector and PVStructure,
    >     although there are others.
    > 
    >     The way I've done this works, but wasn't my first idea.  I won't
    >     call it a hack, but it's not far removed.  So I'm open to alternate
    >     ways to accomplish this.  (though I do think a solution is needed)
    > 
    >     In short, I add a new DBR_VFIELD to the database module without
    >     adding any DBF_* code.  A short (and ugly) usage example is:
    > 
    >     > DBADDR *paddress = ...;
    >     > shared_vector<const void> aval;
    >     > struct VField {
    >     >     const VFieldType *vtype;
    >     >     shared_vector<const void> *aval;
    >     > } varg = {&vfSharedVector, &aval};
    >     > dbGet(paddress, DBR_VFIELD, NULL, &varg, NULL);
    > 
    > 
    >     This project involved changes to multiple modules in order to make
    >     something testable.  Even the changes to Base come in two parts.
    >     Each has accompanying discussion.
    > 
    >     1. Add alarm message and time tag to dbCommon in Base
    > 
    >     Not directly related to DBR_VFIELD, but adds an alarm
    >     message string, and time tag a la. PVA.  Also, adds
    >     a 'utag' server side filter which can apply a bitmask
    >     to the time tag.
    > 
    >     https://github.com/epics-base/epics-base/pull/71
    > 
    >     2. Add DBR_VFIELD in Base
    > 
    >     Also adds a new stdstringinRecord type as an example.
    > 
    >     https://github.com/mdavidsaver/epics-base/pull/1
    > 
    >     3. Apply DBR_VFIELD to QSRV
    > 
    >     Adds svectorinRecord where VAL is shared_vector<const void>
    >     aka reference counted pointer to an immutable array.
    > 
    >     https://github.com/mdavidsaver/pva2pva/pull/3
    > 
    >     4. Add device support for svectorinRecord to a driver
    > 
    >     I picked this TCP socket based driver mainly because
    >     I created a easy simulator.  The driver is changed to
    >     use shared_vector internally instead of copying arrays.
    > 
    >     https://github.com/mdavidsaver/pscdrv/pull/1
    > 
    > 
    >     The end result is driver which handles waveform data more
    >     efficiently, and can also pass a time tag from the (simulated)
    >     device, which can then be used to filter out some monitor updates
    >     (eg. beam vs. no beam).
    >