Re: Issues with PVXS

[Michael Davidsaver via Tech-talk <tech-talk at aps.anl.gov>]

Hi Mark,

On 1/19/26 3:18 PM, Mark Rivers via Tech-talk wrote:

Hi Ralph,

• I would be really surprised if PXVS changed any user-configured data types. Please verify that the new AD plugin really calls PVXS with data structures that are different from what you see on the network.

I'm sure you are correct.  I believe the issue is that PVXS is promoting the datatype during an assignment in the plugin code.  This is not obvious.

This is the line that sets the codec.parameters field in ntndArrayConverterPvxs.cpp:

https://github.com/areaDetector/ADCore/blob/6e950ed29d4372bc0a88836a007446f33d8facd6/ADApp/ntndArrayConverterSrc/ntndArrayConverterPvxs.cpp#L361

    m_value["codec.parameters"] = (int32_t) NDDataTypeToScalar[src->dataType];

m_value is of type pvxs::Value.  It is doing its best to define codec.parameters as being of type int32_t.

However, as far as I can tell PXVS use the StoreType enum to define the datatypes that can be stored.

StoreType is relevant for the in-memory representation, and does store scalar integers as (u)int64.  (wasting up to 7 bytes!)  The wire representation is determined by the TypeCode enum, which includes the full range of PVA supported scalar integer types.  As does the in-memory representation for arrays (see also the ArrayType enum).

StoreType and ArrayType mostly do not cross over.  The exception is when assigning a variant union (aka ANY).  Which is how "codec.parameters" is defined.  Using the "Value::operator=()" will automatically promote integers, as you have found.  Arrays are not promoted.

PVXS does provide a way to assign an ANY with an explicit type.

Value paramval = TypeDef(TypeCode::Int32).create();
paramval = NDDataTypeToScalar[src->dataType];
m_value["codec.parameters"].from(paramval);

To make this slightly more efficient, avoid re-creating the type each time.  (TypeDef is relatively expensive)

// once
this->paramval = TypeDef(TypeCode::Int32).create();

// each time
Value paramval(this->paramval.cloneEmpty());
paramval = NDDataTypeToScalar[src->dataType];
m_value["codec.parameters"].from(paramval);

..

So the issue is that PVXS is not allowing the creation of the same data structures that PVA does, at least not with the Value class.

PVXS supports the full range of scalar integer field types.

There is a further issue.  NTNDArrays can be compressed.  For compressed arrays the datatype is effectively uint8_t, but the actual datatype of the original array must be conveyed.  By convention this is stored in codec.parameters as an int.  The int is one of the values in the ScalarType enum.

I completely agree with Ralph that robust client code interpreting PVA structures should be written to implicitly convert between numeric types when accessing scalar fields.  pvDataCPP/Java made it complex for API users to do this, leading to breakages like you are seeing.  PVXS makes it complex for users not to do this.  (I can't speak directly for Kay's core.pva library, but I suspect the same is true)

The ScalarType enum is defined here in PVA:

https://github.com/epics-base/pvDataCPP/blob/f66492ce4bad7e513cf048e3d5c697a127d2c6ba/src/pv/pvIntrospect.h#L221

For Java it is defined here:

https://github.com/epics-base/epicsCoreJava/blob/master/pvDataJava/src/org/epics/pvdata/pv/ScalarType.java

However, I cannot find an equivalent enum in PVXS.  So ntndArrayConverterPvxs.cpp currently has to include pv/pvIntrospect.h from PVA to get the required definition.  This is ugly since it is mixing code from PVA and PVXS.

In PVXS, the ScalarType enum, along with its cousins, are collapsed down into the single TypeCode enum.

https ://github.com/epics-base/pvxs/blob/48b260d008bd1d65c5f132c54523e27b41d0f4c3/src/pvxs/data.h#L154-L190