Issues with PVXS

[Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>]

Folks,

areaDetector/ADCore was recently enhanced to support a new NDPluginPVXS plugin which uses the PVXS library to export NTNDArrays.    The existing NDPluginPva uses the original PVA support which is part of EPICS base 7.x.  James Souter was largely responsible for this effort.  One can now build ADCore to support PVXS, or PVA, or both, by editing CONFIG_SITE.local.  One switch to use either PVXS or PVA when the IOC starts by changing commonPlugins.cmd.

I expected that using PVXS would not require any modifications to existing pvAccess clients that receive NTNDArrays over pvAccess.  These clients include: 

• The ImageJ NTNDAViewer plugin
• The CSS/Phoebus Image widget
• pvaDriver

However, that is not what I observe.

• The ImageJ viewer only works with uncompressed monochrome images, it does not work with RGB or compressed images.  This is documented in these Github issues: https://github.com/areaDetector/ADCore/issues/561, https://github.com/areaDetector/ADCore/issues/562
• CSS/Phoebus only works with uncompressed images and does not work with LZ or JPEG compressed images.  When the IOC sends JPEG compressed arrays it crashes the IOC.  That is documented in this Github issue: https://github.com/areaDetector/ADCore/issues/556
• pvaDriver does not create identical NDArrays to those on the originating IOC, the NDAttribute datatypes are changed.  This is documented in this Github issue: https://github.com/areaDetector/ADCore/issues/560

The root cause of these problems is shown by the differences seen in an NTNDArray received with pvget  depending on whether the pvAccess server is PVA or PVXS.

This is pvget when using ADSimDetector and NDPluginPva:

(base) [epics@corvette ~]$ pvget 13SIM1:Pva1:Image

13SIM1:Pva1:Image epics:nt/NTNDArray:1.0

    union value

        ubyte[]  [69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,

...

    codec_t codec

        string name

        any parameters

            int  5

    long compressedSize 1048576

    long uncompressedSize 1048576

    dimension_t[] dimension

        dimension_t

            int size 1024

            int offset 0

            int fullSize 1024

            int binning 1

            boolean reverse false

        dimension_t

            int size 1024

            int offset 0

            int fullSize 1024

            int binning 1

            boolean reverse false

    int uniqueId 69

    time_t dataTimeStamp 2026-01-18 11:24:25.940

        long secondsPastEpoch 1768757065

        int nanoseconds 939971208

        int userTag 0

    epics:nt/NTAttribute:1.0[] attribute

        epics:nt/NTAttribute:1.0

            string name ColorMode

            any value

                int  0

            string descriptor Color mode

            int sourceType 0

            string source Driver

        epics:nt/NTAttribute:1.0

            string name Int8

            any value

                byte  -8

            string descriptor Int8 attribute

            int sourceType 3

            string source myAttrFunct1

        epics:nt/NTAttribute:1.0

            string name UInt8

            any value

                ubyte  8

            string descriptor UInt8 attribute

            int sourceType 3

            string source myAttrFunct1

        epics:nt/NTAttribute:1.0

            string name Int16

            any value

                short  -16

            string descriptor Int16 attribute

            int sourceType 3

            string source myAttrFunct1

        epics:nt/NTAttribute:1.0

            string name UInt16

            any value

                ushort  16

            string descriptor UInt16 attribute

            int sourceType 3

            string source myAttrFunct1

...

Note that codec.parameters and the ColorMode attribute are of type "int".  The numeric attributes have datatypes "byte", "ubyte", "short", and "ushort".

This is pvget when using ADSimDetector and NDPluginPVXS:

13SIM1:Pva1:Image epics:nt/NTNDArray:1.0

    union value

        ubyte[]  [28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,

...

    codec_t codec

        string name

        any parameters

            long  5

    long compressedSize 1048576

    long uncompressedSize 1048576

    int uniqueId 29

    time_t dataTimeStamp 2026-01-18 11:38:30.780

        long secondsPastEpoch 1768757910

        int nanoseconds 779778718

    time_t timeStamp 2026-01-18 11:38:30.780

        long secondsPastEpoch 1768757910

        int nanoseconds 779778610

    dimension_t[] dimension

        dimension_t

            int size 1024

            int offset 0

            int fullSize 1024

            int binning 1

            boolean reverse false

        dimension_t

            int size 1024

            int offset 0

            int fullSize 1024

            int binning 1

            boolean reverse false

    epics:nt/NTAttribute:1.0[] attribute

        epics:nt/NTAttribute:1.0

            string name ColorMode

            any value

                long  0

            string[] tags []

            string descriptor Color mode

            alarm_t alarm

                int severity 0

                int status 0

                string message

            time_t timeStamp

                long secondsPastEpoch 0

                int nanoseconds 0

                int userTag 0

            int sourceType 0

            string source Driver

        epics:nt/NTAttribute:1.0

            string name Int8

            any value

                long  -8

            string[] tags []

            string descriptor Int8 attribute

            alarm_t alarm

                int severity 0

                int status 0

                string message

            time_t timeStamp

                long secondsPastEpoch 0

                int nanoseconds 0

                int userTag 0

            int sourceType 3

            string source myAttrFunct1

        epics:nt/NTAttribute:1.0

            string name UInt8

            any value

                ulong  8

            string[] tags []

            string descriptor UInt8 attribute

            alarm_t alarm

                int severity 0

                int status 0

                string message

            time_t timeStamp

                long secondsPastEpoch 0

                int nanoseconds 0

                int userTag 0

            int sourceType 3

            string source myAttrFunct1

        epics:nt/NTAttribute:1.0

            string name Int16

            any value

                long  -16

            string[] tags []

            string descriptor Int16 attribute

            alarm_t alarm

                int severity 0

                int status 0

                string message

            time_t timeStamp

                long secondsPastEpoch 0

                int nanoseconds 0

                int userTag 0

            int sourceType 3

            string source myAttrFunct1

        epics:nt/NTAttribute:1.0

            string name UInt16

            any value

                ulong  16

            string[] tags []

            string descriptor UInt16 attribute

            alarm_t alarm

                int severity 0

                int status 0

                string message

            time_t timeStamp

                long secondsPastEpoch 0

                int nanoseconds 0

                int userTag 0

            int sourceType 3

            string source myAttrFunct1

...

Note that codec.parameters and the ColorMode attribute are of type "long".  The numeric attributes have datatypes "long", "ulong", "long", and "ulong".

The ImageJ NTNDAViewer plugin is written to expect codec.parameters and ColorMode to both be of type "int", and throws errors if they are not.

I suspect that the CSS/Phoebus widget is generating errors for compressed images because it expects codec.parameters to be an int, but it is not.  I have not tracked down why JPEG compression crashes the IOC, and not CSS.

The bottom line is that PVXS promotes all numeric scalars to 64-bit datatypes.  I have not shown it above, but Int32 and UInt32 are also promoted to long, and Float32 is promoted to Float64.

It appears to me that PVXS cannot produce the same network byte stream that PVA does.

I understand that we can rewrite the ImageJ and CSS/Phoebus Java code to convert codec.parameters and ColorMode from long to int.  But the idea of a network protocol is that replacing the server with a different implementation should not require any modifications to the clients.

For the pvaDriver there is no way to fix the fact that PVXS modifies the NDArrays such that the original array cannot be recovered, because the NDAttribute data types have been changed.

Mark