Re: PVD/PVA: Examples for multi-dimensional arrays?

[Marty Kraimer via Core-talk <core-talk@aps.anl.gov>]

On 11/21/18 11:10 AM, Ralph Lange via Core-talk wrote:

Hi Mark,

On Wed, 21 Nov 2018 at 16:37, Mark Rivers <rivers@cars.uchicago.edu> wrote:

What do you mean by "support"?  NTNDArrays are certainly N-dimensional.  Do you want the API to allow extracting rows/columns, etc.?

Hm.

The problem with leaving n-dim support to the application is that every application using n-dim arrays will have to do its own implementation of the accessors. Also, because of that, these n-dim arrays will likely not be compatible between applications.

Does that sound like I am asking for a new normative type?

Cheers,
~Ralph

At the very beginning of PVData development, i. e. the JavaIOC, a deliberate decision was made that pvData would only support 1 dim arrays.
The problem with supporting more dimensions is how many and how to define each dimension.
Just consider a 2d array. It it a 2d matrix? Is it a sparse  matrix? Is it a table?

I also remember a request for pvData defining a map type.
Where do we stop?

At the very beginning it was decided that the following types would be supported

1) scalars of numeric , boolean, and string.
2) 1d arrays of each scalar type
3) structures
4) 1d structure arrays

This was defined in 2006.

The only types that were added later are:

5) union
6) 1d union array.

It is ONLY data of  above types that need to be supported by the pva network protocol.

Also from the beginning it was assumed that more complex types would be supported via structure definitions.
This later became the normative types.
Note that normative types already has definiintions for NTTable, NTMatrix, and NTHistogram.
Each of these is a different type of 2d array.
NTNDArray includes a multi dimensional array plus lots more.

No normative type required any changes to the network protocol.

Recently there was a discussion of pvAccess directly supporting  compression.
Although it was talking about pvAccess it would probably have to involve pvData.
For the same reason for not supporting multi deminsional array compression should NOT be directly supported by pvData/pvAccess.
What kind of compression?
lossless?, JPEG?, etc, etc.
Would each additional type require changes to the pva network protocol.
Note that areaDetecter already supports a few compression types.

Marty