Experimental Physics and
Industrial Control System

Hi Randall,

 

I have made the required changes to NDPluginStdArrays so that compressed NDArray data can be transmitted over Channel Access in $(P)$(R)ArrayData.

 

I just tested with ADSimDetector which was generating 1024x1024 UInt8 images at 10 Hz. NDPluginCodec was configured to compress with JPEG, 85% quality factor.  NDPluginStdArrays was getting its data from NDPluginCodec.

 

This is the result of camonitor of the first 500 elements in 13SIM:image1:ArrayData.  These are the first 3 arrays.

 

corvette:~/devel/areaDetector/ADCore>camonitor -#500 13SIM1:image1:ArrayData

13SIM1:image1:ArrayData 2020-01-24 16:16:12.255831 500 -1 -40 -1 -32 0 16 74 70 73 70 0 1 1 0 0 1 0 1 0 0 -1 -37 0 67 0 5 3 4 4 4 3 5 4 4 4 5 5 5 6 7 12 8 7 7 7 7 15 11 11 9 12 17 15 18 18 17 15 17 17 19 22 28 23 19 20 26 21 17 17 24 33 24 26 29 29 31 31 31 19 23 34 36 34 30 36 28 30 31 30 -1 -64 0 11 8 4 0 4 0 1 1 17 0 -1 -60 0 31 0 0 1 5 1 1 1 1 1 1 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9 10 11 -1 -60 0 -75 16 0 2 1 3 3 2 4 3 5 5 4 4 0 0 1 125 1 2 3 0 4 17 5 18 33 49 65 6 19 81 97 7 34 113 20 50 -127 -111 -95 8 35 66 -79 -63 21 82 -47 -16 36 51 98 114 -126 9 10 22 23 24 25 26 37 38 39 40 41 42 52 53 54 55 56 57 58 67 68 69 70 71 72 73 74 83 84 85 86 87 88 89 90 99 100 101 102 103 104 105 106 115 116 117 118 119 120 121 122 -125 -124 -123 -122 -121 -120 -119 -118 -110 -109 -108 -107 -106 -105 -104 -103 -102 -94 -93 -92 -91 -90 -89 -88 -87 -86 -78 -77 -76 -75 -74 -73 -72 -71 -70 -62 -61 -60 -59 -58 -57 -56 -55 -54 -46 -45 -44 -43 -42 -41 -40 -39 -38 -31 -30 -29 -28 -27 -26 -25 -24 -23 -22 -15 -14 -13 -12 -11 -10 -9 -8 -7 -6 -1 -38 0 8 1 1 0 0 63 0 -93 12 85 114 24 -86 -20 49 85 -40 98 -85 -80 -59 87 97 -118 -82 67 21 93 -122 42 -69 12 85 118 24 -86 -20 49 85 -56 98 -85 -80 -59 87 97 -118 -82 -61 21 93 -122 42 -71 12 85 -14 -49 -19 121 -5 70 -1 0 -62 63 -10 -49 -121 -97 15 47 -1 0 -30 117 -52 58 -74 -83 3 -1 0 -57 -113 102 -126 22 31 -14 -37 -77 56 -1 0 87 -48 126 -13 38 63 -119 97 -118 -82 -61 21 93 -122 42 -69 12 85 118 24 -86 -28 49 85 -40 98 -85 -80 -59 87 97 -118 -82 -61 21 92 -122 42 -69 12 85 118 24 -86 -20 49 85 -40 98 -85 -112 -59 87 97 -118 -82 -61 21 93 -122 42 -69 12 85 114 24 -86 -20 49 85 -40 98 -85 -80 -59 87 97 -118 -82 67 21 93 -122 42

 

13SIM1:image1:ArrayData 2020-01-24 16:16:12.352025 500 -1 -40 -1 -32 0 16 74 70 73 70 0 1 1 0 0 1 0 1 0 0 -1 -37 0 67 0 5 3 4 4 4 3 5 4 4 4 5 5 5 6 7 12 8 7 7 7 7 15 11 11 9 12 17 15 18 18 17 15 17 17 19 22 28 23 19 20 26 21 17 17 24 33 24 26 29 29 31 31 31 19 23 34 36 34 30 36 28 30 31 30 -1 -64 0 11 8 4 0 4 0 1 1 17 0 -1 -60 0 31 0 0 1 5 1 1 1 1 1 1 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9 10 11 -1 -60 0 -75 16 0 2 1 3 3 2 4 3 5 5 4 4 0 0 1 125 1 2 3 0 4 17 5 18 33 49 65 6 19 81 97 7 34 113 20 50 -127 -111 -95 8 35 66 -79 -63 21 82 -47 -16 36 51 98 114 -126 9 10 22 23 24 25 26 37 38 39 40 41 42 52 53 54 55 56 57 58 67 68 69 70 71 72 73 74 83 84 85 86 87 88 89 90 99 100 101 102 103 104 105 106 115 116 117 118 119 120 121 122 -125 -124 -123 -122 -121 -120 -119 -118 -110 -109 -108 -107 -106 -105 -104 -103 -102 -94 -93 -92 -91 -90 -89 -88 -87 -86 -78 -77 -76 -75 -74 -73 -72 -71 -70 -62 -61 -60 -59 -58 -57 -56 -55 -54 -46 -45 -44 -43 -42 -41 -40 -39 -38 -31 -30 -29 -28 -27 -26 -25 -24 -23 -22 -15 -14 -13 -12 -11 -10 -9 -8 -7 -6 -1 -38 0 8 1 1 0 0 63 0 -91 12 85 118 24 -86 -20 49 85 -40 98 -85 -112 -59 87 97 -118 -82 -61 21 93 -122 42 -69 12 85 114 24 -86 -20 49 85 -40 98 -85 -80 -59 87 97 -118 -82 67 21 93 -122 42 -69 12 85 -15 47 -19 121 -5 70 -1 0 -62 65 -10 -49 -121 -97 15 47 -1 0 -30 75 -52 58 -74 -83 3 -1 0 -57 -9 102 -126 22 31 -14 -57 -77 56 -1 0 89 -48 126 -17 38 79 -106 97 -118 -82 -61 21 93 -122 42 -71 12 85 118 24 -86 -20 49 85 -40 98 -85 -80 -59 87 33 -118 -82 -61 21 93 -122 42 -69 12 85 118 24 -86 -28 49 85 -40 98 -85 -80 -59 87 97 -118 -82 -61 21 92 -122 42 -69 12 85 118 24 -86 -20 49 85 -40 98 -85 -112 -59 87 97 -118 -82 -61 21 93 -122 42

 

13SIM1:image1:ArrayData 2020-01-24 16:16:12.450525 500 -1 -40 -1 -32 0 16 74 70 73 70 0 1 1 0 0 1 0 1 0 0 -1 -37 0 67 0 5 3 4 4 4 3 5 4 4 4 5 5 5 6 7 12 8 7 7 7 7 15 11 11 9 12 17 15 18 18 17 15 17 17 19 22 28 23 19 20 26 21 17 17 24 33 24 26 29 29 31 31 31 19 23 34 36 34 30 36 28 30 31 30 -1 -64 0 11 8 4 0 4 0 1 1 17 0 -1 -60 0 31 0 0 1 5 1 1 1 1 1 1 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9 10 11 -1 -60 0 -75 16 0 2 1 3 3 2 4 3 5 5 4 4 0 0 1 125 1 2 3 0 4 17 5 18 33 49 65 6 19 81 97 7 34 113 20 50 -127 -111 -95 8 35 66 -79 -63 21 82 -47 -16 36 51 98 114 -126 9 10 22 23 24 25 26 37 38 39 40 41 42 52 53 54 55 56 57 58 67 68 69 70 71 72 73 74 83 84 85 86 87 88 89 90 99 100 101 102 103 104 105 106 115 116 117 118 119 120 121 122 -125 -124 -123 -122 -121 -120 -119 -118 -110 -109 -108 -107 -106 -105 -104 -103 -102 -94 -93 -92 -91 -90 -89 -88 -87 -86 -78 -77 -76 -75 -74 -73 -72 -71 -70 -62 -61 -60 -59 -58 -57 -56 -55 -54 -46 -45 -44 -43 -42 -41 -40 -39 -38 -31 -30 -29 -28 -27 -26 -25 -24 -23 -22 -15 -14 -13 -12 -11 -10 -9 -8 -7 -6 -1 -38 0 8 1 1 0 0 63 0 -87 12 85 118 24 -86 -28 49 85 -40 98 -85 -80 -59 87 97 -118 -82 -61 21 92 -122 42 -69 12 85 118 24 -86 -20 49 85 -40 98 -85 -112 -59 87 97 -118 -82 -61 21 93 -122 42 -102 -26 123 91 11 25 -17 -17 -18 97 -75 -76 -74 -119 -90 -98 121 -92 9 28 72 -93 44 -20 -57 -123 80 1 36 -98 0 21 -7 -1 0 -5 87 126 -48 87 95 19 47 -92 -16 -81 -123 101 -102 -41 -63 -106 -46 -115 -52 65 73 53 71 83 -111 36 -128 -14 -79 2 1 72 -49 57 1 -40 110 -38 -79 -8 52 49 85 -56 98 -85 -80 -59 87 97 -118 -82 -61 21 93 -122 42 -71 12 85 118 24 -86 -20 49 85 -40 98 -85 -80 -59 87 33 -118 -82 -61 21 93 -122 42 -69 12 85 118 24 -86 -28 49 85 -40

 

I believe these data make sense as the start of a JPEG image.  The first 2 bytes of each array are -1 -40 which is hex FF D8.  According to Wikipedia (https://en.wikipedia.org/wiki/JPEG#Syntax_and_structure) FF D8 is JPEG “Start Of Image”, which makes sense for the first 2 bytes in stream.

 

The changes are on the master branch on Github.

 

Mark

 

 

 

Hi Randall,

 

Ø  So I was expecting that there was some way to just grab the jpeg out of NDPluginCodec as an array of bytes without any conversion going on. 

Ø  I presume the jpeg image exists somewhere within the plugin as a block of bytes.  NDFileJpeg(?) appears to be able to write the bytes of a jpeg out to a file which is pretty much what we want though we want it from a network connection read rather than directed to a file. 

Ø  How does a “compressed array” differ from just an array which happens to have a set of bytes that have been compressed? 

 

That depends on the compression scheme.  I believe JPEG compressed images themselves contain information about the data type, image dimensions and the color scheme (mono, RGB).  But other compression schemes (Blosc, LZ4, BSLZ4) are just a stream of bytes, and additional metadata is required to know how to interpret the data (data type, image dimensions, mono/RGB), etc.

 

When using Channel Access that additional metadata is in the form of additional EPICS PVs.  When using pvAccess that metadata is contained in the NTNDArray object which is transmitted atomically over the wire.

 

Ø  I presume knowing that the array data is compressed, how it’s compressed, etc is important if I want to pass it to another module that wants to work on it as an image. 

Ø  But here we don’t want to pass it on except to a client that knows what it is and how to deal with it.  We just want the bytes.

 

With Channel Access the waveform record will contain just the JPEG bytes, as will the “value” member of the NTNDArray structure.

 

I will work on making NDPluginStdArrays able to work with compressed arrays.

 

Mark

 

 

-----Original Message-----
From: Randall Cayford <[email protected]>
Sent: Friday, January 24, 2020 12:40 PM
To: Mark Rivers <[email protected]>
Cc: Wang Xiaoqiang <[email protected]>; [email protected]
Subject: Re: Getting images from area detector plugins

 

Thanks for the discussion!  We will look into moving to PVA.  However, I now have a bunch of questions (I’m a new EPICS user)!

 

Our clients (several) have been using ca_get_array to get an array and we can convert it on our end to whatever we want.  Our non-EPICs cameras, which we are moving away from in favor of something more standard, have an internal server which produces JPEGs.  These are just read as a byte stream over a socket and that’s what we are trying to duplicate.  We either display them, both QT and IOS can deal with JPEGs directly, or we convert them to a 2D image for image analysis.

 

So I was expecting that there was some way to just grab the jpeg out of NDPluginCodec as an array of bytes without any conversion going on.  I presume the jpeg image exists somewhere within the plugin as a block of bytes.  NDFileJpeg(?) appears to be able to write the bytes of a jpeg out to a file which is pretty much what we want though we want it from a network connection read rather than directed to a file. 

 

How does a “compressed array” differ from just an array which happens to have a set of bytes that have been compressed?  I presume knowing that the array data is compressed, how it’s compressed, etc is important if I want to pass it to another module that wants to work on it as an image.  But here we don’t want to pass it on except to a client that knows what it is and how to deal with it.  We just want the bytes.

 

Randall

 

 

 

> On Jan 24, 2020, at 4:23 AM, Mark Rivers via Tech-talk <[email protected]> wrote:

>

> Hi Xiaoqiang,

>

>

> Thanks for the comment.

>

>

>> Actually it is not a limit of channel access to support passing compressed NDArrays.

>

>

> Yes, I realized after my post yesterday that compressed arrays could possibly be passed over Channel Access as well, and once we hear more about Randall's client I was going to suggest this might be an alternative.

>

>

> However, currently NDPluginStdArrays cannot be used for compressed NDArrays because it does not pass the flag compressionAware=true when its constructor calls the NDPluginDriver constructor.  This means that NDPluginStdArrays will return an error if it receives a compressed array and will not send it over Channel Access.  This would be easy to change.

>

>

>> The client will look at the $(P)$(R)Codec_RBV record to handle it accordingly.

>

> It will also need to look at the  $(P)$(R)CompressedSize_RBV to determine how many bytes to read.

>

> I think it will probably also only work if the data type of the NDArray matches the data type of the waveform record.  If not then NDPluginStdArrays will convert the data to the type of the waveform record, which will corrupt the compressed data.

>

> Mark

>

>

>

> ________________________________

> From: Wang Xiaoqiang <[email protected]>

> Sent: Friday, January 24, 2020 1:33 AM

> To: Mark Rivers

> Cc: [email protected]

> Subject: Re: Getting images from area detector plugins

>

>

> Hi Mark,

>

> Sorry to intercept this thread.

>

> Actually it is not a limit of channel access to support passing compressed NDArrays.

>

> The NDPluginStdArrays could just copy the compressed bytes to output.

> The client will look at the $(P)$(R)Codec_RBV record to handle it accordingly.

>

> I have experimented this idea before the introduction of NDCodec

> plugin. I image this has only got easier.

>

> Best

> Xiaoqiang

>

>

> On 1/24/20 12:44 AM, Mark Rivers via Tech-talk wrote:

>> Hi Randall,

>>

>> You can't pass the JPEG image over EPICS Channel Access, but you can pass it over pvAccess.  The NTNDArray that your pvAccess client receives will be the JPEG compressed image.

>>

>> What type of client are you running, and how will you display it?

>>

>> There are examples of receiving JPEG compressed NTNDArrays in the

>> ImageJ EPICS_NTNDA_Viewer.java in ADViewers

>>

>> https://github.com/areaDetector/ADViewers/tree/master/ImageJ/EPICS_ar

>> eaDetector

>>

>> and in a new Python viewer that Marty Kraimer is writing:

>>

>> https://github.com/mrkraimer/PY_NTNDA_Viewer

>>

>> These viewers first decompress the JPEG images before displaying them.

>>

>> Mark

>>

>>

>> -----Original Message-----

>> From: Tech-talk <[email protected]> On Behalf Of Randall

>> Cayford via Tech-talk

>> Sent: Thursday, January 23, 2020 5:29 PM

>> To: [email protected]

>> Subject: Getting images from area detector plugins

>>

>> We're trying to connect to a gige camera with areadetector (using ADaravis).  We would like to access the camera images via a pv that returns a jpeg of the image rather than an uncompressed byte stream.

>>

>> We've got the camera connected, we've got an NDPluginCodec setup that takes the images and apparently is producing compressed jpeg images.  We can enable acquisition and watch the MEDM screen show frames arriving and degree of compression, etc.

>>

>>  Our problem is:  how do we access those images?  There doesn't seem to be any PV that returns the jpeg.  Perhaps we're supposed to pass the output of the codec to something else but we can't seem to find anything that tells us how to do that.

>>

>> Help would be appreciated.

>>

>> Randall Cayford

>> ALS  at Lawerence Berkeley Lab

>>