I should also mention we use the areaDetector update rate throttling feature as well, in addition to the binning and re-scaling. Usually the standard array plugin does the throttling so that it doesn’t send out arrays faster than between
1Hz-10Hz. Mostly 1-2Hz. The 10Hz (or faster) is sometimes useful for camera-like applications, but then channel access is not the best solution anyway.
I would be interested in seeing how PVAccess (and various clients) performs for compressed camera images that we want to display at 20-30Hz. In the past I’ve used MJPG via the ffmpeg areaDetector plugin, but I found it was unreliable and
occasionally stopped working, so I switched back to channel access which was slower but was stable.
Cheers,
Matt
From: Timo Korhonen <Timo.Korhonen at ess.eu>
Sent: Monday, November 1, 2021 5:31 PM
To: Pearson, Matthew R. <pearsonmr at ornl.gov>; Mark L Rivers <rivers at cars.uchicago.edu>; Ralph Lange <ralph.lange at gmx.de>
Cc: core-talk at aps.anl.gov
Subject: Re: [EXTERNAL] CA gateway chaining
Hi Matt,
Thank you for these numbers. I do not even dare to publicly state what kind of data volumes some of our (accelerator) users are asking for… I think we need to seriously consider binning.
Cheers,
Timo
Hi Timo,
Yes, what Mark said.
For example, a typical machine vision camera or astronomy CCD might be 2048*2048*2 bytes = 8.4MB. So I might choose to use 2x2 or 4x4 binning on this, via the areaDetector ROI plugin, to get down
to either 2MB or 0.5MB (depending on how large the image widget will be on the OPI). And, in addition, I could use the processing plugin to scale to 8-bit greyscale data, so now we get down to 1MB or 0.25MB.
In practice, a monitor will only display up to 8-bit greyscale anyway, and humans can’t see a full 16-bit range of greyscale either.
Similarly, our neutron time-of-flight 1-D spectrums are multiples of 160,000 * 4 bytes = 0.64MB. So these are binned by a factor 10 to get to 64KB.
Cheers,
Matt
Hi Timo,
If your arrays are coming from areaDetector then binning can easily be done in the IOC using NDPluginROI.
If the arrays are images then you may also be able to use NDPluginCodec to compress them, for example with the JPEG compressor. areaDetector
provides Linux and Windows decompressors that can be called from your PVA client. Those are used from the ImageJ plugin, for example.
Mark
Do you do the binning in the IOC? This is something we have thought of but not implemented yet.
Timo
Hi,
We have seen similar issues and one way we dealt with it was to provide a heavily re-binned array for use via gateways. Inside the beamline network we provide both a full un-binned array, hidden
behind ‘detailed’ buttons, as well as the re-binned array screens that users will first encounter on higher level screens. We also limited the array size that the gateway will transmit, so some of the larger arrays don’t work which forces people to view the
re-binned arrays.
Also, depending on the original data type and software, you can ‘compress’ the array data from 16 or 32-bit down to 8-bit data by re-scaling it to 0-255, like a JPEG image. Then the waveform data
type can be set to UCHAR.
The above only works if the waveforms are simply for visualization.
Cheers,
Matt
Right, this is the way we are going for now. It is not exactly trivial to manage but should be doable.
Thanks,
TImo
Just a short follow-up.
We found out that the delays are due to large arrays being monitored over the gateway. Not a surprise, even if we have tried to remind users about the effects with big arrays.
One way to mitigate this - I think this approach was implemented at some point at the SLS - is to route the large array channels through a dedicated separate gateway instance.
(Ease of configuration depends on your naming convention.)