Argonne National Laboratory

Experimental Physics and
Industrial Control System

2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020  Index 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020 
<== Date ==> <== Thread ==>

Subject: Compression in pvAccess
From: Mark Rivers <rivers@cars.uchicago.edu>
To: "core-talk@aps.anl.gov" <core-talk@aps.anl.gov>
Date: Fri, 16 Nov 2018 20:25:09 +0000

Folks,

 

Bruno has written a new plugin for areaDetector called NDPluginCodec.  It compresses and decompresses NDArrays using JPEG (lossy) and Blosc (lossless) compression.  NDArrays now have a .codec field, just as NTNDArrays do.  He implemented these fields in the areaDetector converters for NDArrays <-> NTNDArrays.

 

Using these new capabilities I just set up the following configuration.  The simDetector IOC is running on one machine (happens to be Linux), and the receiving IOC and ImageJ are running on another machine (happens to be Windows).

 

  simDetector detector   =>  NDPluginCodec (compress)   =>  NDPluginPVA   =>   pvaAccess over Ethernet  =>  pvaDetector  =>  NDPluginCodec (decompress)  =>  NDPluginPva  =>  pvaAccess localhost  =>  ImageJ pvAccess viewer

 

By enabling the compression I can reduce the network bandwidth >100x for highly compressible images.

 

The next step would be to eliminate the need for the IOC on the viewer machine, i.e. the viewer can directly decompress the NTNDArray that comes from pvAccess.

 

One way to implement this is to require the decompression code be in the client (ImageJ, Display Builder, etc.).  But it would be much nicer to have the decompression library and compress/decompress methods as part of pvAccess, so then each client does not need to implement it. 

 

In my opinion the top priority should be Java, since Phoebus/CSS/Display Builder and ImageJ are both written in Java.  If we get this working then we can transport images between camera/detector IOCs and remote viewing clients with much lower network bandwidth than is currently required.  We can do both lossless and lossy compression.

 

There is already a Blosc implementation for Java:

https://github.com/Blosc/JBlosc

 

So far I have only found a JNI implementation for JPEG, but hopefully there is a pure Java implementation as well.

 

Thoughts?

 

Mark

 


Replies:
Re: Compression in pvAccess Michael Davidsaver via Core-talk

Navigate by Date:
Prev: Build failed in Jenkins: EPICS-7 #160 Jenkins EPICS PSI
Next: Jenkins build is back to normal : EPICS-7 #161 Jenkins EPICS PSI
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020 
Navigate by Thread:
Prev: Jenkins build is back to normal : EPICS-7 #161 Jenkins EPICS PSI
Next: Re: Compression in pvAccess Michael Davidsaver via Core-talk
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020 
ANJ, 19 Nov 2018 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·