Experimental Physics and
Industrial Control System

New releases of the following areaDetector modules are now available. 

 

areaDetector R3-3

ADCore R3-3

 

****************************************************************************************

These are the major items from the R3-3 release notes for areaDetector:

 

* Changes in the configure/RELEASE* files to ensure that EPICS_BASE is defined last.

* Changes to allow building when some modules come from a Debian package and some do not.

* The INSTALL_GUIDE.md has been updated to describe the new system.

* The RELEASE files in many detector modules have been updated so that they only load RELEASE_LIBS_INCLUDE or

   RELEASE_PRODS_INCLUDE from areaDetector/configure.  This change was previously optional and had already been done for

   some detector modules in previous releases.  This change is now required and so all detector modules have been updated

   to use it.

* Added new ADSpinnaker submodule.  This is for detectors from Point Grey/FLIR using their new Spinnaker SDK.

* Added ADEiger submodule.  This is for Eiger detectors from Dectris.  Thanks to Bruno Martins for this.

 

 

****************************************************************************************

These are the major items from the R3-3 release notes for ADCore:

 

### NDArrayPool design changes

* Previously each plugin used its own NDArrayPool. This design had the problem that it was not really possible

  to enforce the maxMemory limits for the driver and plugin chain.  It is the sum of the memory use by the driver

  and all plugins that matters, not the use by each individual driver and plugin.

* The NDPluginDriver base class was changed to set its pNDArrayPool pointer to the address passed to it in the

  NDArray.pNDArrayPool for the NDArray in the callback.  Ultimately all NDArrays are derived from the driver,

  either directly, or via the NDArrayPool.copy() or NDArrayPool.convert() methods.  This means that plugins

  now allocate NDArrays from the driver's NDArrayPool, not their own. 

* The freelist in NDArrayPool was changed from being an EPICS ellList to an std::multiset.  The freelist is

  now sorted by the size of the NDArray.  This allows quickly finding an NDArray of the correct size,

  and knowing if no such NDArray exists.

* Previously there was no way to free the memory in the freelist, giving the memory back to the operating system

  after a large number of NDArrays had been allocated, without restarting the IOC.  The NDArrayPool class now

  has an emptyFreeList() method that deletes all of the NDArrays in the freelist.  asynNDArrayDriver has a

  new NDPoolEmptyFreeList parameter, and NDArrayBase.template has a new bo record called $(P)$(R)EmptyFreeList

  that will empty the freelist when processed. 

* Improved the efficiency of memory allocation.  Previously the first NDArray that is large enough was returned.

  Now if the size of the smallest available NDArray exceeds the requested size by a factor of 1.5 then the

  memory in that NDArray is freed and reallocated to be the requested size.  Thanks to Michael Huth for the first

  implementation of this.

* These changes are generally backwards compatible. However, startup scripts that set a non-zero value for

  maxMemory in the driver may need to increase this value because all NDArrays are now allocated from this NDArrayPool.

 

### Queued array counting and waiting for plugins to complete

* Previously if one wanted to wait for plugins to complete before the driver indicated that acquisition was complete

  then one needed to set CallbacksBlock=Yes for each plugin in the chain.

* There are 2 problems with setting CallbacksBlock=Yes.

  - It slows down the driver because the plugin is executing in the driver thread and not in its own thread.

  - It is complicated to change all of the required plugin settings from CallbacksBlock=No to CallbacksBlock=Yes.

* The NDPluginDriver base class now increments a NumQueuedArrays counter in the driver that owns each NDArray as it is queued,

  and decrements the counter after the processing is done.

* All drivers have 3 new records:

  - NumQueuedArrays: This record indicates the total number of NDArrays that are currently processing or are queued

    for processing by this driver.

  - WaitForPlugins: This record determines whether AcquireBusy waits for NumQueuedArrays to go to 0 before changing to 0 when acquisition completes.

  - AcquireBusy This is a busy record that is set to 1 when Acquire changes to 1. It changes back to 0 when acquisition completes,

    i.e. when Acquire_RBV=0. If WaitForPlugins is Yes then it also waits for NumQueuedArrays to go to 0 before changing to 0.

* With this new design it should rarely be necessary to change plugins to use CallbacksBlock=Yes.

 

### NDArray, NDArrayPool

* Changes to allow the NDArray class to be inherited by derived classes.  Thanks to Sinisa Veseli for this.

* NDArrayPool::report() now prints a summary of the freeList entries if details>5 and shows the details

  of each array in the freeList if details>10.  This information can be printed with "asynReport 6 driverName" for example.

 

### ntndArrayConverter.cpp

* Added conversion of the NDArray.timeStamp and NDArray.epicsTS fields from EPICS epoch (Jan. 1 1990) to

  Posix epoch (Jan. 1, 1970). Needed because NDArrays use EPICS epoch but pvAccess uses Posix epoch and the

  timestamps shown by pvGet were incorrect for the NTNDArrays.

 

### NDPluginFile

* Fixes to readFileBase() so that the ReadFile PV actually works.  This has now been implemented for NDFileTIFF.

 

### NDFileTIFF

* Added support for readFile() so it is now possible to read a TIFF file into an NDArray using this plugin and

  do callbacks to downstream plugins.

  - All datatypes (NDDataType_t) are supported.

  - It supports Mono, RGB1, and RGB3 color modes.  It also correctly reads files written with RGB2 color mode.

  - It restores the NDArray fields uniqueID, timeStamp, and epicsTS if they are present.

  - It restores all of the NDArray NDAttributes that were written to the TIFF file.

  - One motivation for adding this capability is for the NDPluginProcess plugin to be able to read TIFF files

    for the background and flat field images, rather than needing to collect them each time it is used.

 

### NDPluginProcess

* Load a dedicated TIFF plugin for the NDPluginProcess plugin in commonPlugins.cmd.  This TIFF plugin is used for reading

  background or flatfield TIFF files.

* Add an sseq record to load the background image from a TIFF file. This executes all the following steps:

  1. Saves the current NDArrayPort of the Process plugin to a temporary location

  2. Sets the NDArrayPort to the TIFF plugin.

  3. Enables ArrayCallbacks for the TIFF plugin in case they were disabled.

  4. Process the ReadFile record in the TIFF plugin.  This reads the TIFF file and does callback to the Process plugin.

  5. Loads the NDArray from the callback into the background image.

  6. Restores the previous NDArrayPort from the temporary location.

* Add an sseq record to load the flatfile from a TIFF file.  This executes the same steps as for the background

  above, except that in step 5 it loads the NDArray into the flatfile image.

 

### NDPluginStats

* Changed the time series to use NDPluginTimeSeries, rather than having the time series logic in NDPluginStats.

  This reduced the code by 240 lines, while adding the capability of running in Circular Buffer mode,

  not just a fixed number of time points.

* NOTE: The names of the time series arrays for each statistic have not changed.  However, the name of the PVs to control

  the time series acquisition have changed, for example from $(P)$(R)TSControl, to $(P)$(R)TS:TSAcquire.  This may

  require changes to clients that were controlling time series acquisitions.

* EXAMPLE_commonPlugins.cmd has changed to load an NDPluginTimeSeries plugin and database for each NDPluginStats plugin,

  so the local commonPlugins.cmd file must be updated.

 

### ADApp/op/adl

* Fixes to a number of .adl files to set text widget size and alignment, etc. to improve conversion to .opi, .edl, and .ui files.

 

### ADApp/op/edl/autoconvert

* Major improvement in quality of edm screens (colors, fonts, etc.) thanks to Bruce Hill.

 

### ADApp/pluginTests

* Added a new unit test, test_NDArrayPool to test NDArrayPool::alloc().

* All unit tests were changed to create an asynNDArrayDriver and use the NDArrayPool from that, rather than directly

  creating an NDArrayPool.

 

****************************************************************************************

 

The home page is here:

http://cars.uchicago.edu/software/epics/areaDetector.html

 

The top-level documentation is here:

http://cars.uchicago.edu/software/epics/areaDetectorDoc.html

 

Installation instructions are here:

https://github.com/areaDetector/areaDetector/blob/master/INSTALL_GUIDE.md

 

areaDetector release notes are here:

https://github.com/areaDetector/areaDetector/blob/master/RELEASE.md

 

ADCore release notes are here:

https://github.com/areaDetector/ADCore/blob/master/RELEASE.md

 

Mark