Re: ADEuresys: New areaDetector driver for CoaXPress cameras using Euresys frame grabbers

[Érico Nogueira Rolim via Tech-talk <tech-talk at aps.anl.gov>]

On 10/04/2024 20:11, Mark Rivers via Tech-talk wrote:

Folks,

 

R1-0 of ADEuresys is now available.  ADEuresys is a driver for CoaXPress cameras using PCIe frame grabbers from Euresys.

 

CoaXPress is a high-speed camera interface using multiple coaxial cables.  Each cable can run up to 12.5 Gbits/s, and with 4 parallel cables the total speed is up to 50 Gbits/s.

 

CoaXPress is based on GenICam, so camera features are controlled the same way as with GigEVision and USBVision cameras.  The ADEuresys driver is derived from ADGenICam.

 

The driver has been tested on Windows 10 and Linux (CentOS 9 Stream) with the following cameras:

 

Model	Mikrotron EoSens 2.0MCX12	Adimec Q-12A	ViewWorks VNP604
Pixel dimensions	1920 x 1080	4096 x 3072	14192 x 10640
MegaPixels	2.1	12.6	151.0
CXP version	CXP-12 X4	CXP-6 X4	CXP-6 X4
Frames/s, Mono8	2247	187	6.5
GBytes/s. Mono8	4.66	2.35	1.0
Frames/s, Mono10	1798	152	6.2
GBytes/s. Mono10	7.46	3.83	2.0
Frames/s, Mono12	N.A	N.A	6.2
GBytes/s. Mono12	N.A	N.A	2.0

 

Note that the Mikrotron camera runs at 2,247 frames per second in Mono8 mode.

 

Home: https://github.com/areaDetector/ADEuresys

Documentation: https://areadetector.github.io/areaDetector/ADEuresys/ADEuresys.html

Regarding the file saving benchmarks, quoted below, I can confirm the presence of a global lock in libhdf5.

> I believe this indicates that the rate is limited by a global mutex in the HDF5 library, since neither plugin was CPU bound, and I don’t think they were limited by the NVMe disk write rate.

From https:// docs.hdfgroup.org/hdf5/develop/group___h5.html#ga70bfde4acd009cdd7bcd2f54c594e28a

> The HDF5 library, although not internally multi-threaded, can be built with a thread-safety feature enabled that protects internal data structures with a mutex. In certain circumstances, it may be useful to determine, at run-time, whether the linked HDF5 library was built with the thread-safety feature enabled.

From https:// forum.hdfgroup.org/t/write-separate-files-with-multithreading/11224/2

> this will give you the illusion of multiple threads writing to different files or the same file. I’m saying illusion because only one thread at a time can execute code in the HDF5 library. In other words, these threads will not be executing library code concurrently.

The recommended parallelism scheme for HDF5 is MPI, which IMHO I can only see fitting into areaDetector as some NDFileHDF5MPI plugin which would launch worker processes with mpirun and somehow feed frames to these processes. The other option I can think of, launching multiple instances of an IOC under mpirun and having a custom main() function which calls either the IOC startup function or starts an HDF5 worker based on process rank, while possible, is much more complicated, and doesn't seem scalable, since it requires changes to the main function in every IOC which needs the functionality.

Also from the benchmarks

> I found that writing files with JRaw, both in Stream mode and Capture mode, did not result in a statistically significant improvement over HDF5. In some cases it was faster, but in some it was slower.

I believe the advantage of something like JRaw over HDF5 would be that launching two instances of the plugin on the same IOC and sending frames to them using NDPluginScatter should actually benefit from using multiple threads for I/O. That scenario doesn't seem to be included in benchmark.txt. Do you think it makes sense to test it?

 

Cheers,

Mark

 

Cheers,

Érico

Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente.

Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately.