RE: BEAUTY Archiver and data from Area detector

[<ulrik.pedersen@diamond.ac.uk>]

Hi Mike,

 

I would recommend simply using one of the many file writing plugins that areaDetector ships with: you can store images in jpeg, tiff, netcdf, nexus and my personal favourite: hdf5. The list of supported formats is actually even longer as the GraphicsMagick plugin allow saving to any format the GraphicsMagick library supports... http://cars9.uchicago.edu/software/epics/NDPluginFile.html

 

Relational databases like mysql does not perform particularly well with constant streaming of image data. Certain file formats will do much better – especially when it is time to scale up a little bit (higher res. camera or higher framerate).

 

Cheers,

Ulrik

 

From: tech-talk-bounces@aps.anl.gov [mailto:tech-talk-bounces@aps.anl.gov] On Behalf Of Vítek Michal
Sent: 08 November 2013 17:46
To: tech-talk@aps.anl.gov
Subject: BEAUTY Archiver and data from Area detector

 

Hello,

 

We would like to archive data from cameras (grabbed by EPICS IOC with area detector) with BEAUTY archiver. We have prepared testing setup with 1 camera (512x512 img, 25 FPS, grayscale), BEAUTY archiver, and MySQL db. Each block is running on independent computer, interconnection is over 1GB lan. The network traffic measured on the output of the EPICS IOC is ~6MB/s. The image can be displayed by Control System Studio intensity graph without problems.

 

I have also added following param to the BEAUTY configuration file:

 

org.csstudio.platform.libs.epics/max_array_bytes=300000

 

because it was complaining about the size of the incoming image.

 

The memory available for the Java programs is set to 2GB by OS Environmental variable.

 

When we start the archiver we can see that BEAUTY consumes all available resources (2GB of RAM and 99% CPU) in ~5s and then it crashes. When we changed the PV to be scanned every 5s then the BAUTY maintain to run consuming ~1GB of memory, and ~20% CPU and write duration  ~240s.

 

Thus I would like to ask if BEUATY is suitable for the archiving of images from camera or if I am doing something wrong…? Can anyone help me with this?

 

Cheers,

 

Mike

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom