Re: BEAUTY Archiver and data from Area detector

["Kasemir, Kay" <kasemirk@ornl.gov>]

Hi:

As you noticed, it's not really suitable for archiving images.
In the original implementation, it actually stored every element of an array as a separate table row, to stay compatible with all RDB implementations.
Now it's writing arrays into BLOBs. That's more efficient, but means you can't simply read it back out without decoding the binary BLOB data. In addition, the only documented and implementing encoding of the BLOB is for double-typed data. Other data types could be added, but haven't, so your grayscale image explodes in size as each pixel turns into a double.
Finally, there's currently no tool to read 'image' waveforms out for display.  The CS-Studio Data Browser can show one waveform element over time in its usual scalar plot, or show-one-waveform-at-a-time in its waveform display.

In the future, things might be better once you get your Image as a V4 Image type that can identify itself as such, with width, height, color map, pixel depth, .. Storage formats other than an RDB would be much more suitable for such images.

-Kay

On Nov 8, 2013, at 12:46 PM, Vítek Michal <Michal.Vitek@eli-beams.eu<mailto:Michal.Vitek@eli-beams.eu>> wrote:
Hello,

We would like to archive data from cameras (grabbed by EPICS IOC with area detector) with BEAUTY archiver. We have prepared testing setup with 1 camera (512x512 img, 25 FPS, grayscale), BEAUTY archiver, and MySQL db. Each block is running on independent computer, interconnection is over 1GB lan. The network traffic measured on the output of the EPICS IOC is ~6MB/s. The image can be displayed by Control System Studio intensity graph without problems.

I have also added following param to the BEAUTY configuration file:

org.csstudio.platform.libs.epics/max_array_bytes=300000

because it was complaining about the size of the incoming image.

The memory available for the Java programs is set to 2GB by OS Environmental variable.

When we start the archiver we can see that BEAUTY consumes all available resources (2GB of RAM and 99% CPU) in ~5s and then it crashes. When we changed the PV to be scanned every 5s then the BAUTY maintain to run consuming ~1GB of memory, and ~20% CPU and write duration  ~240s.

Thus I would like to ask if BEUATY is suitable for the archiving of images from camera or if I am doing something wrong…? Can anyone help me with this?

Cheers,

Mike