Experimental Physics and
Industrial Control System

Matt Newville <[email protected]> · Thu, 22 Mar 2012 13:44:01 -0500

Nikolay, Emmanuel,

On Thu, Mar 22, 2012 at 12:42 PM, Malitsky, Nikolay D <[email protected]> wrote:
>>what does an 'extra service' mean?
>>Are you talking about a client/server architecture?
>
> yes. In short, you can consider it as the Channel Archiver extension/upgrade
> with
> (one of) a new driver to the hdf5 backend.  As a result, conceptually
> everything
> (including file structure) would be the same except several extensions like
> support
> of the EPICS 4 structured data types. The type registry and associated
> topics will be
> presented at the next EPICS meeting.

I would be very cautious about archiving data directly into HDF5.  See:
     http://www.hdfgroup.org/hdf5-quest.html#gconc1
     http://www.hdfgroup.org/hdf5-quest.html#grdwt

Saving to some more robust format (say, ASCII files such as SDDS or a
relational database), and then converting to HDF5 for fast and
hierarchical read access is probably a much safer approach.

Whether hierarchical vs relational data storage is the better model
for an archiving system where the principle axis is time is an
interesting theoretical topic, but I think that archiving directly to
HDF5 is very likely to end up with lost data.

HDF5 is great at what it does, but it is not really a database
management system.

Cheers,

--Matt Newville

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System