Re: hdf5 (h5py) anyone?

[Claude Saunders <saunders@aps.anl.gov>]

Hi Emmanuel,
You'll find a number of people using HDF5, but mostly on the beamline science side. My group uses it to store multidimensional data along with all manner of metadata. The multidimensional data is typically detector images (stacked as a cube), but can be anything. It's used at many other places, like Diamond, ESRF, Solei, ANSTO, ....

When I used to work accelerator controls, we reviewed formats and settled on a simpler one that Michael Borland designed and implemented called SDDS. The thought was that the multi-dim dataset support in HDF was not really needed for time-series measurements of individual signals typical in accelerators, and the complexity of these formats (at the time it was netCDF and an earlier version of HDF) made it hard to adopt.

The h5py is a great interface to HDF5, and is easier to use than the straight C interface that many things seem to adopt, like the IDL HDF5 interface.

The main issue with HDF5 seems to be a lack of tools that read it. There's lots of tools and libs that will read tiff, jpg, etc., but much fewer for HDF5. So adopting its use is a bit of a chicken and egg problem. If someone has a favorite image/dataset viewer, and it doesn't understand HDF5, it doesn't make them happy if you hand them an HDF5 file...

But to my mind, it's the best format out there for multidimensional datasets, and it can handle pretty much everything else. 

  - Claude

----- Original Message -----
From: "Emmanuel Mayssat" <emayssat@yahoo.com>
To: "epics" <tech-talk@aps.anl.gov>
Sent: Wednesday, March 21, 2012 7:30:41 PM
Subject: hdf5 (h5py) anyone?



hdf5 is a filesystem database (directory file structure, can span files and computers, etc) 
h5py v2.0 (python interface) was demonstrated recently at PyCon2012. 
Is anyone of you using hdf5 file in an epics context (or other)? 

Regards, 
-- 
E