1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 <2024> | Index | 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 <2024> |
<== Date ==> | <== Thread ==> |
---|
Subject: | EPICS module/service that can be used for saving waveform data/metadata to HDF5 |
From: | "Wlodek, Jakub via Tech-talk" <tech-talk at aps.anl.gov> |
To: | "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov> |
Date: | Tue, 11 Jun 2024 13:22:02 +0000 |
Hi all,
During the EPICS codeathon this week at NSLS-II I mentioned a need for us here at NSLS-II w.r.t. data saving. Essentially, we are beginning to work on hardening the data access security for scientific data, one consequence of this being that operator accounts
that users run their experiments under will no longer have r/w access to data directories, and all data needs to be written by either a softioc service account, or by a workflow service account.
The problem with this, is that for a fair number of detectors/devices that did not have any in-built file saving capabilities (basically anything that isn't an areaDetector driver), the previous approach was to basically
caget the waveform data and any metadata signals, and in the bluesky experiment orchestration layer use the h5py library to save the data, all as this operator account. Since this approach will no longer work due to the new permissions, to work
around this, we've been moving this code into python IOCs that expose some PVs that are then used to more or less call the same functions from bluesky - just now with the file saving done as a softioc account. However, I don't think this is an ideal solution
- it is leading to a lot of one-off IOCs for each of these devices.
I've been thinking of ways to build something more general that will allow for specifying a waveform record and some additional metadata and saving it when asked, and it was suggested to me at the codeathon to reach out here to the community to see if others
have a similar need or ideas on how to handle this.
Thanks!
Jakub
|