1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 <2024> | Index | 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 <2024> |
<== Date ==> | <== Thread ==> |
---|
Subject: | Re: Editing .pb files for the purpose of appending data to them |
From: | Michael Davidsaver via Tech-talk <tech-talk at aps.anl.gov> |
To: | Tynan Ford <TFord at lbl.gov>, Sky Brewer <sky.brewer at ess.eu> |
Cc: | "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov> |
Date: | Tue, 30 Apr 2024 07:47:49 -0700 |
On 4/29/24 08:19, Tynan Ford via Tech-talk wrote:
Hi Sky, We have wondered if it's possible to append data to the appliance pb files since we have data from previous archivers that goes back many years. It would be useful to have that data imported into the appliance so all archived data can be retrieved via the same method. I naively thought we could write a script to create pb files like TESTPV:2001.pb, TESTPV:2002.pb, etc but I'm curious what you think?
It is probably suffering from bit-rot at this point, but my old Channel Archiver to Archiver Appliance direct file converter is still available: https: //github.com/mdavidsaver/ca2aa
Thanks Tynan On Wed, Apr 24, 2024 at 2:46 AM Sky Brewer via Tech-talk <tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>> wrote: Hi Windsor,____ __ __ Some examples of python code for handling the pb files can be found at:____ __ __ https://urldefense.us/v3/__https://github.com/zhangt58/PyArchAppl/blob/master/main/data/pb/decode.py__;!!G_uCfscf7eWS!fvGpipmB7FsIn0XQsrK2CNDsPkaJASxwivcnk14AAtitA7Bx5MtNqAKPuyMGArcX3LaYhxBPVOQGyjxeSiEsE0wKHA$ <https://urldefense.us/v3/__https://github.com/zhangt58/PyArchAppl/blob/master/main/data/pb/decode.py__;!!G_uCfscf7eWS!b5YgNXV8LWnk1VGBnMvzi-WzDnvd0WROFnruh9I275nM8J9YqqhG98bXfcER4KCiJ6M4_LjfjjRhn2omsWogAg71$> ____ https://urldefense.us/v3/__https://github.com/dls-controls/aapy/blob/master/aa/pb.py__;!!G_uCfscf7eWS!fvGpipmB7FsIn0XQsrK2CNDsPkaJASxwivcnk14AAtitA7Bx5MtNqAKPuyMGArcX3LaYhxBPVOQGyjxeSiGYHNJ83Q$ <https://urldefense.us/v3/__https://github.com/dls-controls/aapy/blob/master/aa/pb.py__;!!G_uCfscf7eWS!b5YgNXV8LWnk1VGBnMvzi-WzDnvd0WROFnruh9I275nM8J9YqqhG98bXfcER4KCiJ6M4_LjfjjRhn2omsZN2X7I9$> ____ and ____ https://urldefense.us/v3/__https://gitlab.esss.lu.se/ics-software/py-epicsarchiver/-/blob/master/epicsarchiver/pb.py?ref_type=heads__;!!G_uCfscf7eWS!fvGpipmB7FsIn0XQsrK2CNDsPkaJASxwivcnk14AAtitA7Bx5MtNqAKPuyMGArcX3LaYhxBPVOQGyjxeSiGfvj4MkQ$ <https://urldefense.us/v3/__https://gitlab.esss.lu.se/ics-software/py-epicsarchiver/-/blob/master/epicsarchiver/pb.py?ref_type=heads__;!!G_uCfscf7eWS!b5YgNXV8LWnk1VGBnMvzi-WzDnvd0WROFnruh9I275nM8J9YqqhG98bXfcER4KCiJ6M4_LjfjjRhn2omseE1GJdN$> ____ __ __ Obviously, I prefer my own code 😊____ __ __ In terms of appending data, could you explain your use case? I don’t know why you would want to append data from outside the archiver.____ The archiver makes many assumptions about these files in terms of what data is in them dependent on the name of the file. I would be ____ Careful about altering them.____ __ __ A simple solution can be to push data into the files using a PV! Note that the archiver doesn’t do any checks that timestamps correspond to the current time. ____ Only that they are sequential. So as long as you are adding new events you can push extra events in.____ __ __ A second solution is to use the merge data from another cluster in the ETL process. You can see some documentation at https://urldefense.us/v3/__https://slacmshankar.github.io/epicsarchiver_docs/redundancy.html__;!!G_uCfscf7eWS!fvGpipmB7FsIn0XQsrK2CNDsPkaJASxwivcnk14AAtitA7Bx5MtNqAKPuyMGArcX3LaYhxBPVOQGyjxeSiH7yrtgUA$ <https://urldefense.us/v3/__https://slacmshankar.github.io/epicsarchiver_docs/redundancy.html__;!!G_uCfscf7eWS!b5YgNXV8LWnk1VGBnMvzi-WzDnvd0WROFnruh9I275nM8J9YqqhG98bXfcER4KCiJ6M4_LjfjjRhn2omscva_VhG$> ____ I can probably create an example if you need it.____ __ __ You could then set up a dummy PV to push the data you want in, and allow the archiver ETL process to merge the data for you.____ __ __ I hope that helps.____ Sky____ __ __ __ __ __ __