Experimental Physics and
Industrial Control System

Michael Davidsaver via Tech-talk <tech-talk at aps.anl.gov> · Tue, 30 Apr 2024 07:47:49 -0700

On 4/29/24 08:19, Tynan Ford via Tech-talk wrote:

Hi Sky,

We have wondered if it's possible to append data to the appliance pb files since we have data from previous archivers that goes back many years. It would be useful to have that data imported into the appliance so all archived data can be retrieved via the same method. I naively thought we could write a script to create pb files like TESTPV:2001.pb, TESTPV:2002.pb, etc but I'm curious what you think?

It is probably suffering from bit-rot at this point,
but my old Channel Archiver to Archiver Appliance
direct file converter is still available:

https: //github.com/mdavidsaver/ca2aa

Thanks
Tynan

On Wed, Apr 24, 2024 at 2:46 AM Sky Brewer via Tech-talk <tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>> wrote:

    Hi Windsor,____

    __ __

    Some examples of python code for handling the pb files can be found at:____

    __ __

    https://urldefense.us/v3/__https://github.com/zhangt58/PyArchAppl/blob/master/main/data/pb/decode.py__;!!G_uCfscf7eWS!fvGpipmB7FsIn0XQsrK2CNDsPkaJASxwivcnk14AAtitA7Bx5MtNqAKPuyMGArcX3LaYhxBPVOQGyjxeSiEsE0wKHA$  <https://urldefense.us/v3/__https://github.com/zhangt58/PyArchAppl/blob/master/main/data/pb/decode.py__;!!G_uCfscf7eWS!b5YgNXV8LWnk1VGBnMvzi-WzDnvd0WROFnruh9I275nM8J9YqqhG98bXfcER4KCiJ6M4_LjfjjRhn2omsWogAg71$> ____

    https://urldefense.us/v3/__https://github.com/dls-controls/aapy/blob/master/aa/pb.py__;!!G_uCfscf7eWS!fvGpipmB7FsIn0XQsrK2CNDsPkaJASxwivcnk14AAtitA7Bx5MtNqAKPuyMGArcX3LaYhxBPVOQGyjxeSiGYHNJ83Q$  <https://urldefense.us/v3/__https://github.com/dls-controls/aapy/blob/master/aa/pb.py__;!!G_uCfscf7eWS!b5YgNXV8LWnk1VGBnMvzi-WzDnvd0WROFnruh9I275nM8J9YqqhG98bXfcER4KCiJ6M4_LjfjjRhn2omsZN2X7I9$> ____

    and ____

    https://urldefense.us/v3/__https://gitlab.esss.lu.se/ics-software/py-epicsarchiver/-/blob/master/epicsarchiver/pb.py?ref_type=heads__;!!G_uCfscf7eWS!fvGpipmB7FsIn0XQsrK2CNDsPkaJASxwivcnk14AAtitA7Bx5MtNqAKPuyMGArcX3LaYhxBPVOQGyjxeSiGfvj4MkQ$  <https://urldefense.us/v3/__https://gitlab.esss.lu.se/ics-software/py-epicsarchiver/-/blob/master/epicsarchiver/pb.py?ref_type=heads__;!!G_uCfscf7eWS!b5YgNXV8LWnk1VGBnMvzi-WzDnvd0WROFnruh9I275nM8J9YqqhG98bXfcER4KCiJ6M4_LjfjjRhn2omseE1GJdN$> ____

    __ __

    Obviously, I prefer my own code 😊____

    __ __

    In terms of appending data, could you explain your use case? I don’t know why you would want to append data from outside the archiver.____

    The archiver makes many assumptions about these files in terms of what data is in them dependent on the name of the file. I would be ____

    Careful about altering them.____

    __ __

    A simple solution can be to push data into the files using a PV! Note that the archiver doesn’t do any checks that timestamps correspond to the current time. ____

    Only that they are sequential. So as long as you are adding new events you can push extra events in.____

    __ __

    A second solution is to use the merge data from another cluster in the ETL process. You can see some documentation at https://urldefense.us/v3/__https://slacmshankar.github.io/epicsarchiver_docs/redundancy.html__;!!G_uCfscf7eWS!fvGpipmB7FsIn0XQsrK2CNDsPkaJASxwivcnk14AAtitA7Bx5MtNqAKPuyMGArcX3LaYhxBPVOQGyjxeSiH7yrtgUA$  <https://urldefense.us/v3/__https://slacmshankar.github.io/epicsarchiver_docs/redundancy.html__;!!G_uCfscf7eWS!b5YgNXV8LWnk1VGBnMvzi-WzDnvd0WROFnruh9I275nM8J9YqqhG98bXfcER4KCiJ6M4_LjfjjRhn2omscva_VhG$> ____

    I can probably create an example if you need it.____

    __ __

    You could then set up a dummy PV to push the data you want in, and allow the archiver ETL process to merge the data for you.____

    __ __

    I hope that helps.____

    Sky____

    __ __

    __ __

    __ __

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System