Re: How to use Archiver Appliance to archive PVs of only the latest period of time

["Shankar, Murali" <mshankar@slac.stanford.edu>]

>> Should I use a Python script to directly operate the .pb files in the file system, or to interact with the Archiver Appliance BPL Servlet/HTTP interface

You can directly operate on the .pb files in the file system. As long as the appliance is not writing to these files; which it would not if the files are for older partitions; you can safely delete the files from the file system using os.remove.

Regards,

Murali

---

From: Wang, Lin <wanglin@ihep.ac.cn>
Sent: Monday, October 30, 2017 3:30 AM
To: Shankar, Murali
Cc: tech-talk@aps.anl.gov
Subject: Re: How to use Archiver Appliance to archive PVs of only the latest period of time

 

Hello Murali,

Sorry for following up this thread so late.

> You could then delete the older data using a Python script.

But how should I delete the older data using a Python script? Should I use a Python script to directly operate the .pb files in the file system, or to interact with the Archiver Appliance BPL Servlet/HTTP interface?

Thanks,

Lin

-----Original Messages-----
From:"Shankar, Murali" <mshankar@slac.stanford.edu>
Sent Time:2017-09-14 12:04:32 (Thursday)
To: "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>, "wanglin@ihep.ac.cn" <wanglin@ihep.ac.cn>
Cc:
Subject: Re: How to use Archiver Appliance to archive PVs of only the latest period of time

Hi Lin,

>> In order to save storage space, we just want to archive PVs of the latest one month.

You should be able to create a policy for these 5000 PV's with a partition granularity of a month. That is, in your policies.py file, your LTS (or MTS) can be something like

pb://localhost?name=LTS&rootFolder=${ARCHAPPL_LONG_TERM_FOLDER}&partitionGranularity=PARTITION_MONTH.

If you add PV's to the archiver using this policy, you should be able to store data in chunks of one month each. 

>> the requirement may be adjusted to three or six months in future)

You could then delete the older data using a Python script. Or, if you decided to keep, let's say four months you could add a blackhole plugin after this plugin. For example, 

LTS -> partitionGranularity=PARTITION_MONTH​&hold=5&gather=1 followed by a 

Final -> 'blackhole://localhost'

would keep at least (5-1)=4 months data in your LTS.

For more details, please see http://slacmshankar.github.io/epicsarchiver_docs/api/edu/stanford/slac/archiverappliance/PlainPB/PlainPBStoragePlugin.html​

Hope that helps.

Regards,

Murali