1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 <2022> 2023 2024 | Index | 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 <2022> 2023 2024 |
<== Date ==> | <== Thread ==> |
---|
Subject: | RE: EPICS Archiver Appliance - Auto-delete Old Data |
From: | Rebecca Harding - STFC UKRI via Tech-talk <tech-talk at aps.anl.gov> |
To: | "andrew.wilson at diamond.ac.uk" <andrew.wilson at diamond.ac.uk>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov> |
Date: | Mon, 1 Aug 2022 16:04:27 +0000 |
Hi Andy, That’s brilliant. I will look into this. Thank you for your help. Kind regards, Rebecca Harding (she/her) Controls Software Engineer Central Laser Facility Science and Technology Facilities Council 01235 445129 From: Wilson, Andy (DLSLtd,RAL,LSCI) <andrew.wilson at diamond.ac.uk>
Hi Rebecca, You could do this by configuring the storage URLs for a PV via policies.py, with the last entry as "blackhole://localhost"; then the ETL will delete the data under the specified
conditions. Some of the docs for putting together the URLs are here
https://slacmshankar.github.io/epicsarchiver_docs/api/edu/stanford/slac/archiverappliance/PlainPB/PlainPBStoragePlugin.html Here's a bit of policies.py where for certain PVs we delete the data after 6 months. if userPolicyOverride == 'some-example-policy':
keep_months = 6
[...]
# hold - gather is the minimum number of files that will be retained
# Empirically it seems to hold one less than requested, so we add two.
hold = keep_months + 2
gather = 1
sts_url = 'pb://localhost?name=STS&rootFolder=${ARCHAPPL_SHORT_TERM_FOLDER}&partitionGranularity=PARTITION_15MIN&consolidateOnShutdown=true'
mts_url = 'pb://localhost?name=MTS&rootFolder=${ARCHAPPL_MEDIUM_TERM_FOLDER}&partitionGranularity=PARTITION_DAY&hold=2&gather=1'
lts_url = 'pb://localhost?name=LTS&rootFolder=${ARCHAPPL_LONG_TERM_FOLDER}&partitionGranularity=PARTITION_MONTH&hold=%d&gather=%d' % (hold, gather)
blackhole_url = 'blackhole://localhost'
data_stores = [sts_url, mts_url, lts_url, blackhole_url] Kind regards, Andy From: Tech-talk <tech-talk-bounces at aps.anl.gov>
on behalf of Rebecca Harding - STFC UKRI via Tech-talk <tech-talk at aps.anl.gov> Hello TechTalk, I am using EPICS Archiver Appliance and I am interested in automatically deleting data that is older than a certain threshold. E.g. only keep the last 2 years of data. Has anybody already done this? Does anyone have any ideas about a good way to approach this problem? Kind regards, Rebecca Harding (she/her) Controls Software Engineer Central Laser Facility Science and Technology Facilities Council 01235 445129 This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify
the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own
virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. |