EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
<== Date ==> <== Thread ==>

Subject: Re: EPICS Archiver Appliance - Auto-delete Old Data
From: "Wilson, Andy \(DLSLtd, RAL, LSCI\) via Tech-talk" <tech-talk at aps.anl.gov>
To: "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>, "Harding, Rebecca (STFC,RAL,CLF)" <rebecca.harding at stfc.ac.uk>
Date: Mon, 1 Aug 2022 10:07:28 +0000
Hi Rebecca,

You could do this by configuring the storage URLs for a PV via policies.py, with the last entry as "blackhole://localhost"; then the ETL will delete the data under the specified conditions.


Here's a bit of policies.py where for certain PVs we delete the data after 6 months.
        if userPolicyOverride == 'some-example-policy':
            keep_months = 6
        [...]

        # hold - gather is the minimum number of files that will be retained
        # Empirically it seems to hold one less than requested, so we add two.
        hold = keep_months + 2
        gather = 1

        sts_url = 'pb://localhost?name=STS&rootFolder=${ARCHAPPL_SHORT_TERM_FOLDER}&partitionGranularity=PARTITION_15MIN&consolidateOnShutdown=true'
        mts_url = 'pb://localhost?name=MTS&rootFolder=${ARCHAPPL_MEDIUM_TERM_FOLDER}&partitionGranularity=PARTITION_DAY&hold=2&gather=1'
        lts_url = 'pb://localhost?name=LTS&rootFolder=${ARCHAPPL_LONG_TERM_FOLDER}&partitionGranularity=PARTITION_MONTH&hold=%d&gather=%d' % (hold, gather)
        blackhole_url = 'blackhole://localhost'
        data_stores = [sts_url, mts_url, lts_url, blackhole_url]

Kind regards,
Andy

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Rebecca Harding - STFC UKRI via Tech-talk <tech-talk at aps.anl.gov>
Sent: Monday, August 1, 2022 10:25
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: EPICS Archiver Appliance - Auto-delete Old Data
 

Hello TechTalk,

 

I am using EPICS Archiver Appliance and I am interested in automatically deleting data that is older than a certain threshold. E.g. only keep the last 2 years of data.

 

Has anybody already done this? Does anyone have any ideas about a good way to approach this problem?

 

Kind regards,

 

Rebecca Harding (she/her)

Controls Software Engineer

Central Laser Facility

Science and Technology Facilities Council

rebecca.harding at stfc.ac.uk

01235 445129

 

ukri-stfc

 

This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. 

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 


Replies:
RE: EPICS Archiver Appliance - Auto-delete Old Data Rebecca Harding - STFC UKRI via Tech-talk
References:
EPICS Archiver Appliance - Auto-delete Old Data Rebecca Harding - STFC UKRI via Tech-talk

Navigate by Date:
Prev: EPICS Archiver Appliance - Auto-delete Old Data Rebecca Harding - STFC UKRI via Tech-talk
Next: RE: EPICS Archiver Appliance - Auto-delete Old Data Rebecca Harding - STFC UKRI via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
Navigate by Thread:
Prev: EPICS Archiver Appliance - Auto-delete Old Data Rebecca Harding - STFC UKRI via Tech-talk
Next: RE: EPICS Archiver Appliance - Auto-delete Old Data Rebecca Harding - STFC UKRI via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  <20222023  2024 
ANJ, 14 Sep 2022 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·