Experimental Physics and
Industrial Control System

"Manoussakis, Adamandios via Tech-talk" <tech-talk at aps.anl.gov> · Wed, 2 Dec 2020 21:56:22 +0000

Well that makes sense but definitely troublesome if clocks go out of sync at all with any of the IOCs.

-----Original Message-----
From: Michael Davidsaver <mdavidsaver at gmail.com> 
Sent: Wednesday, December 2, 2020 1:23 PM
To: Manoussakis, Adamandios <manoussakis1 at llnl.gov>; EPICS tech-talk <tech-talk at aps.anl.gov>
Subject: Re: Epics Archiver not pushing to STS?

On 12/2/20 12:45 PM, Manoussakis, Adamandios via Tech-talk wrote:
> Thank you for the help everyone, I will make sure to correct the timing issue.  As for knowing the delta at which it breaks was just curiosity no big deal.

I think the requirement is that archived data have monotonically increasing timestamps.
So one scenario where data is lost would be when the IOC time jumps backwards after some some updates have already been archived.  Then an archiver will discard new updates until IOC time has "caught up" with the time of the last stored sample.

Another, more troublesome scenario is when the IOC time jumps forwards, and some updates are archived.  Then further updates will be dropped until the IOC has "caught up".
This can be nasty if the jump forward was eg. +10 years.

>   I did have one more question with regards to the archiver Han.  The 
> archiver is now posting to STS but when I look at the waveform record data, it looks very incorrect.  I looked at my archived DMM resistance measurements and it looked fine.  I also have CS Studio showing the waveform data live and it looks great but the archived data did not seem like it was storing the data correctly?
> 
>  
> 
> *From:* Jeong Han Lee <citadel.lee at gmail.com>
> *Sent:* Tuesday, December 1, 2020 5:13 PM
> *To:* Manoussakis, Adamandios <manoussakis1 at llnl.gov>
> *Cc:* EPICS tech-talk <tech-talk at aps.anl.gov>
> *Subject:* Re: Epics Archiver not pushing to STS?
> 
>  
> 
> I forgot to tell this. If you want to use that specific script directly outside the systemd service, you need "sudo" permission.
> 
>  
> 
> Your IOC and a host in an archiver appliance should have the same NTP server configuration. Technically, it doesn't matter which NTP server will be used. But better to use a single one instead of multiple ones.
> 
>  
> 
> I don't test the time offset. Maybe some number in the archiver appliance source?  Do you really need this?
> 
>  
> 
> Han
> 
>  
> 
>  
> 
> On Tue, Dec 1, 2020 at 4:09 PM Manoussakis, Adamandios via Tech-talk <tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>> wrote:
> 
>     Ah thanks Han I see the services running now with the status script when using sudo.  Also I seemed to have stumbled into the solution, the clock on the PXIe was set one hour ahead of my laptop.  Apparently the archiver will throw out data that is not close enough to its clock.  Does anyone know what the  margin is between timestamps before the archiver will not archive the data?
> 
>      
> 
>     Thanks for all your help as always!
> 
>      
> 
>      
> 
>      
> 
>      
> 
>     *From:* Jeong Han Lee <citadel.lee at gmail.com <mailto:citadel.lee at gmail.com>>
>     *Sent:* Monday, November 30, 2020 8:28 PM
>     *To:* Manoussakis, Adamandios <manoussakis1 at llnl.gov <mailto:manoussakis1 at llnl.gov>>
>     *Cc:* EPICS tech-talk <tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>>
>     *Subject:* Re: Epics Archiver not pushing to STS?
> 
>      
> 
>     Hi,
> 
>      
> 
>     You may check the storage status via
> 
>     bash /opt/epicsarchiverap/archappl.bash storage
> 
>     Sorry for the typo in the argument.
> 
>      
> 
>     Best,
> 
>     Han
> 
>      
> 
>      
> 
>     On Mon, Nov 30, 2020 at 8:15 PM Jeong Han Lee <citadel.lee at gmail.com <mailto:citadel.lee at gmail.com>> wrote:
> 
>         Hi,
> 
>          
> 
>         Please check the entire discussion at  
> https://epics.anl.gov/tech-talk/2020/msg01959.php 
> <https://epics.anl.gov/tech-talk/2020/msg01959.php>
> 
>          
> 
>         HTH,
> 
>         Han
> 
>          
> 
>          
> 
>         On Mon, Nov 30, 2020 at 5:39 PM Manoussakis, Adamandios via Tech-talk <tech-talk at aps.anl.gov <mailto:tech-talk at aps.anl.gov>> wrote:
> 
>             Hello All,
> 
>              
> 
>             I have been successful in getting the epics archiver appliance to archive PVs that are running in soft IOCs locally on the same machine as the archiver.  But now I tried to archive an IOC that is running on a subnet 192.168.40.x and it is says its connected/archiving but it never pushes to STS. It does show the details from the record for caLab:dmm shown below, which seems like its able to read from it. Settings for archiver appliance.xml below and also the archappl.conf environment variables set for the subnet to be seen.  I am able to also use caget through cmdline to get values from caLab:dmm and CSStudio is able to monitor them but the archiver doesn’t seem to be actually saving the data.
> 
>              
> 
>             <appliances>
> 
>                <appliance>
> 
>                  <identity>appliance0</identity>
> 
>                  <cluster_inetport>localhost:16670</cluster_inetport>
> 
>                  <mgmt_url>http://localhost:17665/mgmt/bpl</mgmt_url 
> <http://localhost:17665/mgmt/bpl%3c/mgmt_url>>
> 
>                  
> <engine_url>http://localhost:17666/engine/bpl</engine_url 
> <http://localhost:17666/engine/bpl%3c/engine_url>>
> 
>                  <etl_url>http://localhost:17667/etl/bpl</etl_url 
> <http://localhost:17667/etl/bpl%3c/etl_url>>
> 
>                  
> <retrieval_url>http://localhost:17668/retrieval/bpl</retrieval_url 
> <http://localhost:17668/retrieval/bpl%3c/retrieval_url>>
> 
>                  
> <data_retrieval_url>http://localhost:17668/retrieval</data_retrieval_u
> rl <http://localhost:17668/retrieval%3c/data_retrieval_url>>
> 
>                </appliance>
> 
>             </appliances>
> 
>              
> 
>              
> 
>             JAVA_HOME="/usr/lib/jvm/java-11-openjdk-amd64"
> 
>             JAVA_OPTS="-Djava.security.egd=file:/dev/./urandom"
> 
>             CATALINA_HOME="/usr/share/tomcat9"
> 
>             CATALINA_OPTS="-XX:MaxMetaspaceSize=256m -Xms512m -Xmx512m -XX:+UseG1GC -ea"
> 
>              
> 
>             ARCHAPPL_APPLIANCES="/opt/epicsarchiverap/appliances.xml"
> 
>             ARCHAPPL_POLICIES="/opt/epicsarchiverap/policies.py"
> 
>             ARCHAPPL_PROPERTIES_FILENAME="/opt/epicsarchiverap/archappl.properties"
> 
>             ARCHAPPL_MYIDENTITY="appliance0"
> 
>             ARCHAPPL_STORAGE_TOP="/arch"
> 
>             ARCHAPPL_SHORT_TERM_FOLDER="/arch/sts/ArchiverStore"
> 
>             ARCHAPPL_MEDIUM_TERM_FOLDER="/arch/mts/ArchiverStore"
> 
>             ARCHAPPL_LONG_TERM_FOLDER="/arch/lts/ArchiverStore"
> 
>             EPICS_CA_ADDR_LIST=192.168.40.200
> 
>             EPICS_CA_AUTO_ADDR_LIST=YES
> 
>              
> 
>              
> 
>             *caLab:dmm PV details section showing Never for STS push*
> 
>              
> 

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System