Re: Archiver Appliance - Limit on PVs

[Eusebio Naif Al-Soliman Fuentes via Tech-talk <tech-talk at aps.anl.gov>]

Hello everyone.

 

I've tried to change those parameters and also give more memory to the JVM by modifying the JAVA_OPTS variable in the sampleStartup.sh file. The performance has improved a little bit but I'm not getting the desired one.

 

Things get worse when I compare it to the old Archive Engine, which running on another VM of similar specifications, has better performance.

 

In particular, I'm running both services in two different VM but with similar specifications.

 

- SO: Rocky Linux 9.5.

- Number of CPUs: 4.

- RAM: 8 GB.

- Primary disk with 40 GB.

- Secondary disk with 256 GB in which "/var" is mounted.

 

With that, I'm able to archive 5000 sinusoidal PVs updating every 500 ms in the VM with the Archive Engine but I'm experiencing data losses with the Archiver Appliance. Also, the VM with the Archiver Appliance is getting full of memory in the root partition even though I have defined the short, medium and long term folders on the "/var" directory, which is mounted on the secondary disk.

 

I'm thinking that maybe the comparison is unfair with the Archiver Appliance, because the documentation describes that it's possible to deploy multiple appliances, but how can this be done? On each device in the cluster a full instance of the Archiver Appliance is deployed? Or each of the WAR Tomcats can be distributed into different devices? Because inside the appliance.xml there are fields for the IP address of each system (ETL, management, engine & retrieval).

 

And finally, is there a guide that describes how to perform this complex deployment?

 

Thanks in advance.

 

Eusebio.

 

---

Saludos/Best regards

University of Granada  

Eusebio Naif Al-Soliman Fuentes
MSc. Data Science and Computer Engineering

eunaif at correo.ugr.es

El 2025-01-30 10:16, Eusebio Naif Al-Soliman Fuentes via Tech-talk escribió:

Hello Sky.

Thanks for your response. I'll try to change those parameters to have a better performance.

Eusebio.

---
Saludos/Best regards

University of Granada

Eusebio Naif Al-Soliman Fuentes
MSc. Data Science and Computer Engineering

eunaif at correo.ugr.es

El 2025-01-29 13:42, Sky Brewer via Tech-talk escribió:

Hi Eusebio,

You probably want to change some options in the properties file. The default file is at

https://urldefense.com/v3/__https://urldefense.us/v3/__https:/*github.com/archiver-appliance/epicsarchiverap/blob/master/src/sitespecific/default/classpathfiles/archappl.properties__;!!G_uCfscf7eWS!ZDBlyldkC0kJvuc3n8z9xDf3skxcaYxblRVjUpIv_HUnNRqJws62_802ixxkHU8RUJ4krRt2zdYohlEI__bIr9Wl$__;Lw!!D9dNQwwGXtA!T58fUAHdAiEMGLrplweuWWKWmq25Bc1fmc7tw6rFNfbGAiJ3dRsxVoRW9f8VeXnk2jdatboKLz6Xi1xUOoqVQ0bl$

Changing each property can then have different performance impact, for instance increasing org.epics.archiverappliance.config.PVTypeInfo.sampleBufferCapacityAdjustment
Will increase the memory usage.

You probably want to increase

org.epics.archiverappliance.mgmt.MgmtRuntimeState.archivePVWorkflowBatchSize

which has a default of 1000. It's how many PVs are acceptable to be in the intialSampling stage.

And decrease

org.epics.archiverappliance.mgmt.bpl.ArchivePVAction.minimumSamplingPeriod = 0.1

which specifies the minimumSamplingPeriod.

In some sense there is not a limit on how small this can be, it's more how much memory you have for the Archiver Appliance to run to. If you saturate the memory, then it will start dropping events.

Hope this helps

Sky