Re: Archiver Appliance - Limit on PVs

[Eusebio Naif Al-Soliman Fuentes via Tech-talk <tech-talk at aps.anl.gov>]

Hello again.

 

After trying to configure the Archiver Appliance on the VM without success, I got myself a test PC to install the Archiver Appliance natively. It has an Intel i7-4790 and 16 GB of RAM.

 

I have made a single machine installation and I am running the 4 Tomcats with 2GB for each one. With the default parameters I have encountered that I can barely archive 10000 PVs. I don't know if anyone can tell me which parameters I need to change in order to archive at least 50000 PVs updating every 500ms.

 

I have also noticed that the problem I had with the root partition filling up with memory in a short time is due to the archappl_install/engine/logs/catalina.out. Does anyone know what can be done to keep that file size under control?

 

Thanks in advance.

 

Eusebio.

---

Saludos/Best regards

University of Granada  

Eusebio Naif Al-Soliman Fuentes

PhD student

eunaif at correo.ugr.es

El 2025-02-03 12:16, Eusebio Naif Al-Soliman Fuentes escribió:

Hello Sky.

Regarding the disk speed, I've tested it with "hdparm" and the "/dev/sdb" partition (in which the "/var" directory is mounted) has the following results:

sudo hdparm -Tt /dev/sdb

/dev/sdb:
 Timing cached reads:   19774 MB in  1.99 seconds = 9935.12 MB/sec
 Timing buffered disk reads: 4084 MB in  3.00 seconds = 1361.22 MB/sec Also, the STS is, in theory, mounted in the "/var" disk, along side the MTS and the LTS. In the sampleStartup.sh file I've the following lines:

# Set the location of short term and long term stores; this is necessary only if your policy demands it
export ARCHAPPL_SHORT_TERM_FOLDER=/var/arch/sts/
export ARCHAPPL_MEDIUM_TERM_FOLDER=/var/arch/mts/
export ARCHAPPL_LONG_TERM_FOLDER=/var/arch/lts/

I don't know if there is another file which overwrites this configuration.

Finally, thanks for the advise with the JVM memory. I thought there was only 1 instance of Java instead of 4.

Thanks, Eusebio.

---
Saludos/Best regards

University of Granada

Eusebio Naif Al-Soliman Fuentes
MSc. Data Science and Computer Engineering

eunaif at correo.ugr.es

El 2025-01-31 14:32, Sky Brewer via Tech-talk escribió:

Hi Eusebio,

You shouldn't have any performance problems with that setup for the Archiver Appliance, and you shouldn't need more appliances unless you are in the 100 000 PV range (on a larger machine).

It's very hard to debug performance problems, even harder from a distance. Some initial ideas

*   How fast is the write interface to the var disk
*   Is the STS on the var disk? Using RAM or the primary disk might help here.
*   The JVM I believe takes whatever you give it, if you are running out of memory you should reduce what you gave it here. The sampleStartup.sh creates a tomcat per war file, so that's 4x whatever memory you changed in the script. Since you have 8GB, probably 1GB is a safe number.

Deployments of the Archiver Appliance unfortunately is still quite site dependent and varied. Something I hope to improve by providing docker containers etc.

Some ideas:
There is a docker container version with all webapps in one instance https://urldefense.com/v3/__https://urldefense.us/v3/__https:/*github.com/archiver-appliance/epicsarchiverap/pkgs/container/epicsarchiverap/336979389?tag=singletomcat-61c7442__;!!G_uCfscf7eWS!ZB5ixC9H9C2CsDUMxdsywR6idGZy4zDBWSUwLuOmWoPyYfQAJ_pV37sgR7lQIGSjEEbabw1POYKd8RRrqkao4Ugg$__;Lw!!D9dNQwwGXtA!X6m_pUxk0nwIIbWXuig5HBrRAi2e6B4-pHmrbV8JWWP0JlUp_g-qx481_gQB5aqMtwUa7k64dI64FkiUqXAMQSVq$

A variety of startup scripts in https://urldefense.com/v3/__https://urldefense.us/v3/__https:/*github.com/archiver-appliance/epicsarchiverap/tree/master/docs/docs/source/samples__;!!G_uCfscf7eWS!ZB5ixC9H9C2CsDUMxdsywR6idGZy4zDBWSUwLuOmWoPyYfQAJ_pV37sgR7lQIGSjEEbabw1POYKd8RRrqnP_f8Fw$__;Lw!!D9dNQwwGXtA!X6m_pUxk0nwIIbWXuig5HBrRAi2e6B4-pHmrbV8JWWP0JlUp_g-qx481_gQB5aqMtwUa7k64dI64FkiUqS8AYc95$

Can't remember if this is accessible outside of Sweden/Denmark, but ESS ansible deployment repository https://urldefense.com/v3/__https://urldefense.us/v3/__https:/*gitlab.esss.lu.se/ics-ansible-galaxy/ics-ans-role-epicsarchiverap__;!!G_uCfscf7eWS!ZB5ixC9H9C2CsDUMxdsywR6idGZy4zDBWSUwLuOmWoPyYfQAJ_pV37sgR7lQIGSjEEbabw1POYKd8RRrqsME4Gdv$__;Lw!!D9dNQwwGXtA!X6m_pUxk0nwIIbWXuig5HBrRAi2e6B4-pHmrbV8JWWP0JlUp_g-qx481_gQB5aqMtwUa7k64dI64FkiUqTM3EKxW$   (not updated for 2.05)

Hope that helps

Sky