RE: CSS (Eclipse and Phoebus) super slow on a server with multiple users

["Jokinen Antti \(F4E\) via Tech-talk" <tech-talk at aps.anl.gov>]

Hello. 

Please allow me to join, as I believe that I am the correct person to answer your questions, as I have installed the machine and have been doing maintenance on it.

>wrt. RAM how much is each java process being allowed?  aka. the -Xmx... argument.

Currently that argument is at 2048 MB, as I've increased it from 1024 MB.

>You can divide 3/4 total RAM size by this and estimate a limit on concurrent java processes before swapping must happen (at which point performance will degrade).

The machine has 112 GB of RAM in total. As I am writing, there are 15 instances of cs-studio running, so we should be ok in that regard. Both top and free report that no swap memory is being used.

>Could you expand on "Firefox and other applications are working as expected".
>Do you mean in a situation where every user is simultaneously running both firefox and phoebus?

The "other applications" mean "besides CSS and Phoebus". Firefox, the MATE terminal, Caja (for example) and other applications feel responsive, even when CSS is not being responsive.

>Does "user count" mean number of cs-studio processes?  number of windows?  number of users logged in?

In this context, number of cs-studio processes running at the same time, by different users.

> When you say "RAM 80%" free.  What specific metric are you referring to?

Basically what is reported in 'free' as 'free' memory, so total - (used + buffered + cached memory). Free doesn't report any swap memory being used.

> Also, are you looking at network bandwidth utilization?

Yes, we are. It doesn't seem to be that much being used (around 200-300 KB received and sent) since other applications are more responsive than cs-studio, opening and closing cs-studio makes it more responsive for a while.

>Are there any network filesystem mounts involved?

Not directly. There is another virtual machine that has a PostgreSQL server, to which the data viewer in cs-studio connects and this one uses an NFS drive to store the data. Although most users have the data viewer open, I have observed the same unresponsive behaviour of cs-studio even without the data viewer being opened.

I hope this clarifies the situation. Thank you for your attention.

André Sancho Duarte

-----Original Message-----
From: Michael Davidsaver <mdavidsaver at gmail.com> 
Sent: Thursday, 18 November 2021 18:49
To: Jokinen Antti (F4E) <Antti.Jokinen at f4e.europa.eu>
Cc: tech-talk at aps.anl.gov
Subject: Re: CSS (Eclipse and Phoebus) super slow on a server with multiple users

On 11/18/21 1:25 AM, Jokinen Antti (F4E) via Tech-talk wrote:
> Hi all,
> 
>  
> 
> We have established a remote server for users to connect from their 
> home. The remote server itself is pretty beasty machine and all of its 
> CentOS tools,

How does "pretty beasty" translate into specs?

wrt. RAM how much is each java process being allowed?  aka. the -Xmx... argument.

You can divide 3/4 total RAM size by this and estimate a limit on concurrent java processes before swapping must happen (at which point performance will degrade).


> Firefox and other applications are working as expected when connected via X2GO to it.

Could you expand on "Firefox and other applications are working as expected".
Do you mean in a situation where every user is simultaneously running both firefox and phoebus?


> However there is something strange going on with CSS (both Eclipse and Phoebus). If there are only a few users on the server logged on remotely, CSS works pretty smoothly but once the user count goes higher e.g. up to 15 or so, something gets sticky and CSS basically slows down (on all users) to being unusable.

Does "user count" mean number of cs-studio processes?  number of windows?  number of users logged in?


> The server key parameters (total server CPU usage <10%, RAM 80% free) are all green. It looks like the problem is not coming from these.

When you say "RAM 80%" free.  What specific metric are you referring to?
Allocated virtual memory?  resident memory?  Does this include swap?

Also, are you looking at network bandwidth utilization?

Are there any network filesystem mounts involved?


> My question is that what could be causing the slowdown? Is there something we have overlooked? Could it be JAVA? HDD/network?
> 
>  
> 
> Thanks a lot!
> 
> Antti  (IFMIF)
>