Experimental Physics and
Industrial Control System

"Jokinen Antti \(F4E\) via Tech-talk" <tech-talk at aps.anl.gov> · Mon, 13 Dec 2021 14:28:05 +0000

>What is "#sar -d"?  "average await" sounds like a disk metric?
>Is this related to eg. https://linux.die.net/man/1/atopsar

After some googling I found this "sar" command and it has proven to be quite useful. I'm not sure if it's related to "atopsar" but after reading quickly about it, it seems that they probably provide similar info from the system.

I think we just got sort of lucky and were experimenting with various different potentially speed boosting settings.  Since then it looks like our celebration was slightly premature as we did not get rid of the slowness completely (although now it is much better). It stills shows up but only after, say, a few days of operation. Also it seems to come and go randomly. 

-----Original Message-----
From: Michael Davidsaver <mdavidsaver at gmail.com> 
Sent: Wednesday, 1 December 2021 17:58
To: Jokinen Antti (F4E) <Antti.Jokinen at f4e.europa.eu>
Cc: tech-talk at aps.anl.gov; Sancho Duarte Andre (F4E-Ext) <Andre.Sancho at ext.f4e.europa.eu>
Subject: Re: CSS (Eclipse and Phoebus) super slow on a server with multiple users

On 11/29/21 5:27 AM, Jokinen Antti (F4E) wrote:
> On the other hand we managed to improve the remote server performance significantly by updating the kernel plus enabling VT-X on the virtualization software. FYI according to "#sar -d" the average await parameter was ~2000ms before whereas now it's always <30ms. Additionally now the server seems not to struggle under a heavy load anymore and for example in the remote user point of view, all programs are evenly responsive/unresponsive. 

What is "#sar -d"?  "average await" sounds like a disk metric?
Is this related to eg.

https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flinux.die.net%2Fman%2F1%2Fatopsar&amp;data=04%7C01%7CAntti.Jokinen%40f4e.europa.eu%7C59fac6c0768f458ea6e508d9b4ebbda8%7C687a8c5c348a4796ab753246db849e8b%7C0%7C0%7C637739746871826503%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=rJVGVVxttSqbKFEUsFvh8%2BnSoOeB5JwjhB8W2bxkJME%3D&amp;reserved=0

I ask because I've run into odd performance regressions with software running in VMs before, and struggled to understand what was going on.  So I'm interested in knowing what you looked at before noticing that the vt-x CPU extension wasn't being used?

eg. one thing I look for on Linux+QEMU is if the VM process opens "/dev/kvm".  (and obviously whether "/dev/kvm" exists)

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System