Experimental Physics and
Industrial Control System

Hi,

>  maximum process limit of the PID of ordinary users in our system is only 4096 max user processes,

What's limited is the number of concurrent processes, not the value of the process id PID.

The display runtime is multi-threaded. It opens each display file in a background thread. If the display includes other "embedded" displays, a separate thread is used for each embedded display. This way, as many CPU cores as possible are used to open those display files in parallel. Compared to the older Eclipse-based "BOY" display runtime that opened all displays on the GUI thread we thus tend to get a faster display startup.

Depending on your top/htop/.. settings, you may see threads listed similar to processes as something like  "userland process threads". If you were hitting the limit of concurrent processes, you would get errors like "cannot create thread" and the display simply won't work, because that's a fatal error. I don't think that's happening in your setup.

> when the display file is opened, the share of java in the cpu will suddenly become very high, but it will slowly decline.

That is exactly what we expect. One or more threads will use as much CPU as possible to quickly open the display, and then the CPU load declines.

While executing a display, threads are used to process updates from PVs, to prepare the representation. In the case of plots, background threads are used to draw the updated plot. At one point, the updated data is shown on the screen, which has to be done in the one and only GUI thread. If you see close to 100% CPU usage in the "QuantumRenderer" or "XOrg" thread/process, that means the system can't draw any faster.

>  Java uses less CPU on another computer with dual monitors (Figure 2)

That suggests that the overall graphics performance is different on your two computers.

You can run "x11perf -all" on those two hosts and compare their basic X graphics performance.

Compared to for example EDM or MEDM where a button was mostly a flat rectangle and an LED was simply a filled circle, newer displays based on Qt or Gtk tend to have widgets rendered with gradients for the fill, with highlights in the border etc. The added workload tends to be handled by graphics hardware performing the gradient fills etc. You can run CSS/phoebus with 

You can try adding options

  -Djdk.gtk.version=2

vs

as well as

        

  -Dprism.forceGPU=true

vs

  -Dprism.forceGPU=false

to see if that makes a difference.

Finally, if your graphics system is a bottleneck and we simply can't display as many updates, you can throttle the display via these settings by increasing the various update delays:

# PV throttle (250)

org.csstudio.display.builder.runtime/update_throttle=250

# Display hesitation to accumulate updates (20)

org.csstudio.display.builder.representation/update_accumulation_time = 20

# Pause after updates (100)

org.csstudio.display.builder.representation/update_delay = 100

# Line Plot update throttle

org.csstudio.display.builder.representation/plot_update_delay = 250

# Image plot update throttle

org.csstudio.display.builder.representation/image_update_delay = 250

-Kay

From: chenhuiting at zjlab.org.cn <chenhuiting at zjlab.org.cn>

Sent: Wednesday, November 2, 2022 2:37 AM

To: Kasemir, Kay <kasemirk at ornl.gov>; tech-talk <tech-talk at aps.anl.gov>

Subject: Re: Re: [EXTERNAL] phoebus display runtime

 

Hi Kay

Thank you very much for your advices.

 

CS Studio (phoebus) software is indeed used on local computers. These computers are used as part of beamline control system. In general, in addition to the inherent operation of the system, only CSS software is used.

 

I have made some comparative analysis over the past day.The hosts participating in the comparison are all the same configuration, possibly different monitors. Figure 1 shows the computer status of one of the running phoebus. The display file that phoebus runs is just some basic widgets, such as Labels,grouping Containers,Rectangles,Action Buttons and so on. However, Java uses less CPU on another computer with dual monitors (Figure 2).

After more than 20 hours of observation, it is found that CSS software can release Java's CPU share when the window is minimized, while Java's CPU share will increase significantly when the normal window interface is restored. And when the display file is opened, the share of java in the cpu will suddenly become very high, but it will slowly decline. The above is just an observation of the CPU memory occupied by Java. By comparison, it is found that Java does not occupy a high percentage of the cup when the keyboard and mouse are not operated for a long time. Therefore, I preliminary judge that the Java of phoebus may only be one of the causes of the slow or even crash of the system, not the root cause. Of course, I still need to solve the problem that Java occupies too much CPU.

It is also observed that the pid value will continue to increase, while the maximum process limit of the PID of ordinary users in our system is only 4096 max user processes, while the max user processes of root users are 127282. Phoebus runs under ordinary users, while the pid of phoebus java often exceeds 4096. I suspect that the system is slow due to the limitation of ordinary users' pid max user processes and the dual impact of Java's pid value overflow, I plan to change the maximum number of processes for ordinary users of several computers, and then compare and analyze them.