EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  <2024 Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  <2024
<== Date ==> <== Thread ==>

Subject: Re: ADVimba CPU usage
From: Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>
To: Sandeep Kumar Malu - STFC UKRI <sandeep-kumar.malu at stfc.ac.uk>
Cc: EPICS tech-talk <tech-talk at aps.anl.gov>
Date: Wed, 17 Jan 2024 12:53:25 +0000
Hi Sandeep,

You did not create an issue an ADVimba as I requested.  Please do that.

Your OPI screen shot shows that when you have the CPU problem that ADVimba driver is not getting any frames and neither are the plugins.  This means that these threads are unlikely to be causing the problem.  The "top -H" screen shot shows that all of the CPU is taken up by the scanOnce and CAC-event tasks.  This indicates that some records are processing very fast, and there is a lot of Channel Access activity.  You need to figure out what records and why.  I suggest you do the following:

  • Run the IOC that is provided in ADVimba.  It seems likely to me that the records that are causing the problem are not the ones in the base IOC, but something else you have added locally.  See if you have the same problem when you run the distributed IOC.
  • It looks like you are running Phoebus as the OPI?  Please use the distributed top-level ADVimba.bob screen, since it has additional diagnostic information that your screen does not have.
  • For your real IOC try commenting out any extra records that you have loaded locally to see which might be causing the problem.

Mark



From: Sandeep Kumar Malu - STFC UKRI <sandeep-kumar.malu at stfc.ac.uk>
Sent: Wednesday, January 17, 2024 3:47 AM
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: EPICS tech-talk <tech-talk at aps.anl.gov>
Subject: Re: ADVimba CPU usage
 
Hi Mark

Thanks for your response.

Please provide the following:
  • The versions of ADCore and ADVimba
    • ADCore - R3-8, ADVimba - Ahead of R1-3 at commit 17dcdb1 dated Jun 30, 2022
  • Is this Windows or Linux?
    • Linux
  • How many cores on the IOC machine?
    • 20 cores
  • When you say 75% of the CPU do you mean 75% of a single core, or 75% of all cores?
    • 75% of all cores... as evident from command "glances"
  • When you remove the external trigger what frame rate are the cameras getting?  It should be 0.
    • FrameRate_RBV is 10 (that's what it's set. I tried setting framerate to zero and still the same CPU usage) but ImageRate_RBV is 0.
  • A screenshot of the AVVimba screen when the problem is happening
    • Attached with Mail.
  • A screenshot of the commonPlugins OPI screen when the problem is happening so we can see what plugins are running and at what rate.
    • I am using Trans, Stats, Overlay and NDStdArray Plugins and the rate is 0 under this scenario.
  • A screenshot of "top -h" in the IOC computer when the problem is happening on.  This will show us what threads are using the CPU.
    • Attached with the mail
Many Thanks
Sandeep

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: 16 January 2024 20:57
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: EPICS tech-talk <tech-talk at aps.anl.gov>; Malu, Sandeep Kumar (-,RAL,CLF) <sandeep-kumar.malu at stfc.ac.uk>
Subject: Re: ADVimba CPU usage
 
The command I meant is “top -H”, not -h.

Mark


On Jan 16, 2024, at 8:47 AM, Mark Rivers via Tech-talk <tech-talk at aps.anl.gov> wrote:


Hi Sandeep,

Please create an issue for this problem here: https://github.com/areaDetector/ADVimba/issues

Please provide the following:
  • The versions of ADCore and ADVimba
  • Is this Windows or Linux?
  • How many cores on the IOC machine?
  • When you say 75% of the CPU do you mean 75% of a single core, or 75% of all cores?
  • When you remove the external trigger what frame rate are the cameras getting?  It should be 0.
  • A screenshot of the AVVimba screen when the problem is happening
  • A screenshot of the commonPlugins OPI screen when the problem is happening so we can see what plugins are running and at what rate.
  • A screenshot of "top -h" in the IOC computer when the problem is happening on.  This will show us what threads are using the CPU.

I just did a quick test of a GC1380CH camera with ADVimba.  It used 30% of 1 core when running at 30 frames/s.  I had only NDPluginPVA running.  When I stopped the camera acquisition the CPU dropped to 0%, as I expected.  When I selected external trigger but have no trigger pulses arriving I also get 0 frames/s and 0% CPU, as I expect.

Mark


From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Sandeep Kumar Malu - STFC UKRI via Tech-talk <tech-talk at aps.anl.gov>
Sent: Tuesday, January 16, 2024 6:08 AM
To: EPICS tech-talk <tech-talk at aps.anl.gov>
Subject: ADVimba CPU usage
 
Hi All

I have been using ADVimba Areadetector driver for AVT MakoG234C (1936x1216) Cameras. 15 cameras are running with external triggers of 10 fps and the CPU usage is ~27%.
However, when I deployed those 15 cameras keeping the external trigger (from the delay generator) off, the CPU usage drastically goes up to ~75%. Also, I noticed the same behaviour when Acquire is OFF or switched to Software Trigger source.
 
Has anyone seen this behaviour when the driver is not actually acquiring the images? Is there any solution to this drastic CPU usage?

Many Thanks
Sandeep

Replies:
Re: ADVimba CPU usage Sandeep Kumar Malu - STFC UKRI via Tech-talk
References:
ADVimba CPU usage Sandeep Kumar Malu - STFC UKRI via Tech-talk
Re: ADVimba CPU usage Mark Rivers via Tech-talk
Re: ADVimba CPU usage Mark Rivers via Tech-talk
Re: ADVimba CPU usage Sandeep Kumar Malu - STFC UKRI via Tech-talk

Navigate by Date:
Prev: Re: Questions about creating a Model 3 Driver for stepper motor axis Torsten Bögershausen via Tech-talk
Next: RE: Questions about creating a Model 3 Driver for stepper motor axis Paul-Elie GUILLOT via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  <2024
Navigate by Thread:
Prev: Re: ADVimba CPU usage Sandeep Kumar Malu - STFC UKRI via Tech-talk
Next: Re: ADVimba CPU usage Sandeep Kumar Malu - STFC UKRI via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  <2024
ANJ, 18 Jan 2024 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·