EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024 
<== Date ==> <== Thread ==>

Subject: Re: Linux memory problems/questions
From: Michael Davidsaver via Tech-talk <tech-talk at aps.anl.gov>
To: Mark Rivers <rivers at cars.uchicago.edu>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>
Date: Thu, 20 May 2021 08:15:34 -0700
On 5/20/21 6:43 AM, Mark Rivers via Tech-talk wrote:
> Folks,
> 
>  
> 
> I am trying to help someone with a problem with memory consumption with an ADSpinnaker camera.  The problem is described in this Github issue:
> 
> https://github.com/areaDetector/ADSpinnaker/issues/7 <https://github.com/areaDetector/ADSpinnaker/issues/7>
> 
>  
> 
> I have been able to reproduce the problem with a BlackFly camera.
> 
>  
> 
> The problem occurs when the camera is in software trigger mode, in which a write to the TriggerSoftware PV causes the camera to acquire the next frame.  The problem is that the resident memory used by the ADSpinnaker application appears to increase slowly, but without bounds, in this mode.  The user has reported that with 3 IOCs running the system memory runs out after a few days, and it hangs up.
> 
>  
> 
> These are the results of 3 acquisitions, each of 1000 frames in this mode.  I used “top” to get the value of VIRT and RES at the beginning and end of each acquisition.  The value of RES was gradually increasing during the entire acquisition.

As I understand it.

VIRT == Sum of all memory pages mapped to a process, some of which
        may be paged out, or not yet used.

RES == Sum of all memory pages not paged out, and which have been
       actually accessed.

If you watch long enough you will probably see RES == VIRT, at which
point VIRT will increase.  Although for the purposes of identifying a
memory leak, VIRT isn't all that interesting.


> VIRT
> 
> 	
> 
> RES
> 
> Start
> 
> 	
> 
> 7508244
> 
> 	
> 
> 306452
> 
> Stop
> 
> 	
> 
> 7508244
> 
> 	
> 
> 308624
> 
> Difference
> 
> 	
> 
> 0
> 
> 	
> 
> 2172
> 
>  
> 
>  
> 
> 	
> 
> VIRT
> 
> 	
> 
> RES
> 
> Start
> 
> 	
> 
> 7508244
> 
> 	
> 
> 309124
> 
> Stop
> 
> 	
> 
> 7508244
> 
> 	
> 
> 311240
> 
> Difference
> 
> 	
> 
> 0
> 
> 	
> 
> 2116
> 
>  
> 
>  
> 
> 	
> 
> VIRT
> 
> 	
> 
> RES
> 
> Start
> 
> 	
> 
> 7508244
> 
> 	
> 
> 311468
> 
> Stop
> 
> 	
> 
> 7508244
> 
> 	
> 
> 313536
> 
> Difference
> 
> 	
> 
> 0
> 
> 	
> 
> 2068
> 
>  
> 
> Note that the virtual memory (VIRT) is not increasing at all.  I think this says that there is not a normal memory leak.  However, the RES memory is increasing by about 2000 KB for 1000 images, or 2 KB per image.  This  increase per image is much less than the size of the image, which is 2 MB.
> 
>  
> 
> This is the result when using the normal internal trigger (free-run) mode of the camera.  Note that RES actually decreased slightly, so there is no problem in this mode.
> 
>  
> 
>  
> 
> 	
> 
> VIRT
> 
> 	
> 
> RES
> 
> Start
> 
> 	
> 
> 7508244
> 
> 	
> 
> 313788
> 
> Stop
> 
> 	
> 
> 7508244
> 
> 	
> 
> 313628
> 
> Difference
> 
> 	
> 
> 0
> 
> 	
> 
> -160
> 
>  
> 
> Can someone explain what can cause RES to increase without bounds, while VIRT is not changing at all?
> 
>  
> 
> Thanks,
> 
> Mark
> 
>  
> 


References:
Linux memory problems/questions Mark Rivers via Tech-talk

Navigate by Date:
Prev: Re: [EXTERNAL] asyn vxi11 rpc issue cross compile for yocto powerpc64-fsl-linux Hu, Yong via Tech-talk
Next: Re: [EXTERNAL] asyn vxi11 rpc issue cross compile for yocto powerpc64-fsl-linux Heinz Junkes via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024 
Navigate by Thread:
Prev: Linux memory problems/questions Mark Rivers via Tech-talk
Next: asyn vxi11 rpc issue cross compile for yocto powerpc64-fsl-linux junkes via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  <20212022  2023  2024 
ANJ, 20 May 2021 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·