Re: Freeing allocated buffers in areaDetector HDF5 Plugin

["Marco A. Barra Montevechi Filho via Tech-talk" <tech-talk at aps.anl.gov>]

Hi, Mark! Thanks for the answer.

For context, the reason we changed from stream mode to capture mode was because our filesystem is mounted via web and we wanted to make one single and as-quick-as-possible access to the storage, instead of accessing it for the whole acquisition.

After reading your e-mail and the stackoverflow post about how memory works on linux, i figured if i instantiated another simDetector and made it request more memory, the first instance usage would decrease if i used the EmptyFreeList record correctly.
So:
- in instance 1 i started capture mode and stored 1029 images. Saved to disk and dbpf in both HDF1 and cam1 EmptyFreeList:
    epics> dbpf 13SIM2:cam1:EmptyFreeList 1

    DBF_STRING:         ""        

    epics> dbpf 13SIM2:HDF1:EmptyFreeList 1

    DBF_STRING:         ""        

- Percentage of instance 1 memory usage is now 14% according to htop
- Start instance 2. Configure HDF5 plugin for capture mode. Start acquisition.
- Instance 1 memory consuption keeps at 14%. Instance 2 memory consuption raises untill its too much and process is killed by system.

am i doing something wrong?

---

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: 03 May 2023 21:43
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>; Marco A. Barra Montevechi Filho <marco.filho at lnls.br>
Subject: Re: Freeing allocated buffers in areaDetector HDF5 Plugin

 

Hi Marco,

"capture" mode is rarely needed.  When you write 1 to Capture it saves all frames into a buffer.  When you press the Save button (WriteFile PV) the frames are written to disk and what is complete they returned to the AD driver's free pool.

If you want to return that memory to the OS then you need to write 1 to the EmptyFreeList PV, which is shown in the Buffers section of the driver screen.  That Buffer section also shows the total number of buffers (NDArrays) allocated and the total memory allocations.  When you press that Empty button the allocated memory (PoolUsedMem PV) will drop to a small number.

Note that "top"may still show a large amount of memory allocated to the detector IOC.  That is because calling free() in Linux does not necessarily return the memory to the operating system.  Here is an explanation:

https://stackoverflow.com/questions/47384177/memory-usage-doesnt-decrease-upon-calling-free

"capture" mode was created when the plugin queue size was fixed at iocInit.  Now that it can be changed at run-time there is very little need for "capture" mode.  Instead, most people use "stream" mode and make the queue size large enough that frames are not dropped.  "stream" mode writes the file at the same time that it is buffering frames, so it is generally faster and requires less memory than "capture" mode.  Do you have a special requirement that calls for "capture" mode?

> Consequently, when allocated buffer is bigger than available memory, application freezes my machine.

When you start your driver you can specify on the iocsh command line the maximum amount of memory it is allowed to use.   If you set that to a value less than your system RAM then it will generate an error but won't freeze your machine.

Mark

---

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Marco A. Barra Montevechi Filho via Tech-talk <tech-talk at aps.anl.gov>
Sent: Wednesday, May 3, 2023 6:41 PM
To: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: Freeing allocated buffers in areaDetector HDF5 Plugin

 

Hi, all!

Im doing some proof of concepts with areaDetector cloned directly from master (commit 78fae55898b9028e756eb080367cd6aad8913b54) and the plugins together with EPICS_BASE 7.

When using the HDF5 plugin and saving with the capture mode, documentation (https://areadetector.github.io/master/ADCore/NDPluginFile.html?highlight=allocate) tells me a buffer will be allocated in memory before saving occurs. Consequently, when allocated buffer is bigger than available memory, application freezes my machine.

The thing is: after the saving occurs, the buffer doesnt seem to be de-allocated. I say that because htop shows memory usage growing continuously when acquire is happening with capture mode on, but memory usage does not decrease once saving is done. I tried using the <PREFIX>FlushNow PV but i only get this error:

epics> dbpf 13SIM2:HDF1:FlushNow 1

DBF_STRING:         "Flush"  

epics> 2023/05/03 20:35:02.845 NDFileHDF5:writeInt32: ERROR status=3, function=168, value=1 old=0

2023/05/03 20:35:02.845 13SIM2:HDF1:FlushNow devBusyAsyn::processCallbackOutput process error

epics>

epics>

epics>

epics> dbpf 13SIM2:HDF1:FlushNow 1

DBF_STRING:         "Flush"  

epics> 2023/05/03 20:35:34.577 NDFileHDF5:writeInt32: ERROR status=3, function=168, value=1 old=0

2023/05/03 20:35:34.577 13SIM2:HDF1:FlushNow devBusyAsyn::processCallbackOutput process error

Is this the correct usage of the PV? I also tried using <PREFIX>NumFramesFlush, but got no result.
Is there any recommended way i can de-allocate this buffer whenever i want? Until now, i need to restart the application every time i want my memory back.

Thanks in advance for any help,

Marco

Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente.

Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately.