Hi
Érico,
- and/or if it's related to the additional allocations that happen in the hot-path since you're allocating additional NDAttributes for
each frame, and there isn't a free-list for them, unlike NDArrays.
That is not really true. The NDAttributeList of an NDArray is not cleared by default when the NDArray is retrieved from the NDArrayPool. This means that once the driver has been running
for a while the NDArrays in the NDArrayPool will already have the NDAttributeList attached. Drivers and plugins then simply update the values of the NDAttributes, there is no new allocation happening. This default behavior can be overridden by setting the
EPICS variable eraseNDAttributes to 1. Then the NDAttributeList will be cleared each time a new NDArray is allocated.
https://areadetector.github.io/areaDetector/ADCore/NDArray.html#ndattribute
Mark
From: Érico Nogueira Rolim <erico.rolim at lnls.br>
Sent: Thursday, September 26, 2024 9:47 AM
To: Matthew Newville <newville at cars.uchicago.edu>; Wolfman, Mark <wolfman at anl.gov>; Pearson, Matthew <pearsonmr at ornl.gov>; Mark Rivers <rivers at cars.uchicago.edu>; tech-talk at aps.anl.gov
Subject: Re: Writing XML directly to area detector's NDAttributesFile PV
Have you profiled this performance impact? It would be interesting to know if the issue is caused by the need to write into multiple HDF5 datasets, and/or if it's related to the additional allocations that happen in the hot-path since you're
allocating additional NDAttributes for each frame, and there isn't a free-list for them, unlike NDArrays. I.e. when you say "pare this down to 4 or 5", is it simply in the XML definition, or do you also configure the driver to not add the NDAttributes?
On 25/09/2024 20:38, Matthew Newville wrote:
Hi Mark,
While using ROIStats as PARAMS in an NDAttributes XML File might seem like a “clean” solution, I would suggest checking carefully whether this causes any performance problem in file saving
for your use case.
As an example, with the Xspress3 driver, images are N detectors (N between 1 and 32, say) x 4096 (energy bins) and are saved very quickly to the HDF5 plugin. In addition, each of the N detectors
has somewhere between 8 and 12 (depending on generation of the Epics driver) “SCA” scalar arrays that are saved using the NDAttributes plugin. With a typical 8-element detector, that can easily mean 80 PARAMS in the XML file being saved. This works but
also impacts the file-saving rate.
Several of these scaler parameters are optional-ish, and I often now pare this down to 4 or 5 (per detector element) PARAMS to be saved as NDAttributes. We’ve talked about changing the driver
to pack these values into the primary array, either extending it or reserving the final 16 energy bins for these statistics. We have not done this yet but have not ruled it out either. And we do not ever write values from the ROIStats plugin (which supports
up to 48 per detector element for classic XRF ROIs) to NDAttributes. We do need those, but those can readily be extracted after the fact.
That is just to say that if you run some benchmarks, you might conclude that extracting those in post-processing is worth the effort. HDF5 is really good for extracting ROIs from NDArray data
and leaving that task as “file consumption” (where it cannot impact data collection performance) instead of “file writing” (where it can) will leave the EPICS driver to just get the main data to disk.
Thanks, Matt. Yes that looks like what I want.
Thank you!
Mark
---
My work schedule may not be the same as your work schedule. Please
don't feel obligated to respond to this e-mail outside of your
normal work hours.
________________________________________
From: Pearson, Matthew <pearsonmr at ornl.gov>
Sent: Wednesday, September 25, 2024 16:25
To: Wolfman, Mark; Rivers, Mark L.; Érico Nogueira Rolim;
tech-talk at aps.anl.gov
Subject: RE: Writing XML directly to area detector's NDAttributesFile PV
Hi Mark, It sounds like you might want to use the ROIStat plugin, which is overall easier to use than multiple ROI->Stats plugin chains:
https: //urldefense. us/v3/__https: //areadetector. github. io/areaDetector/ADCore/NDPluginROIStat. html__;!!G_uCfscf7eWS!faUBDLAcU17P5m2kuvustssdGjeRPPUx2BJZFa1BGFJoOPww0RQUQgnyLWL9QhhGfNRER1gFd_yabH6PGQub$
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd
Hi Mark,
It sounds like you might want to use the ROIStat plugin, which is overall easier to use than multiple ROI->Stats plugin chains:
https://urldefense.us/v3/__https://areadetector.github.io/areaDetector/ADCore/NDPluginROIStat.html__;!!G_uCfscf7eWS!faUBDLAcU17P5m2kuvustssdGjeRPPUx2BJZFa1BGFJoOPww0RQUQgnyLWL9QhhGfNRER1gFd_yabH6PGQub$
And it will output the full NDArray, and allow you to append the statistics parameters as NDAttributes.
ROIStat doesn't have all the features of the main Stats plugin, but it does have Min/Mean/Max/Total for each ROI that is defined (which is often all we need). The single plugin allows multiple ROIs, which are distinguished via Asyn address.
So the plugin chain would be (at least):
Driver->ROIStats->HDF5
Cheers,
Matt
-----Original Message-----
From: Wolfman, Mark <wolfman at anl.gov>
Sent: Wednesday, September 25, 2024 5:08 PM
To: Rivers, Mark L. <rivers at cars.uchicago.edu>; Pearson, Matthew
<pearsonmr at ornl.gov>; Érico Nogueira Rolim
<erico.rolim at lnls.br>;
tech-talk at aps.anl.gov
Subject: [EXTERNAL] Re: Writing XML directly to area detector's NDAttributesFile PV
Hi, Matt and Mark.
Thanks for the info.
Sounds like using PARAM types is better than EPICS_PV types.
A few details I still need to sort out.
I need the full image to be what gets written to the HDF5 file, not the ROI that is being fed into the stats plugin. Also, we often have several ROIs feeding several stats plugins, depending on the experiment. Perhaps I could use the NDGather plugin to combine
the 4 stats plugins outputs with their NDAttributes back into a single stream that then feeds into the HDF plugin? Not quite sure how the array addresses work, but I'll experiment a bit.
Thanks,
Mark
---
My work schedule may not be the same as your work schedule. Please don't feel obligated to respond to this e-mail outside of your normal work hours.
________________________________________
From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Tuesday, September 24, 2024 15:38
To: Pearson, Matthew; Érico Nogueira Rolim; Wolfman, Mark;
tech-talk at aps.anl.gov
Subject: RE: Writing XML directly to area detector's NDAttributesFile PV
I agree with what Matt said. That does mean that the ROI stats plugin must be in the chain that feeds the HDF5 plugin. In your previous scheme that would not have been required, ROI stats could have been running in a separate chain. Mark -----Original ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender This message came from outside your organization.
ZjQcmQRYFpfptBannerEnd
I agree with what Matt said. That does mean that the ROI stats plugin must be in the chain that feeds the HDF5 plugin. In your previous scheme that would not have been required, ROI stats could have been running in a separate chain.
Mark
-----Original Message-----
From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Pearson, Matthew via Tech-talk
Sent: Tuesday, September 24, 2024 10:46 AM
To: Érico Nogueira Rolim <erico.rolim at lnls.br>; Wolfman, Mark Frederick
<wolfman at anl.gov>;
tech-talk at aps.anl.gov
Subject: RE: Writing XML directly to area detector's NDAttributesFile PV
Hi,
I would also suggest adding the ROI stats data via the statistics plugin and not the HDF plugin. Then the 'type' attribute would be PARAM and not EPICS_PV. If the data is added via the HDF plugin it would be possible to get the previous value and not the latest
value, or it's possible the statistics plugin could have already processed a new NDArray and the ROI data then reflects that new data and not the one the HDF plugin is currently saving.
Cheers,
Matt
-----Original Message-----
From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Érico Nogueira Rolim via Tech-talk
Sent: Tuesday, September 24, 2024 11:33 AM
To: Wolfman, Mark Frederick <wolfman at anl.gov>;
tech-talk at aps.anl.gov
Subject: [EXTERNAL] Re: Writing XML directly to area detector's NDAttributesFile PV
Hi Mark!
ADCore version is exactly your issue. This feature was added in ADCore R3-13, in commit 02eb9e04d879ae7a1bf609394c1e408f14191bbd.
Cheers,
Érico
On 24/09/2024 12:24, Wolfman, Mark via Tech-talk wrote:
> Hi, all.
>
> I'm configuring our area detectors to save ROI stats, etc into the HDF files alongside the image data. I'm looking for a way to dynamically set the NDAttributes that get included by the HDF file plugin.
>
> As a test, I'm using the XML snippet pasted below on our ADSimDet detector (ADCore version 3.12.1). If I put the XML into a file, and put the file path into the PV 25idSimDet:HDF1:NDAttributesFile, then I see the correct attribute in the HDF file.
>
> The documentation [1] makes it sound like I can put the XML directly into the PV and it should also work as long as "<Attributes>" is in the string, but I do not see the attribute in the HDF5 file, and I get an IOC console error "asynNDArrayDriver::readNDAttributesFile
error opening file " followed by the XML I put in.
>
> I've also tried putting the XML into the camera's NDAttributesFile PV with similar results, and both with and without the "<?xml ?>" declaration. Any ideas on what I'm doing wrong?
>
> The use case here is that I want Bluesky to include some signals in the AD's HDF file, and also include the right resources in its saved scan entry to find the values later through our data API. Ophyd-async includes support to do this [2] but since my IOC
does not let me put the XML into the NDAttributesFile PV, this doesn't work.
>
> Here is the XML:
>
>> <?xml version="1.0" encoding="UTF-8"?> <Attributes><Attribute
>> name="sim_det_async-sum" type="EPICS_PV"
>> source="25idSimDet:Stats1:Total_RBV" datatype="DOUBLE"
>> description="Sum of the array" /></Attributes>
> Thanks,
> Mark
>
> [1]
>
https://urldefense.us/v3/__https://areadetector.github.io/areaDetector
>/ADCore/NDArray.html*asynndarraydriver__;Iw!!G_uCfscf7eWS!dh0BK5QPgcb-
>IqTxPFl9IiRRsnBYXRh9MQuybGRWbJh-lrGyjMmjBFSPWaY-MwisKfNGRzLMGIbdenqWWN
> X2xw$ [2]
>
https://urldefense.us/v3/__https://github.com/bluesky/ophyd-async/blob
>/main/src/ophyd_async/plan_stubs/_nd_attributes.py__;!!G_uCfscf7eWS!dh
>0BK5QPgcb-IqTxPFl9IiRRsnBYXRh9MQuybGRWbJh-lrGyjMmjBFSPWaY-MwisKfNGRzLM
> GIbdenqlXczNSA$
>
> ---
> My work schedule may not be the same as your work schedule. Please
> don't feel obligated to respond to this e-mail outside of your normal
> work hours.
Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por
engano, por favor avise o remetente e apague-a imediatamente.
Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake,
please notify the sender and delete it immediately.
|
