Re: Writing XML directly to area detector's NDAttributesFile PV

["Wolfman, Mark via Tech-talk" <tech-talk at aps.anl.gov>]

Thanks, Matt. Yes that looks like what I want.

Thank you!

Mark



---
My work schedule may not be the same as your work schedule. Please
don't feel obligated to respond to this e-mail outside of your
normal work hours.

________________________________________
From: Pearson, Matthew <pearsonmr at ornl.gov>
Sent: Wednesday, September 25, 2024 16:25
To: Wolfman, Mark; Rivers, Mark L.; Érico Nogueira Rolim; tech-talk at aps.anl.gov
Subject: RE: Writing XML directly to area detector's NDAttributesFile PV

Hi Mark, It sounds like you might want to use the ROIStat plugin, which is overall easier to use than multiple ROI->Stats plugin chains: https: //urldefense. us/v3/__https: //areadetector. github. io/areaDetector/ADCore/NDPluginROIStat. html__;!!G_uCfscf7eWS!faUBDLAcU17P5m2kuvustssdGjeRPPUx2BJZFa1BGFJoOPww0RQUQgnyLWL9QhhGfNRER1gFd_yabH6PGQub$
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

Hi Mark,

It sounds like you might want to use the ROIStat plugin, which is overall easier to use than multiple ROI->Stats plugin chains:

https://urldefense.us/v3/__https://areadetector.github.io/areaDetector/ADCore/NDPluginROIStat.html__;!!G_uCfscf7eWS!faUBDLAcU17P5m2kuvustssdGjeRPPUx2BJZFa1BGFJoOPww0RQUQgnyLWL9QhhGfNRER1gFd_yabH6PGQub$

And it will output the full NDArray, and allow you to append the statistics parameters as NDAttributes.

ROIStat doesn't have all the features of the main Stats plugin, but it does have Min/Mean/Max/Total for each ROI that is defined (which is often all we need). The single plugin allows multiple ROIs, which are distinguished via Asyn address.

So the plugin chain would be (at least):
Driver->ROIStats->HDF5

Cheers,
Matt

-----Original Message-----
From: Wolfman, Mark <wolfman at anl.gov>
Sent: Wednesday, September 25, 2024 5:08 PM
To: Rivers, Mark L. <rivers at cars.uchicago.edu>; Pearson, Matthew <pearsonmr at ornl.gov>; Érico Nogueira Rolim <erico.rolim at lnls.br>; tech-talk at aps.anl.gov
Subject: [EXTERNAL] Re: Writing XML directly to area detector's NDAttributesFile PV

Hi, Matt and Mark.

Thanks for the info.

Sounds like using PARAM types is better than EPICS_PV types.

A few details I still need to sort out.

I need the full image to be what gets written to the HDF5 file, not the ROI that is being fed into the stats plugin. Also, we often have several ROIs feeding several stats plugins, depending on the experiment. Perhaps I could use the NDGather plugin to combine the 4 stats plugins outputs with their NDAttributes back into a single stream that then feeds into the HDF plugin? Not quite sure how the array addresses work, but I'll experiment a bit.

Thanks,
Mark

---
My work schedule may not be the same as your work schedule. Please don't feel obligated to respond to this e-mail outside of your normal work hours.

________________________________________
From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Tuesday, September 24, 2024 15:38
To: Pearson, Matthew; Érico Nogueira Rolim; Wolfman, Mark; tech-talk at aps.anl.gov
Subject: RE: Writing XML directly to area detector's NDAttributesFile PV

I agree with what Matt said. That does mean that the ROI stats plugin must be in the chain that feeds the HDF5 plugin. In your previous scheme that would not have been required, ROI stats could have been running in a separate chain. Mark -----Original ZjQcmQRYFpfptBannerStart This Message Is From an External Sender This message came from outside your organization.

ZjQcmQRYFpfptBannerEnd

I agree with what Matt said.  That does mean that the ROI stats plugin must be in the chain that feeds the HDF5 plugin.  In your previous scheme that would not have been required, ROI stats could have been running in a separate chain.

Mark


-----Original Message-----
From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Pearson, Matthew via Tech-talk
Sent: Tuesday, September 24, 2024 10:46 AM
To: Érico Nogueira Rolim <erico.rolim at lnls.br>; Wolfman, Mark Frederick <wolfman at anl.gov>; tech-talk at aps.anl.gov
Subject: RE: Writing XML directly to area detector's NDAttributesFile PV

Hi,

I would also suggest adding the ROI stats data via the statistics plugin and not the HDF plugin. Then the 'type' attribute would be PARAM and not EPICS_PV. If the data is added via the HDF plugin it would be possible to get the previous value and not the latest value, or it's possible the statistics plugin could have already processed a new NDArray and the ROI data then reflects that new data and not the one the HDF plugin is currently saving.

Cheers,
Matt

-----Original Message-----
From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Érico Nogueira Rolim via Tech-talk
Sent: Tuesday, September 24, 2024 11:33 AM
To: Wolfman, Mark Frederick <wolfman at anl.gov>; tech-talk at aps.anl.gov
Subject: [EXTERNAL] Re: Writing XML directly to area detector's NDAttributesFile PV

Hi Mark!

ADCore version is exactly your issue. This feature was added in ADCore R3-13, in commit 02eb9e04d879ae7a1bf609394c1e408f14191bbd.

Cheers,
Érico

On 24/09/2024 12:24, Wolfman, Mark via Tech-talk wrote:
> Hi, all.
>
> I'm configuring our area detectors to save ROI stats, etc into the HDF files alongside the image data. I'm looking for a way to dynamically set the NDAttributes that get included by the HDF file plugin.
>
> As a test, I'm using the XML snippet pasted below on our ADSimDet detector (ADCore version 3.12.1). If I put the XML into a file, and put the file path into the PV 25idSimDet:HDF1:NDAttributesFile, then I see the correct attribute in the HDF file.
>
> The documentation [1] makes it sound like I can put the XML directly into the PV and it should also work as long as "<Attributes>" is in the string, but I do not see the attribute in the HDF5 file, and I get an IOC console error "asynNDArrayDriver::readNDAttributesFile error opening file " followed by the XML I put in.
>
> I've also tried putting the XML into the camera's NDAttributesFile PV with similar results, and both with and without the "<?xml ?>" declaration. Any ideas on what I'm doing wrong?
>
> The use case here is that I want Bluesky to include some signals in the AD's HDF file, and also include the right resources in its saved scan entry to find the values later through our data API. Ophyd-async includes support to do this [2] but since my IOC does not let me put the XML into the NDAttributesFile PV, this doesn't work.
>
> Here is the XML:
>
>> <?xml version="1.0" encoding="UTF-8"?> <Attributes><Attribute
>> name="sim_det_async-sum" type="EPICS_PV"
>> source="25idSimDet:Stats1:Total_RBV" datatype="DOUBLE"
>> description="Sum of the array" /></Attributes>
> Thanks,
> Mark
>
> [1]
> https://urldefense.us/v3/__https://areadetector.github.io/areaDetector
>/ADCore/NDArray.html*asynndarraydriver__;Iw!!G_uCfscf7eWS!dh0BK5QPgcb-
>IqTxPFl9IiRRsnBYXRh9MQuybGRWbJh-lrGyjMmjBFSPWaY-MwisKfNGRzLMGIbdenqWWN
> X2xw$ [2]
> https://urldefense.us/v3/__https://github.com/bluesky/ophyd-async/blob
>/main/src/ophyd_async/plan_stubs/_nd_attributes.py__;!!G_uCfscf7eWS!dh
>0BK5QPgcb-IqTxPFl9IiRRsnBYXRh9MQuybGRWbJh-lrGyjMmjBFSPWaY-MwisKfNGRzLM
> GIbdenqlXczNSA$
>
> ---
> My work schedule may not be the same as your work schedule. Please
> don't feel obligated to respond to this e-mail outside of your normal
> work hours.



Aviso Legal: Esta mensagem e seus anexos podem conter informações confidenciais e/ou de uso restrito. Observe atentamente seu conteúdo e considere eventual consulta ao remetente antes de copiá-la, divulgá-la ou distribuí-la. Se você recebeu esta mensagem por engano, por favor avise o remetente e apague-a imediatamente.

Disclaimer: This email and its attachments may contain confidential and/or privileged information. Observe its content carefully and consider possible querying to the sender before copying, disclosing or distributing it. If you have received this email by mistake, please notify the sender and delete it immediately.