RE: ADSimDetector and ADURL - runtime issues with FFT and HDF5

[Mark Rivers via Tech-talk <tech-talk@aps.anl.gov>]

I fixed the issues with the st*.cmd* files in iocHDF5Test and pushed to Github.

 

Ø  I am able to create a HDF5 file (for either ADSimDetector or ADURL) as requested by the test5.sh script (I cloned the logic of this for the ADURL module).

Ø  These files are large (>~100MB), and clearly contain the simulated and URL data, respectively.

Ø  I’m able to open the file(s) with the hdfview application (after closing the IOC, to avoid any file write/read share issues).

Ø  But, I’m unable to view any of the data because there are no attribute nodes under the ‘test5_001.h5’ root, in the viewer.

Ø  Viewing the ‘properties’ on the ‘test5_001.h5’ node in hdfview, shows no attributes and no datasets.

 

 

I cannot reproduce that problem.  I just started the simDetector with the new st.cmd in the iocHDF5Test directory.  I then ran test5.sh script.  It produced these 2 files:

-rw-rw-r-- 1 epics domain users  69M Nov 19 11:03 test5_001.h5

-rw-rw-r-- 1 epics domain users 212K Nov 19 11:03 test5_001.txt

 

I then opened test5_001.h5 in HDF5.  I was able to display the data in hdfview with no problems as shown in the screen shot below.

 

 

Mark

 

 

From: Heesterman, Peter J <Peter.Heesterman@ukaea.uk>
Sent: Monday, November 19, 2018 9:31 AM
To: Engbretson, Mark S. <engbretson@anl.gov>; Mark Rivers <rivers@cars.uchicago.edu>
Subject: RE: ADSimDetector and ADURL - runtime issues with FFT and HDF5

 

Hi Mark(s),

 

I am using simDetector 2-8, which I believe is the most recent.

My comment about the st.cmd script referred to the iocHDF5Test sub-folder (which I was seeking to exercise), and not the main iocSimDetector folder (where the script is correct).

- I’m sorry that was unclear.

 

Are you talking about EPICS_CA_MAX_ARRAY_BYTES?

It is set as an environment variable which is applied to shells through /etc/profile.d/custom.sh.

 

NB, data visualisation with ImageJ works for me with the ADURL module with the values given, but fails for the SimDetector module (with or without attempting to use HDF5).

 

Cheers,

 

Peter.

 

 

From: Engbretson, Mark S. <engbretson@anl.gov>
Sent: 19 November 2018 15:04
To: Heesterman, Peter J <Peter.Heesterman@ukaea.uk>
Subject: RE: ADSimDetector and ADURL - runtime issues with FFT and HDF5

 

But did it make any difference? You might have to set this both in the ADURL and SIMDetector soft IOC st,cmd *AND* in for environment for IMAGEJ to work.

 

From: Heesterman, Peter J <Peter.Heesterman@ukaea.uk>
Sent: Monday, November 19, 2018 8:59 AM
To: Engbretson, Mark S. <engbretson@anl.gov>
Subject: RE: ADSimDetector and ADURL - runtime issues with FFT and HDF5

 

Hi Mark,

 

I have the latest sources.

My comment about the paths refers to the iocHDF5Test sub-folder that I was working with, not the iocSimDetector folder (where it’s correct).

I’m sorry this wasn’t clear.

 

I has set EPICS_CA_MAX_ARRAY_BYTES to 30000000 which is overkill I think, so I reduced it to 10000000.

 

Cheers,

 

Peter.

 

 

From: Engbretson, Mark S. <engbretson@anl.gov>
Sent: 19 November 2018 14:44
To: Heesterman, Peter J <Peter.Heesterman@ukaea.uk>
Subject: RE: ADSimDetector and ADURL - runtime issues with FFT and HDF5

 

I don’t think that you have the latest sources for ADSimDetector since the paths are correct on the sources that I have.

 

If ImageJ   doesn’t work, chances are the value for

 

EPICS_CA_MAX_ARRAY_BYTES

 

Is either not set or set to be a value too small for the actual image data.

 

 

From: tech-talk-bounces@aps.anl.gov <tech-talk-bounces@aps.anl.gov> On Behalf Of Heesterman, Peter J via Tech-talk
Sent: Monday, November 19, 2018 8:24 AM
To: EPICS Tech-Talk (tech-talk@aps.anl.gov) <tech-talk@aps.anl.gov>
Subject: ADSimDetector and ADURL - runtime issues with FFT and HDF5

 

Hi,

 

I’m seeking to use these modules with HDF5 data archiving.

Accordingly, I’ve been looking at the iocHDF5Test within ADSimDetector, and focused on test5.sh.

 

I’m working with an Ubuntu 18.04 VM.

I have:

    areaDetector and ADCore R3-3-2

    ADSupport R1-5

  ADViewers R1-2

    ADURL R2-2

    Asyn4-33

    Base 3.15.6

 

There are some issues that I need to raise:

 

a. st.cmd in ADSimDetector hasn’t been updated to reflect module paths – in particular lines like

        dbLoadRecords("$(ADCORE)/db/simDetector.template","P=$(PREFIX),R=cam1:,PORT=$(PORT),ADDR=0,TIMEOUT=1"), where the file is no longer in ADCore.

    This is easily corrected. Actually, I believe st_base.cmd from iocSimDetector can just be used.

 

b. When I run the sim detector and try to visualise with ImageJ, I get green on both 13SIM1:image1 and 13SIM1:image2.

    But, I don’t get any image displayed for either.

 

c. I note an error:

    NDPluginDriver::connectToArrayPort Error calling pasynManager->connectDevice to array port FFT1 address 0, status=3, error=asynManager:connectDevice port FFT1 not found.

 

    Could my (b) have anything to do with this error?

    I understand that image2 would be the FFT of the simulated data generated for image1?

 

d. I am able to create a HDF5 file (for either ADSimDetector or ADURL) as requested by the test5.sh script (I cloned the logic of this for the ADURL module).

    These files are large (>~100MB), and clearly contain the simulated and URL data, respectively.

    I’m able to open the file(s) with the hdfview application (after closing the IOC, to avoid any file write/read share issues).

    But, I’m unable to view any of the data because there are no attribute nodes under the ‘test5_001.h5’ root, in the viewer.

 

    Viewing the ‘properties’ on the ‘test5_001.h5’ node in hdfview, shows no attributes and no datasets.

 

    I refer to http://cars9.uchicago.edu/software/epics/NDFileHDF5.html , which I’ve studied in some detail.

 

I am able to visualise data from the ADURL module, quite normally.

 

What is it that I’m missing?

 

Many thanks,

 

Peter.