RE: XSPRESS3 hdf5 problems

[Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>]

That looks like it might be a question for Quantum.  It seems like you have told it to use 6 channels, but it is only using 4.

 

Mark

 

 

From: John Dobbins <john.dobbins at cornell.edu>
Sent: Monday, January 27, 2020 2:22 PM
To: Mark Rivers <rivers at cars.uchicago.edu>; tech-talk at aps.anl.gov; John Dobbins <john.dobbins at cornell.edu>; Matt Newville <newville at cars.uchicago.edu>
Subject: Re: XSPRESS3 hdf5 problems

 

My IOC is built but I have trouble with xspress3config, note that my Xspress3 chassis has more than 4 channels.

 

I changed the Y_SIZE (max_channels)  in xspress3/iocs/xspress3IOC/iocBoot/iocxspress3-4Channel/st.cmd to 6

 

xspress3Config("XSP3", "6", "1", "192.168.0.1", "16384", 16384, "4096", 0, 0, 1, 0)

 

but I get messages from xsp3_config that I don't understand

 

xspress3_config: Card 0

xspress3_config: host10GigeIPaddress 192.168.0.65

xspress3_config: femIPaddress 192.168.0.2

xspress3_config: fem10GigeIPaddress 192.168.0.66

xspress3_config: femMACaddress 2.0.0.0.0.0

xspress3_config: 1GigePort 30123

xspress3_config: 10GigePort 30124

xspress3_config: nchan 4

xspress3_config: scope module name xsp3_scope0

xsp3_config: Configuring node on path 0 name=192.168.0.2

 

then later

 

xsp3_config: Configuration found 6 chans_per_card on path 0

xsp3_set_num_chan: Max number of channels on path 0 is set to 6

xsp3_set_num_chan: Number of channels on path 0 is set as 4

Calculating generation from Revision 0x01003006, major=3, minor=6 => determined generation=0

Configuring software scalers for 16384 time frames

2020/01/27 14:12:31.130 xsp3_config returned XSP3_ERROR.

2020/01/27 14:12:31.130 API Error Message: xsp3_config: xsp3_mkmod3d: module 'xsp3_scalers0' exists but is wrong size (11,6, 16384) not (11, 4, 16384)

.

2020/01/27 14:12:31.130 EPICS Driver Calling Function: Xspress3::connect.

2020/01/27 14:12:31.130 XSPRESS3:det1:CONNECT devAsynInt32::processCallbackOutput process error

 

 

This error appears to come from the Quantum supplied library (?)

 

I don't know what it is trying to tell me

 

John

 

 

 

---

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of John Dobbins via Tech-talk <tech-talk at aps.anl.gov>
Sent: Monday, January 27, 2020 12:27 PM
To: Mark Rivers <rivers at cars.uchicago.edu>; tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: Re: XSPRESS3 hdf5 problems

 

Mark,

Thanks,  adding STATIC_BUILD=YES to xspress3/iocs/xspress3IOC/xspress3App/src/Makefile works.

John

 

---

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Monday, January 27, 2020 12:18 PM
To: John Dobbins <john.dobbins at cornell.edu>; tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Cc: Matt Newville <newville at cars.uchicago.edu>
Subject: RE: XSPRESS3 hdf5 problems

 

Hi John,

 

This is what I get when I build with gcc 4.8.5 on a Centos 7 system with base 7.0.3.1.  It works when I build statically (STATIC_BUILD=YES).

 

/usr/bin/g++ -o xspress3App -Wl,-Bstatic -L/home/epics/devel/xspress3-2-2/lib/linux-x86_64 -L/home/epics/devel/xspress3-2-2/iocs/xspress3IOC/lib/linux-x86_64 -L/home/epics/devel/areaDetector-3-8/ADCore/lib/linux-x86_64 -L/home/epics/devel/areaDetector-3-8/ADSupport/lib/linux-x86_64 -L/home/epics/devel/asyn-4-37/lib/linux-x86_64 -L/home/epics/devel/autosave-5-10/lib/linux-x86_64 -L/home/epics/devel/busy-1-7-2/lib/linux-x86_64 -L/home/epics/devel/calc-3-7-3/lib/linux-x86_64 -L/home/epics/devel/devIocStats-3-1-16/lib/linux-x86_64 -L/home/epics/devel/seq-2-2-5/lib/linux-x86_64 -L/home/epics/devel/sscan-2-11-3/lib/linux-x86_64 -L/usr/local/epics-devel/base-7.0.3.1/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/xspress3-2-2/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/xspress3-2-2/iocs/xspress3IOC/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/areaDetector-3-8/ADCore/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/areaDetector-3-8/ADSupport/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/asyn-4-37/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/autosave-5-10/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/busy-1-7-2/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/calc-3-7-3/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/devIocStats-3-1-16/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/seq-2-2-5/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/sscan-2-11-3/lib/linux-x86_64 -Wl,-rpath,/usr/local/epics-devel/base-7.0.3.1/lib/linux-x86_64           -rdynamic -m64         xspress3App_registerRecordDeviceDriver.o xspress3AppMain.o    -lxspress3Epics -lxspress3 -limg_mod -lNDPlugin -lADBase -lqsrv -lntndArrayConverter -lnt -lpvDatabase -lpvAccessIOC -lpvAccessCA -lpvAccess -lpvData -lnetCDF -lMagick++ -lcoders -lMagick -ljbig -ljp2 -lbzlib -lpng -lwebp -llcms -lttf -lwmf -lfilters -ltiff -lxml2 -lNeXus -lhdf5 -lhdf5_hl -lbitshuffle -lblosc -lszip -lzlib -ljpeg -lasyn -lautosave -lbusy -lcalc -ldevIocStats -lsscan -lseq -lpv -ldbRecStd -ldbCore -lca -lCom -Wl,-Bdynamic -lboost_system -lX11 -lXext -lpthread   -lreadline -lm -lrt -ldl –lgcc

 

If I edit xspress3/iocs/xspress3IOC/xspress3App/src/Makefile to have STATIC_BUILD=NO it fails the same as you:

 

/usr/bin/g++ -o xspress3App  -L/home/epics/devel/xspress3-2-2/lib/linux-x86_64 -L/home/epics/devel/xspress3-2-2/iocs/xspress3IOC/lib/linux-x86_64 -L/home/epics/devel/areaDetector-3-8/ADCore/lib/linux-x86_64 -L/home/epics/devel/areaDetector-3-8/ADSupport/lib/linux-x86_64 -L/home/epics/devel/asyn-4-37/lib/linux-x86_64 -L/home/epics/devel/autosave-5-10/lib/linux-x86_64 -L/home/epics/devel/busy-1-7-2/lib/linux-x86_64 -L/home/epics/devel/calc-3-7-3/lib/linux-x86_64 -L/home/epics/devel/devIocStats-3-1-16/lib/linux-x86_64 -L/home/epics/devel/seq-2-2-5/lib/linux-x86_64 -L/home/epics/devel/sscan-2-11-3/lib/linux-x86_64 -L/usr/local/epics-devel/base-7.0.3.1/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/xspress3-2-2/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/xspress3-2-2/iocs/xspress3IOC/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/areaDetector-3-8/ADCore/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/areaDetector-3-8/ADSupport/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/asyn-4-37/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/autosave-5-10/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/busy-1-7-2/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/calc-3-7-3/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/devIocStats-3-1-16/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/seq-2-2-5/lib/linux-x86_64 -Wl,-rpath,/home/epics/devel/sscan-2-11-3/lib/linux-x86_64 -Wl,-rpath,/usr/local/epics-devel/base-7.0.3.1/lib/linux-x86_64           -rdynamic -m64         xspress3App_registerRecordDeviceDriver.o xspress3AppMain.o    -lxspress3Epics -lxspress3 -limg_mod -lNDPlugin -lADBase -lqsrv -lntndArrayConverter -lnt -lpvDatabase -lpvAccessIOC -lpvAccessCA -lpvAccess -lpvData -lnetCDF -lMagick++ -lcoders -lMagick -ljbig -ljp2 -lbzlib -lpng -lwebp -llcms -lttf -lwmf -lfilters -ltiff -lxml2 -lNeXus -lhdf5 -lhdf5_hl -lbitshuffle -lblosc -lszip -lzlib -ljpeg -lasyn -lautosave -lbusy -lcalc -ldevIocStats -lsscan -lseq -lpv -ldbRecStd -ldbCore -lca -lCom  -lboost_system -lX11 -lXext

/bin/ld: /home/epics/devel/xspress3-2-2/lib/linux-x86_64/libxspress3.a(xspress3_histogram.o): undefined reference to symbol 'pthread_cancel@@GLIBC_2.2.5'

//lib64/libpthread.so.0: error adding symbols: DSO missing from command line

collect2: error: ld returned 1 exit status

make[1]: *** [xspress3App] Error 1

make[1]: Leaving directory `/home/epics/devel/xspress3-2-2/iocs/xspress3IOC/xspress3App/src/O.linux-x86_64'

make: *** [install.linux-x86_64] Error 2

 

We have not noticed the problem because we normally build statically.  I will investigate.

 

Meanwhile, you can probably work around the problem by adding STATIC_BUILD=YES to xspress3/iocs/xspress3IOC/xspress3App/src/Makefile.

 

Mark

 

 

From: John Dobbins <john.dobbins at cornell.edu>
Sent: Monday, January 27, 2020 10:45 AM
To: tech-talk at aps.anl.gov
Cc: Mark Rivers <rivers at cars.uchicago.edu>; Matt Newville <newville at cars.uchicago.edu>
Subject: Re: XSPRESS3 hdf5 problems

 

Mark, Matt,

 

I am attempting to build epics-modules/xspress3 (latest master).

 

On CentOS Linux release 7.5.1804, I have built base 3.15.7, areaDetector R3-7. 

 

If BUILD_IOCS = NO then Xspress3 build succeeds but with BUILD_IOCS=YES I get

 

/usr/bin/g++ -o xspress3App  -L/home/xspress3/epics/synApps_6_1/support/xspress/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/xspress/iocs/xspress3IOC/lib/linux-x86_64 -L/home/xspress3/epics/base-3.15.7/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/areaDetector-R3-7/ADCore/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/areaDetector-R3-7/ADSupport/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/asyn-R4-36/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/autosave-R5-10/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/busy-R1-7-2/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/calc-R3-7-3/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/iocStats-3-1-16/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/seq-2-2-6/lib/linux-x86_64 -L/home/xspress3/epics/synApps_6_1/support/sscan-R2-11-3/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/xspress/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/xspress/iocs/xspress3IOC/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/base-3.15.7/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/areaDetector-R3-7/ADCore/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/areaDetector-R3-7/ADSupport/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/asyn-R4-36/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/autosave-R5-10/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/busy-R1-7-2/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/calc-R3-7-3/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/iocStats-3-1-16/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/seq-2-2-6/lib/linux-x86_64 -Wl,-rpath,/home/xspress3/epics/synApps_6_1/support/sscan-R2-11-3/lib/linux-x86_64           -rdynamic -m64            xspress3App_registerRecordDeviceDriver.o xspress3AppMain.o    -lxspress3Epics -lxspress3 -limg_mod -lNDPlugin -lADBase -lnetCDF -lMagick++ -lcoders -lMagick -ljbig -ljp2 -lbzlib -lpng -lwebp -llcms -lttf -lwmf -lfilters -ltiff -lxml2 -lNeXus -lhdf5 -lhdf5_hl -lbitshuffle -lblosc -lszip -lzlib -ljpeg -lasyn -lautosave -lbusy -lcalc -ldevIocStats -lsscan -lseq -lpv -ldbRecStd -ldbCore -lca -lCom  -lX11 -lXext 

/usr/bin/ld: /home/xspress3/epics/synApps_6_1/support/xspress/lib/linux-x86_64/libxspress3.a(xspress3_histogram.o): undefined reference to symbol 'pthread_cancel@@GLIBC_2.2.5'

//usr/lib64/libpthread.so.0: error adding symbols: DSO missing from command line

collect2: error: ld returned 1 exit status

 

 

On my machine

 

locate libpthread

/usr/lib64/libpthread-2.17.so

/usr/lib64/libpthread.so

/usr/lib64/libpthread.so.0

/usr/lib64/libpthread_nonshared.a

 

 

readelf -s /lib64/libpthread.so.0 | grep pthread_cancel

   200: 000000000000cae0   167 FUNC    GLOBAL DEFAULT   13 pthread_cancel@@GLIBC_2.2.5

   233: 0000000000000000     0 FILE    LOCAL  DEFAULT  ABS pthread_cancel.c

   485: 00000000000111b4    12 OBJECT  LOCAL  DEFAULT   16 _thread_db_pthread_cancel

   508: 000000000000fe30   260 FUNC    LOCAL  DEFAULT   13 pthread_cancel_init

   949: 000000000000cae0   167 FUNC    GLOBAL DEFAULT   13 pthread_cancel

 

gcc 4.8.5

 

Any ideas?

 

John

 

 

---

From: Matt Newville <newville at cars.uchicago.edu>
Sent: Thursday, January 23, 2020 12:04 PM
To: John Dobbins <john.dobbins at cornell.edu>
Cc: Mark Rivers <rivers at cars.uchicago.edu>; tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: Re: XSPRESS3 hdf5 problems

 

Hi John,

 

Just to follow up: I don't recall ever seeing that error from HDF5 with epics-modules/xspress3.  I believe there were some known problems with HDF5 that were fixed around AD 2, but I don't recall the details. 

 

 

On Thu, Jan 23, 2020 at 10:55 AM John Dobbins <john.dobbins at cornell.edu> wrote:

Mark,

 

Thanks. I'll give that a try.

 

John

 

---

From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Thursday, January 23, 2020 11:27 AM
To: John Dobbins <john.dobbins at cornell.edu>
Cc: tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>; Matt Newville <newville at cars.uchicago.edu>
Subject: RE: XSPRESS3 hdf5 problems

 

Hi John,

 

Ø  I have contacted Quantum.  I thought that while waiting to hear from them it would make sense to build an IOC from source. On github I see

Ø  epics-modules/xspress3

 

The IOC that Quantum distributes is ancient, using areaDetector R1-9-1.

 

That is the one I recommend you use.  Matt Newville in our group here at the APS is maintaining it.  It is working with the latest version of ADCore, ADSupport, etc.

 

It uses NDPluginROIStat for ROI statistics.

 

There are new OPI screens.

 

Mark

 

 

 

From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of John Dobbins via Tech-talk
Sent: Thursday, January 23, 2020 8:15 AM
To: tech-talk at aps.anl.gov
Subject: XSPRESS3 hdf5 problems

 

All,

 

I have a new 5 channel Quantum Xspress3 data readout unit which when taking multiple frames will start generating hdf5 related error messages at what seems like random (will acquire between a few or thousands of frames before generating errors).  Once these messages start they seem to continue until the IOC segfaults. 

 

repeating pair of error messages on the IOC console:

 

 HDF5-DIAG: Error detected in HDF5 (1.8.12) thread 0:

  #000: ../../src/H5Dio.c line 231 in H5Dwrite(): can't prepare for writing data

    major: Dataset

    minor: Write failed

  #001: ../../src/H5Dio.c line 348 in H5D__pre_write(): file selection+offset not within extent

    major: Dataspace

    minor: Out of range

 

This IOC is a binary that comes with the detector. 

 

I have contacted Quantum.  I thought that while waiting to hear from them it would make sense to build an IOC from source. On github I see

 

epics-modules/xspress3

 

and

 

quantumdetectors/xspress3-epics  forked from the above.

 

Any advice on which i should go with? Any suspicions regarding the hdf5 errors?

 

Thanks,

 

John Dobbins

 

Research Support Specialist

Cornell High Energy Synchrotron Source

Cornell University

 

www.chess.cornell.edu

 

--

--Matt Newville