EPICS Home EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

Format page
for printing


Search Tech-talk
1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <2020>  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <2020>  2021  2022  2023  2024 
<== Date ==> <== Thread ==>
Subject: Re: ADProsilica crashing on RHEL7 3.10.0-1127
From: "Wlodek, Jakub via Tech-talk" <tech-talk at aps.anl.gov>
To: "'Dunning, Michael'" <mdunning at slac.stanford.edu>, "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov>, Mark Rivers <rivers at cars.uchicago.edu>
Cc: "Condamoor, Shantha" <scondam at slac.stanford.edu>
Date: Fri, 25 Sep 2020 19:34:47 +0000
Hi Michael,

Whenever I need to copy IOC executables that rely on vendor shared libraries, I put the .so libraries with the distribution,
and include a script that sets the LD_LIBRARY_PATH environment variable to include the location of those libraries
as a part of the IOC startup sequence. This way the user can start the IOC without having to install these
shared libraries to a system location that might require root access.

Best,
Jakub
From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Mark Rivers via Tech-talk <tech-talk at aps.anl.gov>
Sent: Friday, September 25, 2020 3:29 PM
To: 'Dunning, Michael' <mdunning at slac.stanford.edu>; tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Cc: Condamoor, Shantha <scondam at slac.stanford.edu>
Subject: RE: ADProsilica crashing on RHEL7 3.10.0-1127
 

Hi Mike,

 

Ø  So it's not a static build, but I guess this is expected.  I'm guessing that until the vendor releases source code or archive libraries, that there's just no way around this?

Ø  I suppose one option is to drop the .so libraries in a directory whose path exists on both the development and production servers, e.g. /usr/local/lib/ or somewhere like that?

 

I think it as close to a static build as you can get, given that the vendor distributes .so files.  In my experience vendors in the past used to distribute both .a and .so files, but recently only .so files.  The same problem exists on Windows, where the vendor almost always builds their library dynamically and distributes .lib and .dll files.

 

My practice in areaDetector has been the following:

-          For Linux areaDetector includes the .so files.  This is required so that anyone can build the package without having to install anything from the vendor on their machine.  It also means, for example that all you need to copy to another machine is ADVimba/bin/rhel7-x86_64/ and ADVimba/lib/rhel7-x86_64/, ADVimba/dbd, and ADVimba/db (plus OPI files).

 

-          For Windows areaDetector includes the .lib files, but not the .dll files.  The .lib files are all that is required to build the application.  The .dll files are needed to run the application. I do this because the vendor normally also installs drivers as part of their package, and so a local installation is usually necessary anyway, and that will include the .dlls. I have also found vendors happy to allow me to distribute .h and .lib files, and less happy for me to distribute .dll files.  The .lib file defines the API, and so typically needs to be updated less often than the .dll which defines the implementation and can be changed to fix bugs without changing the .lib file.

 

Ø  Is there a better way of dealing with this in the EPICS build system, or some other clever trick?

Ø  I guess this could be viewed as a more general question of how to do a static IOC build when you only have shared object libraries.

 

I think you need to accept that you will need to copy the .so files as well as the executables.  After all, you must also already be copying the dbd/ files and db/ files, not just the executable.

 

Mark

 

 

 

From: Dunning, Michael <mdunning at slac.stanford.edu>
Sent: Friday, September 25, 2020 2:09 PM
To: Mark Rivers <rivers at cars.uchicago.edu>; tech-talk at aps.anl.gov
Cc: Pearson, Matthew R. <pearsonmr at ornl.gov>; Dunning, Michael <mdunning at slac.stanford.edu>; Condamoor, Shantha <scondam at slac.stanford.edu>
Subject: Re: ADProsilica crashing on RHEL7 3.10.0-1127

 

I've been testing ADVimba on both rhel6 and rhel7 and it seems to be working well.  Thanks a lot for your work on this module Mark.

I'm trying to use this module with a static IOC build (i.e. setting SHARED_LIBRARIES=NO and STATIC_BUILD=YES).

We build all IOCs in this way and then copy them over to and run them on a production server with the same architecture, which doesn't have access to our development fileserver.


As you know, for linux the vendor support has two parts:
vimbaSupportC  --> No source code, no archive (.a) libraries, only shared object (.so) libraries
vimbaSupportCPP  --> Built by ADVimba from vendor source code

 

And the libs section of the IOC Makefile looks like this:

...

# Add locally compiled object code

PROD_LIBS += ADVimba

PROD_LIBS += ADGenICam

PROD_LIBS_WIN32 += VimbaCPP VimbaC VimbaImageTransform

PROD_LIBS_Linux += VimbaCPP

PROD_SYS_LIBS_Linux += VimbaC VimbaImageTransform

...



If I build my 'static' IOC, I get a binary which needs the .so libraries:

 $ ldd bin/rhel7-x86_64/vimbaApp
linux-vdso.so.1 =>  (0x00007ffe985dc000)
libVimbaC.so => /afs/slac.stanford.edu/g/spear/vol8/epics/modules/areaDetector/R3.9/ADVimba/lib/rhel7-x86_64/libVimbaC.so (0x00007ff613966000)
libVimbaImageTransform.so => /afs/slac.stanford.edu/g/spear/vol8/epics/modules/areaDetector/R3.9/ADVimba/lib/rhel7-x86_64/libVimbaImageTransform.so (0x00007ff61366c000)
libX11.so.6 => /lib64/libX11.so.6 (0x00007ff61332e000)
libXext.so.6 => /lib64/libXext.so.6 (0x00007ff61311c000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff612f00000)
libreadline.so.6 => /lib64/libreadline.so.6 (0x00007ff612cba000)
libncurses.so.5 => /lib64/libncurses.so.5 (0x00007ff612a93000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007ff612869000)
librt.so.1 => /lib64/librt.so.1 (0x00007ff612661000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007ff61245d000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007ff612156000)
libm.so.6 => /lib64/libm.so.6 (0x00007ff611e54000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff611c3e000)
libc.so.6 => /lib64/libc.so.6 (0x00007ff611870000)
libxcb.so.1 => /lib64/libxcb.so.1 (0x00007ff611648000)
/lib64/ld-linux-x86-64.so.2 (0x00007ff613c3a000)
libXau.so.6 => /lib64/libXau.so.6 (0x00007ff611444000)


$ objdump -p bin/rhel7-x86_64/vimbaApp | grep NEEDED
  NEEDED               libVimbaC.so
  NEEDED               libVimbaImageTransform.so
  NEEDED               libX11.so.6
  NEEDED               libXext.so.6
  NEEDED               libpthread.so.0
  NEEDED               libreadline.so.6
  NEEDED               libncurses.so.5
  NEEDED               libtinfo.so.5
  NEEDED               librt.so.1
  NEEDED               libdl.so.2
  NEEDED               libstdc++.so.6
  NEEDED               libm.so.6
  NEEDED               libgcc_s.so.1
  NEEDED               libc.so.6

 

 

So it's not a static build, but I guess this is expected.  I'm guessing that until the vendor releases source code or archive libraries, that there's just no way around this?

I suppose one option is to drop the .so libraries in a directory whose path exists on both the development and production servers, e.g. /usr/local/lib/ or somewhere like that?

 

Is there a better way of dealing with this in the EPICS build system, or some other clever trick?

I guess this could be viewed as a more general question of how to do a static IOC build when you only have shared object libraries.

 

 

 

Thanks,

Mike

From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Pearson, Matthew R. via Tech-talk <tech-talk at aps.anl.gov>
Sent: Tuesday, July 28, 2020 3:06 PM
To: Mark Rivers <rivers at cars.uchicago.edu>; tech-talk at aps.anl.gov <tech-talk at aps.anl.gov>
Subject: RE: ADProsilica crashing on RHEL7 3.10.0-1127

 

Hi,

Quick update on this. A power cycle on both cameras fixed both issues.

However, we will plan to move to ADVimba regardless.

Cheers,
Matt

-----Original Message-----
From: Tech-talk <tech-talk-bounces at aps.anl.gov> On Behalf Of Pearson, Matthew R. via Tech-talk
Sent: Friday, July 24, 2020 10:07 AM
To: Mark Rivers <rivers at cars.uchicago.edu>
Cc: tech-talk at aps.anl.gov
Subject: [EXTERNAL] RE: ADProsilica crashing on RHEL7 3.10.0-1127

Hi Mark & Mike,

The camera is reachable. I tried a non-existent camera, and the IOC starts up normally with the usual message:

prosilicaConfig("AVT.CAM", 10.111.24.99, 100, 0, 0, 0, 10)
2020/07/24 09:59:13.252 prosilica:connectCamera: Cannot find camera 10.111.24.99
prosilica:prosilica: cannot connect to camera 10.111.24.99, manually connect when available.

Also thanks Mike for the info. I also see it affecting a sub-set of our cameras. The two affected cameras are one beamline, and we have a working camera on a different beamline (same Linux versions though). Out of the 2 affected cameras, one crashes with the same seg fault, and the other acts differently:

prosilicaConfig("AVT.CAM", 10.111.24.11, 100, 0, 0, 0, 10)
2020/07/24 10:03:18.670 prosilica:connectCamera: unable to get sensor data on camera 5122144
prosilica:prosilica: cannot connect to camera 10.111.24.11, manually connect when available.

I will be able to head in to the beamline on Monday to power cycle the cameras. We have more than a week before our facility starts up again, so we may try switching to ADVimba.

Cheers,
Matt


-----Original Message-----
From: Mark Rivers <rivers at cars.uchicago.edu>
Sent: Thursday, July 23, 2020 7:49 PM
To: Pearson, Matthew R. <pearsonmr at ornl.gov>
Cc: tech-talk at aps.anl.gov
Subject: [EXTERNAL] Re: ADProsilica crashing on RHEL7 3.10.0-1127

Hi Matt,


Is that camera reachable?  Does it crash if you try to connect to a non-existent camera?


I am updating one of our Linux machines to that version of Centos 7.  I can see if I have the problem.  All of our production AVT/Prosilica cameras are now running ADVimba, but I can try a test with ADProsilica.


Mark



________________________________
From: Tech-talk <tech-talk-bounces at aps.anl.gov> on behalf of Pearson, Matthew R. via Tech-talk <tech-talk at aps.anl.gov>
Sent: Thursday, July 23, 2020 4:41 PM
To: tech-talk at aps.anl.gov
Subject: ADProsilica crashing on RHEL7 3.10.0-1127


Hi,



We've been using ADProsilica to control our AVT Manta cameras for many years. This uses the legacy PvApi library from AVT. It seems to be failing under our latest RHEL7 update (3.10.0-1127.13.1.el7.x86_64).



I've pasted the stack trace below. The problem seems to be deep inside libPvAPI.a, which is linked into libprosilica.so. libPvAPI.a was provided by AVT, but the last build was in 2015.



Anyone else seen this issue?



I know there is a newer areaDetector driver for AVT cameras, so it may be the case that we just have to switch to using that.



Cheers,

Matt



Program received signal SIGSEGV, Segmentation fault.

0x00007ffff7b61a7a in PGc::TiXmlAttributeSet::Find(char const*) const () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

Missing separate debuginfos, use: debuginfo-install glibc-2.17-307.el7.1.x86_64 jbigkit-libs-2.0-11.el7.x86_64 libgcc-4.8.5-39.el7.x86_64 libjpeg-turbo-1.2.90-8.el7.x86_64 libstdc++-4.8.5-39.el7.x86_64 libtiff-4.0.3-32.el7.x86_64 ncurses-libs-5.9-14.20130511.el7_4.x86_64 readline-6.2-11.el7.x86_64 sssd-client-1.16.4-37.el7_8.3.x86_64 zlib-1.2.7-18.el7.x86_64

(gdb)

(gdb)

(gdb)

(gdb)

(gdb) bt

#0  0x00007ffff7b61a7a in PGc::TiXmlAttributeSet::Find(char const*) const () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#1  0x00007ffff7b61acd in PGc::TiXmlElement::Attribute(char const*) const () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#2  0x00007ffff7b683db in SearchForNamedNode(PGc::TiXmlNode*, char const*) () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#3  0x00007ffff7b4dd55 in cGcContext::GetNode(char const*, bool) () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#4  0x00007ffff7b60ad2 in cGcBoolNode::SetupFromXML(PGc::TiXmlNode*, PGc::TiXmlNode*) () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#5  0x00007ffff7b4bef5 in cGcContext::BuildNodeFromXML(char const*, PGc::TiXmlNode*) () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#6  0x00007ffff7b4fcc2 in cGcContext::ProcessXMLDocument() () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#7  0x00007ffff7b5020c in cGcContext::LoadXMLString(char const*, bool) () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#8  0x00007ffff7b4ba12 in cGcInterface::InjectXMLString(char const*, bool) () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#9  0x00007ffff7b3730f in cPvGigEGenicam::Opening(unsigned int, unsigned int) () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#10 0x00007ffff7b43071 in pPvRawCamera::Open(unsigned int) () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#11 0x00007ffff7b303c5 in PvCameraOpenByAddr () from /home/controls/epics/ADProsilica/master/lib/linux-x86_64/libprosilica.so

#12 0x00007ffff7b2b3d7 in prosilica::connectCamera (this=0x6b7030) at ../prosilica.cpp:1317

#13 0x00007ffff7b2d952 in prosilica::prosilica (this=0x6b7030, portName=0x6b6ff0 "AVT.CAM", cameraId=0x6b6ff8 "10.111.24.10", maxBuffers=100, maxMemory=0, priority=0, stackSize=0,

    maxPvAPIFrames=10) at ../prosilica.cpp:1808

#14 0x00007ffff7b2d14b in prosilicaConfig (portName=0x6b6ff0 "AVT.CAM", cameraId=0x6b6ff8 "10.111.24.10", maxBuffers=100, maxMemory=0, priority=0, stackSize=0, maxPvAPIFrames=10)

    at ../prosilica.cpp:1689

#15 0x00007ffff7b2da2b in configprosilicaCallFunc (args=0x61ff60) at ../prosilica.cpp:1839

#16 0x00007ffff5cfc987 in iocshBody (pathname=0x7fffffffde25 "iocBoot/iocbl12-SampleCamera/st.cmd", commandLine=0x0) at ../../../src/libCom/iocsh/iocsh.cpp:771

#17 0x00007ffff5cfccaa in iocsh (pathname=0x7fffffffde25 "iocBoot/iocbl12-SampleCamera/st.cmd") at ../../../src/libCom/iocsh/iocsh.cpp:835

#18 0x0000000000405c0b in main (argc=2, argv=0x7fffffffda88) at ../bl12-SampleCameraMain.cpp:17

Replies:
Re: ADProsilica crashing on RHEL7 3.10.0-1127 Johnson, Andrew N. via Tech-talk
References:
ADProsilica crashing on RHEL7 3.10.0-1127 Pearson, Matthew R. via Tech-talk
Re: ADProsilica crashing on RHEL7 3.10.0-1127 Mark Rivers via Tech-talk
RE: ADProsilica crashing on RHEL7 3.10.0-1127 Pearson, Matthew R. via Tech-talk
RE: ADProsilica crashing on RHEL7 3.10.0-1127 Pearson, Matthew R. via Tech-talk
Re: ADProsilica crashing on RHEL7 3.10.0-1127 Dunning, Michael via Tech-talk
RE: ADProsilica crashing on RHEL7 3.10.0-1127 Mark Rivers via Tech-talk
Navigate by Date:
Prev: RE: ADProsilica crashing on RHEL7 3.10.0-1127 Mark Rivers via Tech-talk
Next: Re: ADProsilica crashing on RHEL7 3.10.0-1127 Johnson, Andrew N. via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <2020>  2021  2022  2023  2024 
Navigate by Thread:
Prev: RE: ADProsilica crashing on RHEL7 3.10.0-1127 Mark Rivers via Tech-talk
Next: Re: ADProsilica crashing on RHEL7 3.10.0-1127 Johnson, Andrew N. via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <2020>  2021  2022  2023  2024 
ANJ, 25 Sep 2020 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·