Experimental Physics and
Industrial Control System

"Zimoch Dirk \(PSI\) via Core-talk" <core-talk at aps.anl.gov> · Thu, 16 Jul 2020 16:23:46 +0000

With most gnu tool chains, I had no problems. But somewhere around 2.4 the behavior has changed.

If program P depends on library A and library A depends on library B but the program does not use any symbols from B, why should it care? Why should the linker ld care? There is nothing in P that needs to be resolved from B and A has already been built. The user may use -lA but not -lB.

The run-time linker ld.so is a completely different thing of course. When P is started, ld.so looks for A in (among others) the path that was given with -rpath when P had been built. Then when loading A, it finds the dependency on B and looks in the -rpath that was given when A had been built and loads B.

But -rpath must contain the path at run time, which may be completely different from the -L or -rpath-link path at build time, especially for cross architectures.

The linker ld has no need whatsoever for B at the time of building P. And I guess (older?) gnu tool chain versions did not care at all. Or used -L to find it.

But now some linkers find that A has a dependency on B and wants to see it. Why? I don't know. Maybe to check that there are no conflicts, for example dependencies on multiple different versions of a third library C (e.g.  libc.so). But it does not use -L (or -rpath) to find those dependencies (as the example proved) . It uses -rpath-link.

Let's check with asyn and an old tool chain without -rpath-link
$ strings ./asyn/asynPortDriver/unittest/O.eldk52-e500v2/asynPortDriverTest | grep 'lib.*so'
/lib/ld.so.1
libasyn.so
libCom.so
libstdc++.so.6
libm.so.6
libgcc_s.so.1
libc.so.6
No trace of libdbRecStd.so or libca.so. But it has libasyn.so, so let's check there:
$ strings lib/eldk52-e500v2/libasyn.so | grep 'lib.*so'
libdbRecStd.so
libdbCore.so
libca.so
libCom.so
libpthread.so.0
libreadline.so.6
librt.so.1
libdl.so.2
libstdc++.so.6
libm.so.6
libgcc_s.so.1
libc.so.6
Aha! Here is libdbRecStd.so. Thus, the linker did not care about libdbRecStd.so when building asynPortDriverTest. Why should it? It is not used in the program.

BTW: Do not use ldd to find out as it calls the dynamic linker and thus recursively shows the indirectly loaded libraries as well. That is not what is written in the file!

Now with -rpath-link enabled:
$ strings ./asyn/asynPortDriver/unittest/O.eldk52-e500v2/asynPortDriverTest | grep 'lib.*so'
/lib/ld.so.1
libasyn.so
libCom.so
libstdc++.so.6
libm.so.6
libgcc_s.so.1
libc.so.6
Nothing changed.

Let's try an architecture, that actually needs -rpath-link:
$ strings ./asyn/asynPortDriver/unittest/O.eldk53-ppc4xxFP/asynPortDriverTest | grep 'lib.*so'
/lib/ld.so.1
libasyn.so
libCom.so
libstdc++.so.6
libm.so.6
libgcc_s.so.1
libc.so.6
Still no trace of libdbRecStd.so.

That means the linker only wants to see the indirectly loaded library but does not actually do anything with it. But it refuses to build the program without it.

Now I actually removed libdbRecStd.so from my computer and still could rebuild asynPortDriverTest with the toolchain that did not need -rpath-link. But I got the warning.  That means, the linker checked as well for indirectly loaded libraries. But It is only a warning. The program was still built successfully! That also means that the linker must have used -L to find the library. 

I can do some more tests tomorrow. In particular I can find out which linker versions cause problems and which not.

BTW: Mark's fix of course solves the problem as well because now the program is linked explicitly with all base libraries.

Dirk

> -----Ursprüngliche Nachricht-----
> Von: Michael Davidsaver <mdavidsaver at gmail.com>
> Gesendet: Donnerstag, 16. Juli 2020 17:16
> An: Zimoch Dirk (PSI) <dirk.zimoch at psi.ch>
> Cc: 'core-talk at aps.anl.gov' <core-talk at aps.anl.gov>
> Betreff: Re: EPICS linker problem
> 
> > Some linkers do not use -L for indirectly linked libs.
> 
> Are you able to elaborate on this?  I know of a similar sounding issue
> with some older versions of binutils (I don't know the exact range)
> related to -rpath $ORIGIN which I worked around by passing -rpath-link
> as well.  I thought that this was limited to use of $ORIGIN though.
> 
> https://sourceware.org/bugzilla/show_bug.cgi?id=20535
> 
> https://github.com/epics-base/epics-
> base/blob/7d2e352f6a86eeec8c469917253e5db4af5c9c03/src/tools/makeRPat
> h.py#L69-L72
> 
> > using gcc 4.4.2, 4.7.2
> 
> which binutils (versions) are involved?   (aka. "ld --version")
> 
> 
> On 7/16/20 2:51 AM, Zimoch Dirk (PSI) via Core-talk wrote:
> > Hello folks,
> >
> > Today I found that some Linux (cross) architectures (using gcc 4.4.2, 4.7.2)
> require an additional linker flag: -Wl,-rpath-link in addition to -L and -Wl,-rpath. I
> noticed when compiling asyn.
> > It worked for me to change in
> <BASE>/configure/os/CONFIG.Common.linuxCommon
> > PRODDIR_RPATH_LDFLAGS_YES = $(PROD_DEPLIB_DIRS:%=-Wl,-rpath,%)
> > to
> > PRODDIR_RPATH_LDFLAGS_YES = $(PROD_DEPLIB_DIRS:%=-Wl,-rpath,%)
> $(PROD_DEPLIB_DIRS:%=-Wl,-rpath-link,%)
> > (Maybe the same should be done for SHRLIBDIR_RPATH_LDFLAGS_YES.)
> >
> > Why this? Some linkers do not use -L for indirectly linked libs. E.g. libA.so
> depends on libB.so and the program is linked with -lA but not with -lB, then some
> linkers require that B is found in -rpath-link and do not consider -L or -rpath for
> it. (Maybe the other linkers where it works do not even care about B?)
> >
> > In asyn this is the case for test code which links with -lasyn -lCom only. But
> libasyn.so had been linked with -ldbRecStd -ldbCore -lca -lCom and the linker
> refuses to use those other libraries from -L<EPICS_BASE_LIBDIR> or -Wl,-
> rpath<EPICS_BASE_LIBDIR>.
> >
> > /opt/eldk-5.3/powerpc-4xx/sysroots/i686-eldk-linux/usr/bin/ppc440e-
> linux/powerpc-linux-g++  -D_GNU_SOURCE -D_DEFAULT_SOURCE           -m32 -
> mcpu=440fp -mhard-float --sysroot=/opt/eldk-5.3/powerpc-
> 4xx/sysroots/ppc440e-linux  -DUNIX  -Dlinux  -fno-strict-aliasing    -O3 -g -g   -
> Wall             -I. -I../O.Common -I. -I. -I.. -I../../../../include/compiler/gcc -
> I../../../../include/os/Linux -I../../../../include          -I/usr/local/epics/base-
> 7.0.4.1/include/compiler/gcc -I/usr/local/epics/base-7.0.4.1/include/os/Linux -
> I/usr/local/epics/base-7.0.4.1/include        -c ../asynPortDriverTest.cpp
> > /opt/eldk-5.3/powerpc-4xx/sysroots/i686-eldk-linux/usr/bin/ppc440e-
> linux/powerpc-linux-g++ -o asynPortDriverTest  -
> L/afs/psi.ch/group/8211/dirk/git/DRV/asyn/lib/eldk53-ppc4xxFP -
> L/usr/local/epics/base-7.0.4.1/lib/eldk53-ppc4xxFP -Wl,-
> rpath,/afs/psi.ch/group/8211/dirk/git/DRV/asyn/lib/eldk53-ppc4xxFP -Wl,-
> rpath,/usr/local/epics/base-7.0.4.1/lib/eldk53-ppc4xxFP       --sysroot=/opt/eldk-
> 5.3/powerpc-4xx/sysroots/ppc440e-linux    -rdynamic         asynPortDriverTest.o
> -lasyn -lCom
> > /net/gfa-fs-4/export/cross_tools/eldk-5.3/powerpc-4xx/sysroots/i686-eldk-
> linux/usr/bin/ppc440e-linux/../../libexec/ppc440e-linux/gcc/powerpc-
> linux/4.7.2/ld: warning: libdbRecStd.so, needed by
> /afs/psi.ch/group/8211/dirk/git/DRV/asyn/lib/eldk53-ppc4xxFP/libasyn.so, not
> found (try using -rpath or -rpath-link)
> > /net/gfa-fs-4/export/cross_tools/eldk-5.3/powerpc-4xx/sysroots/i686-eldk-
> linux/usr/bin/ppc440e-linux/../../libexec/ppc440e-linux/gcc/powerpc-
> linux/4.7.2/ld: warning: libdbCore.so, needed by
> /afs/psi.ch/group/8211/dirk/git/DRV/asyn/lib/eldk53-ppc4xxFP/libasyn.so, not
> found (try using -rpath or -rpath-link)
> > /net/gfa-fs-4/export/cross_tools/eldk-5.3/powerpc-4xx/sysroots/i686-eldk-
> linux/usr/bin/ppc440e-linux/../../libexec/ppc440e-linux/gcc/powerpc-
> linux/4.7.2/ld: warning: libca.so, needed by
> /afs/psi.ch/group/8211/dirk/git/DRV/asyn/lib/eldk53-ppc4xxFP/libasyn.so, not
> found (try using -rpath or -rpath-link)
> > /afs/psi.ch/group/8211/dirk/git/DRV/asyn/lib/eldk53-ppc4xxFP/libasyn.so:
> undefined reference to `scanIoRequest'
> > [... more undefined reference errors...]
> >
> > Even though the warning suggests using -rpath or -rpath-link, only -rpath-link
> actually does the job.
> >
> > Is there any (old) Linux architecture where adding -rpath-link may cause
> problems?
> >
> > Dirk
> >

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System