Re: EPICS build system DESTDIR support

["J. Lewis Muir" <jlmuir@imca-cat.org>]

On 10/05, Andrew Johnson wrote:
> Hi Lewis,
> 
> On 10/05/2016 12:53 PM, J. Lewis Muir wrote:
> > What would it take to get DESTDIR support added to the EPICS build
> > system?  I haven't looked, but I would think doing it for Linux and
> > macOS would not be too hard, but I don't know about Windows or other
> > platforms.  And then there's cross-compiling, which is probably tricky,
> > and for which I'd be interested in VxWorks.
> > 
> > There was a thread about this two years ago at [1].
> 
> > [1] http://www.aps.anl.gov/epics/tech-talk/2014/msg01500.php
> 
> The main complication with doing something like this is that the act of
> building Base involves running programs that were compiled earlier in
> the same build. Those programs need to be able to find shared libraries
> that have also only just been built, and the way we implemented that on
> Unix-like systems was to use the linker's RPATH.

Hi, Andrew!

(Sorry to be revisiting this old thread.)

Hmm, yes, that's an issue, then.

What would you think about a restructured build that builds a bootstrap
version first and then uses the bootstrap version to build a final
version after that?  In other words, the bootstrap version would be
built and installed to a temporary build directory.  Then you use the
bootstrap version to build the final version which gets installed under
the DESTDIR location.  This way it doesn't use the version of itself
that it's currently building.  The bootstrap could be a full build,
but ideally it would be a minimal build (i.e., just enough for the
bootstrap).  This feels like a more traditional bootstrapping scheme to
me, and I think it would solve the problem you described above.

> We added the LINKER_USE_RPATH=NO support for Michael to use with the
> Debian packages, but as he mentioned in the above thread, doing that
> requires you to set LD_LIBRARY_PATH or use the /etc/ld.so.conf mechanism
> to set the shared library search path at both build-time and run-time.
> Editing the RPATH in existing binaries doesn't seem to be well
> supported, and isn't an approach I would be keen on.

On which platforms do you mean it doesn't seem well supported?  It seems
like it's at least supported on Linux and macOS.

I think the dynamic linker environment variable (LD_LIBRARY_PATH on
Linux and DYLD_LIBRARY_PATH on macOS) and /etc/ld.so.conf approaches
are both undesirable because they require special configuration of the
environment or system just to run the executable.  On top of that,
I'm thinking the environment variable approach won't even work on
macOS Sierra (and El Capitan too?) with SIP enabled since /bin/sh is a
so-called protected process, and [1] says:

  Spawning children processes of processes restricted by System
  Integrity Protection, such as by launching a helper process in a
  bundle with NSTask or calling the exec(2) command, resets the Mach
  special ports of that child process. Any dynamic linker (dyld)
  environment variables, such as DYLD_LIBRARY_PATH, are purged when
  launching protected processes.

So, even if DYLD_LIBRARY_PATH was set, it might get removed from the
executable's environment by SIP depending on how it gets executed.

> Michael's message in the above thread said:
> > $ make FINAL_LOCATION=/usr/local/epics INSTALL_LOCATION=/tmp/builddir LINKER_USE_RPATH=NO
> > 
> > The LINKER_USE_RPATH=NO is necessary as rpath is always set using INSTALL_LOCATION.
> 
> It occurs to me that we might be able to set RPATH using FINAL_LOCATION
> instead of INSTALL_LOCATION if the GNUmake build scripts were to set
> LD_LIBRARY_PATH for the build-time search resolution. I don't know if
> this would help in solving your problem, it might.

That's a good idea, but unfortunately I think it might not work on macOS
with SIP enabled because of what I described above.

> Note that on Windows there is no equivalent to RPATH, so there the
> GNUmake build scripts just add the INSTALL_BIN directory to PATH (DLLs
> are installed into the bin directory, not lib). Users have to manage
> their own PATH when building and running IOCs and client programs,
> although we do generate a script for IOCs to set PATH based on the IOC's
> RELEASE file.

OK, good to know; thanks for that.

> If you or someone else wants to develop an alternative way to handle
> these build issues the core developers would be happy to advise and
> help, but any replacement must support builds on Linux, Windows and
> macOS/darwin at minimum, and handle cross-builds for the majority of our
> current cross-targets.

OK, thanks!

Regards,

Lewis

[1] https://developer.apple.com/library/content/documentation/Security/Conceptual/System_Integrity_Protection_Guide/RuntimeProtections/RuntimeProtections.html