Re: EPICS build system DESTDIR support

[Andrew Johnson <anj@aps.anl.gov>]

Hi Lewis,

On 06/20/2017 03:38 PM, J. Lewis Muir wrote:
> What would you think about a restructured build that builds a bootstrap
> version first and then uses the bootstrap version to build a final
> version after that?  In other words, the bootstrap version would be
> built and installed to a temporary build directory.  Then you use the
> bootstrap version to build the final version which gets installed under
> the DESTDIR location.  This way it doesn't use the version of itself
> that it's currently building.  The bootstrap could be a full build,
> but ideally it would be a minimal build (i.e., just enough for the
> bootstrap).  This feels like a more traditional bootstrapping scheme to
> me, and I think it would solve the problem you described above.

Unfortunately that approach seems rather complicated and brittle to me
(meaning it would be easy to break by someone who commits build system
changes without explicitly thinking about this particular configuration
of the build).

My preferred approach would be to replace all programs that need to run
at build-time with Perl scripts, but we still rely heavily on Flex and
Antelope to create lexers and parsers for the IOC (for DBD/DB and AS
files), and there are no Perl equivalents which can generate C or C++ code.

>> Editing the RPATH in existing binaries doesn't seem to be well
>> supported, and isn't an approach I would be keen on.
> 
> On which platforms do you mean it doesn't seem well supported?  It seems
> like it's at least supported on Linux and macOS.

There are tools which can edit the RPATH setting in an ELF binary, but
IIRC they can only shorten the path, never make it longer. That's an
indication to me that what they are doing probably isn't completely
kosher, and I would rather not rely on arbitrary limits such as the
length of the RPATH that we first compile in. Until the standard GNU
binutils programs (objcopy and/or elfedit) provide a way to change an
RPATH I would be reluctant to use this approach in Base itself.

> I think the dynamic linker environment variable (LD_LIBRARY_PATH on
> Linux and DYLD_LIBRARY_PATH on macOS) and /etc/ld.so.conf approaches
> are both undesirable because they require special configuration of the
> environment or system just to run the executable. 

I did some experiments recently to add another build configuration such
that you can set LINKER_USE_RPATH = ORIGIN and it sets the built-in
RPATH to $ORIGIN/../../lib/$(T_A). $ORIGIN is supported by most Unix
linking loaders and gets replaced by the directory where the binary is
found, so at least for building Base the build-time programs will work
at any location without having to set LD_LIBRARY_PATH or equivalent.

However when it comes to building modules or IOC applications using the
Base installation the $ORIGIN rpath would only work if you installed
*everything* into the same destination directory, which is obviously not
desirable. Without that you would still have to set LD_LIBRARY_PATH to
encompass the lib/$(T_A) dir's of all the dependent modules which export
shared libraries. I have a second patch which does that at build-time,
but the result didn't seem ideal.

> On top of that,
> I'm thinking the environment variable approach won't even work on
> macOS Sierra (and El Capitan too?) with SIP enabled since /bin/sh is a
> so-called protected process, and [1] says:
> 
>   Spawning children processes of processes restricted by System
>   Integrity Protection, such as by launching a helper process in a
>   bundle with NSTask or calling the exec(2) command, resets the Mach
>   special ports of that child process. Any dynamic linker (dyld)
>   environment variables, such as DYLD_LIBRARY_PATH, are purged when
>   launching protected processes.
> 
> So, even if DYLD_LIBRARY_PATH was set, it might get removed from the
> executable's environment by SIP depending on how it gets executed.

That quotation is discussing launching processes inside a bundle. I
don't think we would do that — I'm not a MacOS expert, but we don't
create an EPICS Base bundle for the Mac, so that seems irrelevant to me.
You might want to experiment with building Base for MacOS with
LINKER_USE_RPATH = NO and LD_LIBRARY_PATH set to see if you have
problems with it. I'd be happy to send you my $ORIGIN patch(es) to
experiment with too — I didn't make the necessary changes to the
darwinCommon files yet, but they should be quite simple when you see the
other examples.

HTH,

- Andrew

-- 
Arguing for surveillance because you have nothing to hide is no
different than making the claim, "I don't care about freedom of
speech because I have nothing to say." -- Edward Snowdon