Experimental Physics and
Industrial Control System

Michael Davidsaver via Core-talk <core-talk at aps.anl.gov> · Sun, 8 Nov 2020 10:29:13 -0800

On 11/6/20 4:00 PM, Johnson, Andrew N. wrote:
> Hi Michael,
> 
> Sorry, long answer…
> 
> On Nov 6, 2020, at 1:47 AM, Michael Davidsaver <mdavidsaver at gmail.com <mailto:mdavidsaver at gmail.com>> wrote:
>>
>> In configure/os/CONFIG.Common.UnixCommon
>>
>>> LDLIBS_STATIC_YES = LDLIBS
>>> LDLIBS_SHARED_NO = LDLIBS
>>> PROD_LDLIBS += $($(firstword $(LDLIBS_STATIC_$(STATIC_BUILD)) \
>>>               $(LDLIBS_SHARED_$(SHARED_LIBRARIES))))
>>
>> I think I've broken my brain tonight trying to figure out what all
>> is going on here.
> 
> Firstly this is in the Common.UnixCommon file, which gets included before the arch-specific files so this file can’t use if … endif conditionals that depend on things that get set or overridden later. It might be able to use a $(if …) construct instead, but may have been written before that feature got added to GNUmake.
> 
>> I think I understand what is going on here in pseudo-code as:
>>
>>> if STATIC_BUILD==YES || SHARED_LIBRARIES==NO
>>> PROD_LDLIBS += $(LDLIBS)
>>> endif
>>
>> Does this seem a correct translation?
>>
>> Why is SHARED_LIBRARIES being tested?  It seems redundant.
>> STATIC_BUILD=NO implies SHARED_LIBRARIES=YES
> 
> SHARED_LIBRARIES controls whether we build .so/.dll versions of libraries.
> STATIC_BUILD controls whether we give the linker the flag -static or -Bstatic or however it’s spelled on that target when creating executables.
> 
> On Windows those two settings /have/ to be exclusive, but on other arch’s they don’t. On those arch’s we always construct lib.a files, but if SHARED_LIBRARIES is YES we also create lib.so files from the same set of .o files. On Windows the compiler has to be told which kind of library it's building, so the same set of .obj files can’t be used to link both a .dll and a lib.a hence the exclusivity requirement.
> 
> Thus for non-Windows builds you can set both flags to NO simultaneously and everything should still work – at least that was the case when Janet was responsible for the build system. I just built 3.15 configured NO + NO and the resulting softIoc ran quite happily, so I don’t seem to have broken that yet.
> 
> Why might you want to use NO + NO?

I had no idea that you considered this to be a valid combination.
At least I can understand why I was confused.

> Well sometimes it is not possible or desirable to link some code into a shared library (say some of it is written in assembler and isn’t PIC). The Makefile for that library can just set SHARED_LIBRARIES=NO and only the lib.a will be built even if Base has it set to YES. The linker will prefer to use a lib.so but will take a lib.a instead if the other doesn't exist, so this doesn’t break anything. This is how APS has traditionally built our extensions (MEDM, ALH, StripTool etc.) so the executables use shared libraries from Base but any internal libraries within the tool are all included into the binary.
> 
> The setting YES+YES is probably less useful (build lib.so files but link binaries statically – it ought to work, but seems fairly pointless).

As it happens, this is the combination I've long used (and recommended) where supported.
Since I consider the two settings mostly orthogonal on *nix, it makes sense to me for
shared libraries to always be built.  Then individual applications are left with the
choice of whether to link support modules statically.

(imo, the example of non-PIC assembly is a bit of a reach these days)

> So why is PROD_LDLIBS set this way? PROD_LDLIBS is used to construct the linker command-line for creating PRODs (i.e. executables). The complete definition is this:
> 
>> PROD_LDLIBS = $(addprefix -l, $($*_LDLIBS) $(PROD_LIBS) $(USR_LIBS)) \
>>     $(STATIC_LDLIBS) \
>>     $(addprefix -l, $($*_SYS_LIBS) $(PROD_SYS_LIBS) $(USR_SYS_LIBS))
>>
>> LDLIBS_STATIC_YES = LDLIBS
>> LDLIBS_SHARED_NO = LDLIBS
>> PROD_LDLIBS += $($(firstword $(LDLIBS_STATIC_$(STATIC_BUILD)) \
>>                $(LDLIBS_SHARED_$(SHARED_LIBRARIES))))
> 
> The first line generates the -ldbCore -lca -lCom parts from the lists of “regular” libraries the application wants including. The second line (STATIC_LDLIBS) holds the flags to switch the linker back to dynamic linking for the system libraries when STATIC_BUILD is YES. The third line adds the system libraries like the first line. The 7+8th lines append $(LDLIBS) if STATIC_BUILD is YES or SHARED_LIBRARIES is NO or both, which is what you translated it to above. So LDLIBS will /not/ be used when STATIC_BUILD is NO /and/ SHARED_LIBRARIES is YES, which is the default case for the workstation arch’s. 
> 
> So what’s in LDLIBS? It gets set in CONFIG_COMMON to this, which generates a list of target-specific libraries and linker flags from variables set by later build CONFIG files:
> 
>> LDLIBS = $(POSIX_LDLIBS) $(ARCH_DEP_LDLIBS) $(DEBUG_LDLIBS) \
>>          $(OP_SYS_LDLIBS) $(GNU_LDLIBS_$(GNU))
> 
> For the workstation arch's we have already linked all those target-specific libraries into libCom.so when we created that (LDLIBS is appended to SHRLIB_LDLIBS unconditionally, which is used instead of PROD_LDLIBS for lib.so linking), so there is no need to link against them again. If you aren’t linking your executable with libCom you'd probably need a different set of system libraries anyway.

Ok, I understand now.

> If you can see a simpler/shorter way to code this feel free to suggest it, but GNUmake's $(if) can’t do equality comparisons so it may have a few other $(findstring) or $(filter) calls to get the logic right.

Since we don't agree on the NO+NO case, any simplification to this logic would
probably lead to unnecessary conflict.  So I'll content myself with encouraging
you to add comments to less than obvious makefile logic like this.

> HTH,
> 
> - Andrew
> 
> -- 
> Complexity comes for free, simplicity you have to work for.

I try

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System