Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018 
<== Date ==> <== Thread ==>

Subject: RE: Problems with parallel make on seq-2-1-7
From: Mark Rivers <rivers@cars.uchicago.edu>
To: Benjamin Franksen <benjamin.franksen@helmholtz-berlin.de>, "tech-talk@aps.anl.gov" <tech-talk@aps.anl.gov>
Date: Fri, 6 Jul 2012 11:36:33 +0000
I believe this is the same problem which we've experienced, but now fixed, in every synApps module that defines a new record.

The .dbd files must be processed to generate the .h files.  When running without parallel make things worked fine with the existing src/Makefile rules.  However, when running parallel make it would occasionally fail because it tries to compile the record.c file before the record.h file exists.

This is the change I made earlier this week in synApps/vme/vmeApp/src/Makefile:

corvette:vme/vmeApp/src>svn diff -rPREV Makefile 
Index: Makefile
===================================================================
--- Makefile    (revision 14860)
+++ Makefile    (working copy)
@@ -62,3 +62,6 @@
 #----------------------------------------
 #  ADD RULES AFTER THIS LINE
 
+# Parallel build sometimes fails.
+# Make dependences on epidRecord.h explicit.
+vmeRecord$(OBJ):          $(COMMON_DIR)/vmeRecord.h

Once I added such lines to all Makefiles in synApps modules that build new record types then a parallel make of synApps has never failed. 

The problem we are seeing with seq looks the same.

Mark


________________________________________
From: Benjamin Franksen [benjamin.franksen@helmholtz-berlin.de]
Sent: Friday, July 06, 2012 5:59 AM
To: Mark Rivers; tech-talk@aps.anl.gov
Subject: Re: Problems with parallel make on seq-2-1-7

Hi Mark

I add tech-talk to the list since this is a problem in the EPICS base rules.

On Friday, July 06, 2012, you wrote:
> With the old version it failed about 1 in 10 times at most.
>
> With the new version there is a more serious problem.  It fails even
> without a parallel make!
>
> I did a parallel make on the clean and uninstall, but then a non-parallel
> make on the build:
>
> corvette:~/devel/seq>make -sj uninstall
>
> corvette:~/devel/seq>make -s
> mkdir ../../../include
> mkdir ../../../lib/linux-x86
> mkdir ../../../lib/vxWorks-68040
> mkdir ../../../bin/vxWorks-68040
> mkdir ../../../lib/vxWorks-ppc603_long
> mkdir ../../../bin/vxWorks-ppc603_long
> mkdir ../../../lib/vxWorks-ppc604_long
> mkdir ../../../bin/vxWorks-ppc604_long
> mkdir ../../../bin/linux-x86
> mkdir ../../../dbd
> ../snl.re:16:17: fatal error: snl.h: No such file or directory
> compilation terminated.
> make[3]: *** [lexer.o] Error 1
> make[2]: *** [install.linux-x86] Error 2
> make[1]: *** [snc.install] Error 2
> make: *** [src.install] Error 2
> corvette:~/devel/seq>
>
> If I do a parallel make on the build it actually works most if not all of
> the time, but a non-parallel make fails.  This seems strange.

I tried this on another comupter that runs a fedora linux today and I could
reproduce this. The make version is 3.82 (on my system it is 3.81) and this
is probably the same version you have.

After the failed make (without -j) search for lexer.d; you won't find it.
That means there is no dependency of lexer.o on snl.h and thus it does not
get created before trying to compile lexer.c. If you add snl.h explicitly to
the prerequisites of lexer.o

lexer.o: snl.h

Then the build works fine. However, this dependency should have been
generated automatically, there should be a file lexer.d that lists all the
dependencies of lexer.o on all the header files lexer.c includes.

The parallel build works because the rules that generates snl.c and snl.h is
executed in parallel and therefore in this case the snl.h (accidentally)
exists. This is a excerpt of the relevanmt rules in the sequencer's Makefile
(the fixed version, not the one in 2.1.7):

snl.c: snl.lem snl.lt $(LEMON)
        $(RM) snl.c snl.h
        $(LEMON) o=. $< # this command creates snl.c and snl.h

snl.h: snl.c
        # empty recipy needed to work around make bug #35455

vpath %.re ..

lexer.c: snl.re
        $(RE2C) -s -b -o $@ $<

These rules are fine. What's wrong is teh rules in base that build the
dependency files (.d). The following is from RULES_BUILD:

%$(OBJ): %.c
        @$(HDEPENDS_CMD)
        @$(RM) $@
        $(COMPILE.c) $(call PATH_FILTER,$<) $(COMPILE_FILTER.c)

%$(OBJ): %.cc
        @$(HDEPENDS_CMD)
        @$(RM) $@
        $(COMPILE.cpp) $(call PATH_FILTER,$<) $(COMPILE_FILTER.cpp)

%$(OBJ): %.cpp
        @$(HDEPENDS_CMD)
        @$(RM) $@
        $(COMPILE.cpp) $(call PATH_FILTER,$<) $(COMPILE_FILTER.cpp)

This means .d files are not generated before the rules to compile a .c file
fires. Instead they are generated together with the object files. This is
bad, since it means that the header files must all exist or be explicitly
added as dependencies.

Instead, there should explicit rules that generate .d from .c:

%$(DEP): %.c
        @$(HDEPENDS_CMD)

and similar for .cc and .cpp. When I add such a rule to the Makefile,
everything build just fine on the fedora system with make 3.82. The
$(HDEPENDS_CMD) can then be removed from the object generation rules.

There remains the question what do about this problem? Should I add the
extra dependency (lexer.o: snl.h) to the Makefile? Or should I add the
missing rules for .d files?

Cheers
Ben


Replies:
Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
References:
Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen

Navigate by Date:
Prev: Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
Next: Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018 
Navigate by Thread:
Prev: Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
Next: Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  <20122013  2014  2015  2016  2017  2018 
ANJ, 18 Nov 2013 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·