Experimental Physics and Industrial Control System
I believe this is the same problem which we've experienced, but now fixed, in every synApps module that defines a new record.
The .dbd files must be processed to generate the .h files. When running without parallel make things worked fine with the existing src/Makefile rules. However, when running parallel make it would occasionally fail because it tries to compile the record.c file before the record.h file exists.
This is the change I made earlier this week in synApps/vme/vmeApp/src/Makefile:
corvette:vme/vmeApp/src>svn diff -rPREV Makefile
Index: Makefile
===================================================================
--- Makefile (revision 14860)
+++ Makefile (working copy)
@@ -62,3 +62,6 @@
#----------------------------------------
# ADD RULES AFTER THIS LINE
+# Parallel build sometimes fails.
+# Make dependences on epidRecord.h explicit.
+vmeRecord$(OBJ): $(COMMON_DIR)/vmeRecord.h
Once I added such lines to all Makefiles in synApps modules that build new record types then a parallel make of synApps has never failed.
The problem we are seeing with seq looks the same.
Mark
________________________________________
From: Benjamin Franksen [[email protected]]
Sent: Friday, July 06, 2012 5:59 AM
To: Mark Rivers; [email protected]
Subject: Re: Problems with parallel make on seq-2-1-7
Hi Mark
I add tech-talk to the list since this is a problem in the EPICS base rules.
On Friday, July 06, 2012, you wrote:
> With the old version it failed about 1 in 10 times at most.
>
> With the new version there is a more serious problem. It fails even
> without a parallel make!
>
> I did a parallel make on the clean and uninstall, but then a non-parallel
> make on the build:
>
> corvette:~/devel/seq>make -sj uninstall
>
> corvette:~/devel/seq>make -s
> mkdir ../../../include
> mkdir ../../../lib/linux-x86
> mkdir ../../../lib/vxWorks-68040
> mkdir ../../../bin/vxWorks-68040
> mkdir ../../../lib/vxWorks-ppc603_long
> mkdir ../../../bin/vxWorks-ppc603_long
> mkdir ../../../lib/vxWorks-ppc604_long
> mkdir ../../../bin/vxWorks-ppc604_long
> mkdir ../../../bin/linux-x86
> mkdir ../../../dbd
> ../snl.re:16:17: fatal error: snl.h: No such file or directory
> compilation terminated.
> make[3]: *** [lexer.o] Error 1
> make[2]: *** [install.linux-x86] Error 2
> make[1]: *** [snc.install] Error 2
> make: *** [src.install] Error 2
> corvette:~/devel/seq>
>
> If I do a parallel make on the build it actually works most if not all of
> the time, but a non-parallel make fails. This seems strange.
I tried this on another comupter that runs a fedora linux today and I could
reproduce this. The make version is 3.82 (on my system it is 3.81) and this
is probably the same version you have.
After the failed make (without -j) search for lexer.d; you won't find it.
That means there is no dependency of lexer.o on snl.h and thus it does not
get created before trying to compile lexer.c. If you add snl.h explicitly to
the prerequisites of lexer.o
lexer.o: snl.h
Then the build works fine. However, this dependency should have been
generated automatically, there should be a file lexer.d that lists all the
dependencies of lexer.o on all the header files lexer.c includes.
The parallel build works because the rules that generates snl.c and snl.h is
executed in parallel and therefore in this case the snl.h (accidentally)
exists. This is a excerpt of the relevanmt rules in the sequencer's Makefile
(the fixed version, not the one in 2.1.7):
snl.c: snl.lem snl.lt $(LEMON)
$(RM) snl.c snl.h
$(LEMON) o=. $< # this command creates snl.c and snl.h
snl.h: snl.c
# empty recipy needed to work around make bug #35455
vpath %.re ..
lexer.c: snl.re
$(RE2C) -s -b -o $@ $<
These rules are fine. What's wrong is teh rules in base that build the
dependency files (.d). The following is from RULES_BUILD:
%$(OBJ): %.c
@$(HDEPENDS_CMD)
@$(RM) $@
$(COMPILE.c) $(call PATH_FILTER,$<) $(COMPILE_FILTER.c)
%$(OBJ): %.cc
@$(HDEPENDS_CMD)
@$(RM) $@
$(COMPILE.cpp) $(call PATH_FILTER,$<) $(COMPILE_FILTER.cpp)
%$(OBJ): %.cpp
@$(HDEPENDS_CMD)
@$(RM) $@
$(COMPILE.cpp) $(call PATH_FILTER,$<) $(COMPILE_FILTER.cpp)
This means .d files are not generated before the rules to compile a .c file
fires. Instead they are generated together with the object files. This is
bad, since it means that the header files must all exist or be explicitly
added as dependencies.
Instead, there should explicit rules that generate .d from .c:
%$(DEP): %.c
@$(HDEPENDS_CMD)
and similar for .cc and .cpp. When I add such a rule to the Makefile,
everything build just fine on the fedora system with make 3.82. The
$(HDEPENDS_CMD) can then be removed from the object generation rules.
There remains the question what do about this problem? Should I add the
extra dependency (lexer.o: snl.h) to the Makefile? Or should I add the
missing rules for .d files?
Cheers
Ben
- Replies:
- Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
- References:
- Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
- Navigate by Date:
- Prev:
Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
- Next:
Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
<2012>
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
- Next:
Re: Problems with parallel make on seq-2-1-7 Benjamin Franksen
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
<2012>
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024