1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 <2015> 2016 2017 2018 2019 2020 2021 2022 2023 2024 | Index | 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 <2015> 2016 2017 2018 2019 2020 2021 2022 2023 2024 |
<== Date ==> | <== Thread ==> |
---|
Subject: | RE: Base R3.15.2-rc1 release |
From: | Mark Rivers <[email protected]> |
To: | "'Johnson, Andrew N.'" <[email protected]> |
Cc: | EPICS Tech-Talk <[email protected]> |
Date: | Tue, 5 May 2015 12:24:27 +0000 |
Hi Andrew, Thanks for the suggestions of using –jN and trying make 4.1. I have done a series of tests with make 3.81 (GNUWin32 version) and make 4.1 (version you built). The results are quite interesting. Test conditions: -
Windows 7 64-bit computer -
8 cores -
Visual Studio 2010 compiler -
EPICS_HOST_ARCH=win32-x86-static -
Local installations of base-3.14.12.5 and base-3.15.2-rc1 on C: drive. -
Remote installation of base-3.15.2-rc1 on a Linux file server I first tested building base-3.14.12.5 with both versions of make, and 1 (-s), 2 (-sj2), 4 (-sj4), 8 (-sj8) and unlimited (-sj) numbers of make threads. The following table show the seconds to build as a function of the version of make and the make options. make options make version -s -sj2 -sj4 -sj8 -sj 3.81 394 391 383 383 94 4.1 369 199 119 85 91 These results show a serious problem with make 3.81. There is no significant decrease in the execution time as the number of threads is increased, unless there is no limit on the
number of threads. Using the Windows Task Manager I could easily see why: as the number of threads was increased, using –sj8 for example, there were multiple instances of “make” running. However, there was only at most one instance of the a CPU bound tasks
such as the compiler (cl.exe), linker (link.exe), or perl (perl.exe) running at once. Only when an unlimited number of threads was allowed (-sj) were there then multiple instances of cl.exe, link.exe, and perl.exe. Make 4.1, on the other hand appears to work well. The execution time dropped smoothly with an increased number of threads. It was a minimum when the number of threads matched the
number of cores (8), and increased slightly from that when an unlimited number of threads was allowed. I then tested building base-3.15.2-rc1. I only tested with –sj, -sj8 (4.1 only because of above results) and –sj. With –sj and –sj8 I tested several times with each version of make.
This is the seconds to build or whether the build hung up. make options make version -s -sj8 -sj 3.81 644 157 150 149 (previous testing of 3.81 on with local installation of 3.15.2-rc1 did result in occasional hangs) 4.1 623 147 hung 157 hung 150 hung 155 hung 154 167 When make 4.1 hung this was the final output: main.cc exServer.cc exPV.cc exVectorPV.cc exScalarPV.cc exAsyncPV.cc exChannel.cc Generating code Finished generating code When it was hung make was using 100% of a single core CPU. Finally I tested building both base-3.14.12.5 and base-3.15.2-rc1 on a remote Linux file server make options make version base version -s -sj8 -sj 3.81 3.14.12.5 1583 479 3.81 3.15.2-rc1 2549 hung (2/2 times) 4.1 3.14.12.5 1693 568 547 4.1 3.15.2-rc1 2798 1178 hung (2/2 times) When make 3.81 or 4.1 hung building on the remote Linux file server this was the final output: nfa.c misc.c gen.c ecs.c dfa.c ccl.c epicsTempFile.cpp When it hung make was using 100% of a single core CPU. My observations and conclusions: -
make 3.81 does not work properly with –jN, but it does work properly with –j (unlimited threads). -
make 3.81 and 4.1 both work fine building 3.14.12.5 either locally or remotely on a Linux file server with –sj (unlimited threads). -
make 3.81 and 4.1 both frequently hang up when building 3.15.2-rc1 locally with –sj. 4.1 appears to with OK with –sj8. -
make 3.81 and 4.1 both fail 100% of the time when building 3.15.2-rc-1 remotely with –sj. This does NOT happen with 3.14.12.5. Note that this cannot be because the all
8 cores are 100% busy, because they are never more than about 40% busy. The build is completely I/O bound when building remotely, but –sj still hangs. -
3.15.2-rc1 takes 1.5 to 2 times longer to build than 3.14.12.5. Mark From: Johnson, Andrew N. [mailto:[email protected]]
Hi Mark, Could you try using my Make 4.1 executable instead, which you can find linked from the Base - Windows page under Other Build Tools. I do remember a comment in the release notes about some improvement to parallel builds on Windows in version
4, although I don't know if this will help or not. I usually use 'make -sj4' to limit the parallelism to 4 which I think might prevent some problems; obviously you can adjust the number of processes to see what works best on your machine. If you don't specify a limit make will keep adding
processes until the CPU load gets too high, and I think from experience that's not a very good strategy on Windows. -- Sent from my iPad
|