I just restarted the build of this PR after downloading the full build log of the one job that failed after 1 hr:
Build
EPICS Base 7 base-7.0-1411 failed
Commit 43d6ed95f2 by Jeremy
Lorelli on 12/22/2024 8:25 PM:
Add EPICS_DIAGNOSE_AS attribute for clang
The symptoms look different but may be related: In this case toolchain.c was successfully built, but despite the job being stopped after the 1hr the last entry in the log was at 00:19:36.
…
[00:01:08] ===== \/ \/ \/ ===== START: Build the main module =====
[00:01:08] make -C ./configure install
[00:01:09] make[1]: Entering directory 'C:/projects/epics-base/configure'
[00:01:09] perl -CSD ../src/tools/makeMakefile.pl O.windows-x64-static ../..
[00:01:09] make[1]: Leaving directory 'C:/projects/epics-base/configure'
[00:01:09] make[1]: Entering directory 'C:/projects/epics-base/configure'
[00:01:09] perl -CSD -MExtUtils::Command -e mkpath O.Common
[00:01:09] make[1]: Leaving directory 'C:/projects/epics-base/configure'
[00:01:09] make -C O.windows-x64-static -f ../Makefile TOP=../.. \
[00:01:09] T_A=windows-x64-static install
[00:01:12] make[2]: Entering directory 'C:/projects/epics-base/configure/O.windows-x64-static'
[00:01:12] cl -nologo -C -E -nologo -FC -D__STDC__=0 -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -I. -I../O.Common -I. -I. -I.. -I../../include/compiler/msvc -I../../include/os/WIN32
-I../../include ../toolchain.c > TOOLCHAIN.windows-x64-static.windows-x64-static
[00:01:12] toolchain.c
[00:01:12] make[2]: Leaving directory 'C:/projects/epics-base/configure/O.windows-x64-static'
[00:01:12] make[2]: Entering directory 'C:/projects/epics-base/configure/O.windows-x64-static'
[00:01:12] Installing CFG file ../../cfg/TOOLCHAIN.windows-x64-static.windows-x64-static
[00:01:12] mkdir ../../cfg
[00:01:12] make[2]: Leaving directory 'C:/projects/epics-base/configure/O.windows-x64-static'
…
[00:19:36] make[4]: Entering directory 'C:/projects/epics-base/modules/pva2pva/pdbApp/O.windows-x64-static'
[00:19:36] cl -EHsc -GR -DUSE_TYPED_RSET -DQSRV_API_BUILDING -I../../common -I../../p2pApp -DEPICS_BASE="\"C:/projects/epics-base\"" -DNOMINMAX -nologo -FC -D__STDC__=0 -D_CRT_SECURE_NO_DEPRECATE
-D_CRT_NONSTDC_NO_DEPRECATE -Ox -Oy- -W3 -w44355 -w44344 -w44251 -D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING -MT -TP -I. -I../O.Common -I. -I. -I.. -IC:/projects/epics-base/include/compiler/msvc -IC:/projects/epics-base/include/os/WIN32 -IC:/projects/epics-base/include
-IC:/projects/epics-base/include/compiler/msvc -IC:/projects/epics-base/include/os/WIN32 -IC:/projects/epics-base/include -c ../softMain.cpp
[00:19:36] softMain.cpp
[00:19:36] make[4]: Leaving directory 'C:/projects/epics-base/modules/pva2pva/pdbApp/O.windows-x64-static'
[00:19:36] make[4]: Entering directory 'C:/projects/epics-base/modules/pva2pva/pdbApp/O.windows-x64-static'
[00:19:36] cl -EHsc -GR -DUSE_TYPED_RSET -DQSRV_API_BUILDING -I../../common -I../../p2pApp -DEPICS_BASE="\"C:/projects/epics-base\"" -DNOMINMAX -nologo -FC -D__STDC__=0 -D_CRT_SECURE_NO_DEPRECATE
-D_CRT_NONSTDC_NO_DEPRECATE -Ox -Oy- -W3 -w44355 -w44344 -w44251 -D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING -MT -TP -I. -I../O.Common -I. -I. -I.. -IC:/projects/epics-base/include/compiler/msvc -IC:/projects/epics-base/include/os/WIN32 -IC:/projects/epics-base/include
-IC:/projects/epics-base/include/compiler/msvc -IC:/projects/epics-base/include/os/WIN32 -IC:/projects/epics-base/include -c softIocPVA_registerRecordDeviceDriver.cpp
[00:19:36] softIocPVA_registerRecordDeviceDriver.cpp
[00:19:36] make[4]: Leaving directory 'C:/projects/epics-base/modules/pva2pva/pdbApp/O.windows-x64-static'
Note that our CI log-files don’t include any command(s) which hang, and even making the build verbose probably won’t fix that. We give GnuMake the flags
--j<n> --output-sync=target which only logs the commands it executed and their output
after they are done, either successfully or by failing. Another build follows the above line with this linker command to create the qsrv.lib so that might be the cause here:
[00:19:58] make[4]: Entering directory 'C:/projects/epics-base/modules/pva2pva/pdbApp/O.windows-x64-static'
[00:19:58] lib -nologo -verbose -out:qsrv.lib pvif.obj qsrv.obj pdb.obj pdbsingle.obj demo.obj imagedemo.obj pdbgroup.obj configparse.obj dbf_copy.obj tpool.obj pvalink.obj pvalink_lset.obj
pvalink_jlif.obj pvalink_link.obj pvalink_channel.obj
[00:19:58] Appending pvalink_channel.obj
[00:19:58] Appending pvalink_link.obj
[00:19:58] Appending pvalink_jlif.obj
[00:19:58] Appending pvalink_lset.obj
[00:19:58] Appending pvalink.obj
[00:19:58] Appending tpool.obj
[00:19:58] Appending dbf_copy.obj
[00:19:58] Appending configparse.obj
[00:19:58] Appending pdbgroup.obj
[00:19:58] Appending imagedemo.obj
[00:19:58] Appending demo.obj
[00:19:58] Appending pdbsingle.obj
[00:19:58] Appending pdb.obj
[00:19:58] Appending qsrv.obj
[00:19:58] Appending pvif.obj
[00:19:58] make[4]: Leaving directory 'C:/projects/epics-base/modules/pva2pva/pdbApp/O.windows-x64-static'
[00:19:58] make[4]: Entering directory 'C:/projects/epics-base/modules/pva2pva/pdbApp/O.windows-x64-static'
[00:19:58] Installing library C:/projects/epics-base/lib/windows-x64-static/qsrv.lib
[00:19:58] make[4]: Leaving directory 'C:/projects/epics-base/modules/pva2pva/pdbApp/O.windows-x64-static'
Just another $0.02 with further evidence.
- Andrew
Complexity comes for free, Simplicity you have to work for.
On 1/8/25, 2:00 AM, "Core-talk" <core-talk-bounces at aps.anl.gov> wrote:
Dear colleagues,
Working on a feature branch for ci-scripts, I am puzzled about failing AppVeyor builds, e.g.
where a large number of VisualStudio builds are consistently getting stuck in Base at or after compiling 'toolchain.c'.
I repeated the build a few times - this is a hard issue.
Does anyone have an idea what could be going on?