EPICS Home EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

Format page
for printing


Search Tech-talk
2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <2020>  2021  2022  2023  2024  Index 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <2020>  2021  2022  2023  2024 
<== Date ==> <== Thread ==>
Subject: Base CI tests on MacOS failing
From: Ralph Lange via Core-talk <core-talk at aps.anl.gov>
To: EPICS Core Talk <core-talk at aps.anl.gov>
Date: Mon, 27 Apr 2020 15:01:44 +0200
Dear all,

The 3.15 and 7.0 branches are now switched over to use the shared ci-scripts module for their Travis-CI and AppVeyor builds.

On Travis, the MacOS builds fail their tests. The epicsTimerTest fails the 5% error threshold in some of its cases. Typically like this:
epicsTimerTest.tap ........... 1/41
not ok  4 - 5.604376 < 5.000000, delay = 1.100000 s, error = 0.061648 s (5.6 %)
not ok  5 - 12.089890 < 5.000000, delay = 1.200000 s, error = 0.145079 s (12.1 %)
not ok  7 - 9.455878 < 5.000000, delay = 1.400000 s, error = 0.132382 s (9.5 %)
not ok 16 - 5.582973 < 5.000000, delay = 2.300000 s, error = 0.128408 s (5.6 %)
not ok 20 - 5.313693 < 5.000000, delay = 2.700000 s, error = 0.143470 s (5.3 %)
epicsTimerTest.tap ........... Failed 5/41 subtests

This seems to have gotten worse as the shared ci-scripts now run the tests on Travis-CI using parallel make (as on AppVeyor/Windows) - the former scripts were running the tests serially.

How should we proceed? Timing is notoriously weird on CI builder machines - should this test be run with higher thresholds on CI? Or be moved to the performance tests (not run as part of the build at all)?

Cheers,
~Ralph

Replies:
Re: Base CI tests on MacOS failing Torsten Bögershausen via Core-talk
Re: Base CI tests on MacOS failing Michael Davidsaver via Core-talk
Navigate by Date:
Prev: [Bug 1861612] Re: callbackRequestDelay not waiting for 1/60 sec on vxWorks Freddie Akeroyd via Core-talk
Next: Re: Base CI tests on MacOS failing Torsten Bögershausen via Core-talk
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <2020>  2021  2022  2023  2024 
Navigate by Thread:
Prev: Build failed: EPICS Base base-7.0-597 AppVeyor via Core-talk
Next: Re: Base CI tests on MacOS failing Torsten Bögershausen via Core-talk
Index: 2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  <2020>  2021  2022  2023  2024 
ANJ, 27 Apr 2020 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·