Experimental Physics and
Industrial Control System

Michael Davidsaver via Core-talk <core-talk at aps.anl.gov> · Mon, 27 Apr 2020 07:52:52 -0700

On 4/27/20 6:01 AM, Ralph Lange via Core-talk wrote:
> Dear all,
> 
> The 3.15 and 7.0 branches are now switched over to use the shared ci-scripts module for their Travis-CI and AppVeyor builds.
> 
> On Travis, the MacOS builds fail their tests. The epicsTimerTest fails the 5% error threshold in some of its cases. Typically like this:
> 
>     epicsTimerTest.tap ........... 1/41
>     not ok  4 - 5.604376 < 5.000000, delay = 1.100000 s, error = 0.061648 s (5.6 %)
>     not ok  5 - 12.089890 < 5.000000, delay = 1.200000 s, error = 0.145079 s (12.1 %)
>     not ok  7 - 9.455878 < 5.000000, delay = 1.400000 s, error = 0.132382 s (9.5 %)
>     not ok 16 - 5.582973 < 5.000000, delay = 2.300000 s, error = 0.128408 s (5.6 %)
>     not ok 20 - 5.313693 < 5.000000, delay = 2.700000 s, error = 0.143470 s (5.3 %)
>     epicsTimerTest.tap ........... Failed 5/41 subtests
> 
> 
> This seems to have gotten worse as the shared ci-scripts now run the tests on Travis-CI using parallel make (as on AppVeyor/Windows) - the former scripts were running the tests serially.
> 
> How should we proceed? Timing is notoriously weird on CI builder machines - should this test be run with higher thresholds on CI? Or be moved to the performance tests (not run as part of the build at all)?

Back in Feb. I added testImpreciseTiming() to epicsUnitTest.h, and used it
in some of the common (apparently) false positives failures.  Starting with
epicsEventTest (on 7.0) to mark these tests specific TODO on CI runs.

https://github.com/epics-base/epics-base/commit/819b0de65b92d29d2ab26b0a2e36390ba0e17f71

So having the appveyor tests run with EPICS_TEST_IMPRECISE_TIMING=YES is a
first step.

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System