Re: Build failed in Jenkins: epics-base-3.16-linux32-test #13

[Michael Davidsaver <mdavidsaver@gmail.com>]

On 02/15/2016 07:07 PM, Andrew Johnson wrote:
> On 02/15/2016 05:33 PM, Michael Davidsaver wrote:
>> On 02/12/2016 12:33 PM, Andrew Johnson wrote:
>>> On 02/12/2016 11:06 AM, APS Jenkins wrote:
>>>> src/std/rec/test/O.linux-x86/arrayOpTest.tap ............. ok
>>>> src/std/rec/test/O.linux-x86/asTest.tap .................. ok
>>>> Bailout called.  Further testing stopped:  testMonitorWait() exceeds timeout
>>>> FAILED--Further testing stopped: testMonitorWait() exceeds timeout
>>> My fix was just to re-run the tests; given that this failure has
>>> happened before on a different architecture, I think the 10 second
>>> timeout delay in testMonitorWait() is just too short for a busy build
>>> slave. Since the effect of failure is to kill all further testing I have
>>> bumped it all the way up to 60 seconds.
>> It's happened again.  From the .tap log the timeout is a symptom of a
>> previous failure.  I'm guessing this was the case previously.
> This time it failed on the linux-x86_64 Jenkins master CPU; previously
> it was on the linux-x86 (i.e. 32-bit) build slave, so the two are not
> quite the same.
>
>>> 1..10
>>> ############################################################################
>>> ## EPICS R3.16.0-DEV $$Date$$
>>> ## EPICS Base built Feb 15 2016
>>> ############################################################################
>>> ok  1 - dbGetField("rec:ai", 8) -> 0.000000e+00 == 0.000000e+00
>>> ok  2 - dbGetField("rec:ai.INP", 0) -> "0" == "0"
>>> not ok  3 - dbPutField(rec:link1.PROC, 5, ...) == -1
>>> Bail out! testMonitorWait() exceeded 60 second timeout
>> I haven't yet been able to replicate this locally.
> IIRC previous failures didn't show any failed tests before testAbort()
> was called, so this may be a different problem and not just a case of a
> too-busy CPU.


>  Could there be something racy in the above test?

That "-1" stands out.  So far the only candidate I see is in
dbCaPutLinkCallback() if the target link isn't connected or writable. 
Which would mean that the main thread is racing the dbCa thread to
CA_CONNECT this link.