Experimental Physics and
Industrial Control System

Mark Rivers <[email protected]> · Thu, 28 Dec 2017 15:39:28 +0000

> Am I missing anything?

Many unit test failures.

> -----Original Message-----
> From: Michael Davidsaver [mailto:[email protected]]
> Sent: Thursday, December 28, 2017 9:38 AM
> To: Mark Rivers; 'Andrew Johnson'; '[email protected]'
> Subject: Re: Problem building example application on windows-x64
> 
> On 12/28/2017 05:53 AM, Mark Rivers wrote:
> >
> >> However, in this particular case building with c++11 should avoid a crash here as
> initialization
> >> order of local static variables is well defined in new c++ versions.
> >
> > Are you suggesting to require c++11 for base 7.0.1.1?  That would mean not supporting
> VS2010.
> 
> No.  I was suggesting a possible workaround so that you could move forward.
> 
> > I  was in fact trying to see if the problem would go away with VS2015 when I ran into
> the problem I reported yesterday with VS2015 failing because of the error building
> Com.res.
> 
> So much for that.  I manged to miss this issue.  Time to open some tickets I think.
> So far I count four distinct symptoms, and the test failures.
> 
> What I see so far:
> 
> 1. Global ctor ordering issues needs to be opened against pvDataCPP and pvAccessCPP
>    (any probably others).
> 
> 2. The osiSockAttach() issue against pvAccessCPP.
> 
> 3. The Com.res failure against Base (on launchpad)
> 
> 4. The timestamp issue w/ genVersionHeader.pl against Base
> 
> 
> Am I missing anything?
> 
> 
> > Mark
> >
> >
> > ________________________________
> > From: Michael Davidsaver <[email protected]>
> > Sent: Wednesday, December 27, 2017 6:02 PM
> > To: Mark Rivers; 'Andrew Johnson'; '[email protected]'
> > Subject: Re: Problem building example application on windows-x64
> >
> > Looks like more global constructor order fun...  This has been a persistent problem
> > due to an annoying "optimization" of storing pointers to factories in global variables.
> >
> > eg. from clientContextImpl.cpp
> >
> >> PVDataCreatePtr BaseRequestImpl::pvDataCreate = getPVDataCreate();
> >
> > or responseHandlers.cpp
> >
> >> static PVDataCreatePtr pvDataCreate = getPVDataCreate();
> >
> > or serializationHelper.cpp
> >
> >> PVDataCreatePtr SerializationHelper::_pvDataCreate(getPVDataCreate());
> >
> > and that's just in pvAccessCPP.  If any of these get ordered before the initialization in
> pvDataCPP,
> > then bang...
> >
> > The DLL loading process probably ensures the necessary ordering, but not so with static
> linking.
> >
> > Fixing this generally is time consuming and tricky as I don't know of a way to detect
> potential
> > issues before a crash occurs.
> >
> > However, in this particular case building with c++11 should avoid a crash here as
> initialization
> > order of local static variables is well defined in new c++ versions.
> >
> >
> > On 12/27/2017 04:29 PM, Mark Rivers wrote:
> >> I rebuilt the windows-x64-static architecture with HOST_OPT=NO so I could get a
> stack trace in the example application.  Here it is:
> >>
> >>
> >>
> >>>          test.exe!epicsMutex::lock()  Line 273 + 0x5 bytes     C++
> >>
> >>            test.exe!epics::pvData::Lock::Lock(epicsMutex & m={...})  Line 45 + 0x46 bytes
> C++
> >>
> >>            test.exe!epics::pvData::FieldCreate::getFieldCreate()  Line 1459 + 0x12 bytes
> C++
> >>
> >>            test.exe!epics::pvData::getFieldCreate()  Line 1230  C++
> >>
> >>            test.exe!epics::pvData::PVDataCreate::PVDataCreate()  Line 358 + 0x23 bytes
> C++
> >>
> >>            test.exe!epics::pvData::PVDataCreate::getPVDataCreate()  Line 617 + 0x21
> bytes            C++
> >>
> >>            test.exe!epics::pvData::getPVDataCreate()  Line 1656          C++
> >>
> >>            test.exe!epics::pvAccess::`dynamic initializer for 'pvDataCreate''()  Line 58 + 0x1a
> bytes   C++
> >>
> >>            test.exe!_initterm(void (void)* * pfbegin=0x000000013f6ada00, void (void)* *
> pfend=0x000000013f6adf48)  Line 873        C
> >>
> >>            test.exe!_cinit(int initFloatingPrecision=1)  Line 301            C
> >>
> >>            test.exe!__tmainCRTStartup()  Line 262 + 0xa bytes            C
> >>
> >>            test.exe!mainCRTStartup()  Line 189 C
> >>
> >>            kernel32.dll!00000000775259cd()
> >>
> >>             [Frames below may be incorrect and/or missing, no symbols loaded for
> kernel32.dll]
> >>
> >>            ntdll.dll!000000007775a561()            ''
> >>
> >>
> >>
> >> In epicsMutex::lock this=0.
> >>
> >>
> >>
> >> That is called from this code in lock.h
> >>
> >>     explicit Lock(Mutex &m)
> >>
> >>     : mutexPtr(m), locked(true)
> >>
> >>     { mutexPtr.lock();}
> >>
> >>
> >>
> >> In that code the debugger tells me that "m id=???", Error CXX0030 Expression cannot
> be evaluated.
> >>
> >>
> >>
> >> This is Visual Studio 2010.
> >>
> >>
> >>
> >> Mark
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> *From:*Mark Rivers
> >> *Sent:* Wednesday, December 27, 2017 3:15 PM
> >> *To:* 'Andrew Johnson'; [email protected]
> >> *Subject:* RE: Problem building example application on windows-x64
> >>
> >>
> >>
> >> OK, that patch let me get the example application to compile.
> >>
> >>
> >>
> >> However, there are problems when running it.  The behavior I see is different on
> windows-x64 and windows-x64-static.
> >>
> >>
> >>
> >> - On windows-x64 the IOC application runs OK but when I try to exit the application I
> get the error I reported in my first message today, i.e. epicsSocketDestroy errors.  ^C will
> not kill the IOC application, I need to use the Task Manager.  This is a serious problem, and
> as I said earlier it affects other IOCs such as simDetector.
> >>
> >>
> >>
> >> J:\epics\devel\example\iocBoot\ioctest>..\..\bin\windows-x64\test.exe st.cmd
> >>
> >> #!../../bin/windows-x64-static/test
> >>
> >> < envPaths
> >>
> >> epicsEnvSet("IOC","ioctest")
> >>
> >> epicsEnvSet("TOP","J:/epics/devel/example")
> >>
> >> epicsEnvSet("EPICS_BASE","H:/epics-devel/base-7.0.1")
> >>
> >> cd "J:/epics/devel/example"
> >>
> >> ## Register all support components
> >>
> >> dbLoadDatabase "dbd/test.dbd"
> >>
> >> test_registerRecordDeviceDriver pdbbase
> >>
> >> ## Load record instances
> >>
> >> dbLoadTemplate "db/user.substitutions"
> >>
> >> dbLoadRecords "db/testVersion.db", "user=rivers"
> >>
> >> dbLoadRecords "db/dbSubExample.db", "user=rivers"
> >>
> >> #var mySubDebug 1
> >>
> >> #traceIocInit
> >>
> >> cd "J:/epics/devel/example/iocBoot/ioctest"
> >>
> >> iocInit
> >>
> >> Starting iocInit
> >>
> >>
> ##########################################################################
> ##
> >>
> >> ## EPICS R7.0.1.1
> >>
> >> ## EPICS Base built Dec 16 2017
> >>
> >>
> ##########################################################################
> ##
> >>
> >> iocRun: All initialization complete
> >>
> >> 2017-12-27T14:52:54.595 No broadcast addresses found or specified - empty address
> list!
> >>
> >> ## Start any sequence programs
> >>
> >> #seq sncExample, "user=rivers"
> >>
> >> epics>
> >>
> >> epics>
> >>
> >> epics>
> >>
> >> epics>
> >>
> >> epics> exit
> >>
> >> epicsSocketDestroy: failed to close a socket because "WINSOCK Error 10093"
> >>
> >> 2017-12-27T14:53:16.250 Receive thread for UDP socket 0.0.0.0:0 has not exited.
> >>
> >> epicsSocketDestroy: failed to close a socket because "WINSOCK Error 10093"
> >>
> >> 2017-12-27T14:53:21.296 Receive thread for UDP socket 164.54.160.31:5076 has not
> exited.
> >>
> >> epicsSocketDestroy: failed to close a socket because "WINSOCK Error 10093"
> >>
> >> 2017-12-27T14:53:26.333 Receive thread for UDP socket 0.0.0.0:5076 has not exited.
> >>
> >> epicsSocketDestroy: failed to close a socket because "WINSOCK Error 10093"
> >>
> >>
> >>
> >>
> >>
> >> On windows-x64-static the IOC crashes immediately, without even starting to execute
> the st.cmd file:
> >>
> >>
> >>
> >> J:\epics\devel\example>make -sj clean uninstall
> >>
> >>
> >>
> >> J:\epics\devel\example>make -sj
> >>
> >> devXxxSoft.c
> >>
> >> xxxRecord.c
> >>
> >> testHello.c
> >>
> >> dbSubExample.c
> >>
> >> devtestVersion.c
> >>
> >> initTrace.c
> >>
> >> testMain.cpp
> >>
> >> test_registerRecordDeviceDriver.cpp
> >>
> >> ../devtestVersion.c(27) : warning C4267: '=' : conversion from 'size_t' to 'epicsUInt32',
> possible loss of data
> >>
> >> ../xxxRecord.c(211) : warning C4244: '=' : conversion from 'epicsFloat64' to 'float',
> possible loss of data
> >>
> >> ../xxxRecord.c(211) : warning C4244: '=' : conversion from 'epicsFloat64' to 'float',
> possible loss of data
> >>
> >> Appending initTrace.obj
> >>
> >> Appending testHello.obj
> >>
> >> Appending devtestVersion.obj
> >>
> >> Appending dbSubExample.obj
> >>
> >> Appending devXxxSoft.obj
> >>
> >> Appending xxxRecord.obj
> >>
> >>
> >>
> >> J:\epics\devel\example>cd iocBoot\ioctest
> >>
> >>
> >>
> >> J:\epics\devel\example\iocBoot\ioctest>..\..\bin\windows-x64-static\test.exe st.cmd
> >>
> >>
> >>
> >> At that point Windows pops up a dialog box telling me that test.exe has crashed.
> Opening the Visual Studio debugger tells me that it is an access violation on address 0.
> >>
> >>
> >>
> >> Is there a test in EPICS base that instantiates the example IOC?
> >>
> >>
> >>
> >> Mark
> >>
> >>
> >>
> >>
> >>
> >>> -----Original Message-----
> >>
> >>> From: [email protected] <mailto:[email protected]>
> [mailto:[email protected]] On Behalf
> >>
> >>> Of Andrew Johnson
> >>
> >>> Sent: Wednesday, December 27, 2017 1:04 PM
> >>
> >>> To: [email protected] <mailto:[email protected]>
> >>
> >>> Subject: Re: Problem building example application on windows-x64
> >>
> >>>
> >>
> >>> Hi Mark,
> >>
> >>>
> >>
> >>> On 12/27/2017 12:05 PM, Mark Rivers wrote:
> >>
> >>>> I just did this again, but running makeBaseApp.pl with Windows rather
> >>
> >>>> than Linux.  I get the same problem. It looks like the problem is that
> >>
> >>>> testVersion.h is changing because the timestamp is incrementing by 1
> >>
> >>>> second between the time it is created and checked?
> >>
> >>>
> >>
> >>> I agree; it seems to be a problem with the genVersionHeader.pl script
> >>
> >>> that Michael added in 3.16.1 and the dependency rules. This is the
> >>
> >>> relevant part of your build log:
> >>
> >>>
> >>
> >>>> perl -CSD H:/epics-devel/base-7.0.1/bin/windows-x64/genVersionHeader.pl
> >>
> >>>> -t ../../.. -N testVERSION -V "" ../O.Common/testVersion.h
> >>
> >>>>
> >>
> >>>> Updating VCS header ../O.Common/testVersion.h
> >>
> >>>>     testVERSION = "2017-12-27T11:39:33"
> >>
> >>>>
> >>
> >>>> perl -CSD H:/epics-devel/base-7.0.1/bin/windows-x64/mkmf.pl  -m
> >>
> >>>> devtestVersion.d -I. -I../O.Common -I. -I. -I..
> >>
> >>>> -I../../../include/compiler/msvc -I../../../include/os/WIN32
> >>
> >>>> -I../../../include -IH:/epics-devel/base-7.0.1/include/compiler/msvc
> >>
> >>>> -IH:/epics-devel/base-7.0.1/include/os/WIN32
> >>
> >>>> -IH:/epics-devel/base-7.0.1/include        devtestVersion.obj
> >>
> >>>> ../devtestVersion.c
> >>
> >>>>
> >>
> >>>> perl -CSD H:/epics-devel/base-7.0.1/bin/windows-x64/genVersionHeader.pl
> >>
> >>>> -t ../../.. -N testVERSION -V "" ../O.Common/testVersion.h
> >>
> >>>>
> >>
> >>>> Updating VCS header ../O.Common/testVersion.h
> >>
> >>>>     testVERSION = "2017-12-27T11:39:34"
> >>
> >>>>
> >>
> >>>> perl -CSD H:/epics-devel/base-7.0.1/bin/windows-x64/mkmf.pl  -m
> >>
> >>>> devtestVersion.d -I. -I../O.Common -I. -I. -I..
> >>
> >>>> -I../../../include/compiler/msvc -I../../../include/os/WIN32
> >>
> >>>> -I../../../include -IH:/epics-devel/base-7.0.1/include/compiler/msvc
> >>
> >>>> -IH:/epics-devel/base-7.0.1/include/os/WIN32
> >>
> >>>> -IH:/epics-devel/base-7.0.1/include        devtestVersion.obj
> >>
> >>>> ../devtestVersion.c
> >>
> >>>>
> >>
> >>>> perl -CSD H:/epics-devel/base-7.0.1/bin/windows-x64/genVersionHeader.pl
> >>
> >>>> -t ../../.. -N testVERSION -V "" ../O.Common/testVersion.h
> >>
> >>>>
> >>
> >>>> Updating VCS header ../O.Common/testVersion.h
> >>
> >>>>     testVERSION = "2017-12-27T11:39:35"
> >>
> >>>
> >>
> >>> whereas on Linux I get:
> >>
> >>>
> >>
> >>>> perl -CSD /home/phoebus/ANJ/epics/base/7.0/bin/linux-x86_64/genVersionHeader.pl
> >>
> >>>>  -t ../../.. -N testVERSION -V "" ../O.Common/testVersion.h
> >>
> >>>>
> >>
> >>>> Updating VCS header ../O.Common/testVersion.h
> >>
> >>>>     testVERSION = "2017-12-27T12:08:23-0600"
> >>
> >>>>
> >>
> >>>> /usr/bin/gcc  -D_GNU_SOURCE -D_DEFAULT_SOURCE         -
> >>
> >>> DUSE_TYPED_RSET
> >>
> >>>> -D_X86_64_  -DUNIX  -Dlinux     -O3 -g   -Wall      -mtune=generic     -m64
> >>
> >>>> -Wpointer-arith -fvisibility=hidden -fPIC -I. -I../O.Common -I. -I. -I..
> >>
> >>>> -I../../../include/compiler/gcc -I../../../include/os/Linux
> >>
> >>> -I../../../include
> >>
> >>>> -I/home/phoebus/ANJ/epics/base/7.0/include/compiler/gcc
> >>
> >>>> -I/home/phoebus/ANJ/epics/base/7.0/include/os/Linux
> >>
> >>>> -I/home/phoebus/ANJ/epics/base/7.0/include        -MM -MF devtestVersion.d
> >>
> >>>> ../devtestVersion.c
> >>
> >>>>
> >>
> >>>> perl -CSD /home/phoebus/ANJ/epics/base/7.0/bin/linux-x86_64/genVersionHeader.pl
> >>
> >>>>  -t ../../.. -N testVERSION -V "" ../O.Common/testVersion.h
> >>
> >>>>
> >>
> >>>> Keeping VCS header ../O.Common/testVersion.h
> >>
> >>>>     testVERSION = "2017-12-27T12:08:23-0600"
> >>
> >>>
> >>
> >>> There seems to be a mismatch between the assumptions that script makes
> >>
> >>> about file-system modification times and how GNUmake works on Windows.
> >>
> >>> The old DOS FAT file-system had a time-stamp granularity of 2 seconds
> >>
> >>> and that has resulted in some workarounds being added to GNUmake on
> >>
> >>> Windows, e.g. see http://www.delorie.com/djgpp/v2faq/faq22_18.html or
> >>
> >>> there may be a delay added at the end of the phase-1 run before GNUmake
> >>
> >>> re-reads the Makefiles - that might explain what's happening here.
> >>
> >>>
> >>
> >>> Michael: How about we drop the seconds field completely from the build
> >>
> >>> date/time-stamp? That should prevent this kind of problem from
> >>
> >>> recurring. It would still trigger a re-creation of the header whenever
> >>
> >>> there is a roll-over, but that should only happen once, not continuously
> >>
> >>> like Mark is seeing.
> >>
> >>>
> >>
> >>> Mark: If you want to try out that fix, edit
> >>
> >>> base-7.0/src/tools/genVersionHeader.pl and make this change:
> >>
> >>>
> >>
> >>>> diff --git a/src/tools/genVersionHeader.pl b/src/tools/genVersionHeader.pl
> >>
> >>>> index 5851ea8..8e50fec 100644
> >>
> >>>> --- a/src/tools/genVersionHeader.pl
> >>
> >>>> +++ b/src/tools/genVersionHeader.pl
> >>
> >>>> @@ -19,7 +19,7 @@ use POSIX qw(strftime);
> >>
> >>>>  use strict;
> >>
> >>>>
> >>
> >>>>  # RFC 8601 date+time w/ zone (eg "2014-08-29T09:42:47-0700")
> >>
> >>>> -my $tfmt = '%Y-%m-%dT%H:%M:%S';
> >>
> >>>> +my $tfmt = '%Y-%m-%dT%H:%M';
> >>
> >>>>  $tfmt .= '%z' unless $^O eq 'MSWin32'; # %z returns zone name on Windows
> >>
> >>>>  my $now = strftime($tfmt, localtime);
> >>
> >>>>
> >>
> >>>
> >>
> >>> - Andrew
> >>
> >>>
> >>
> >>> --
> >>
> >>> Arguing for surveillance because you have nothing to hide is no
> >>
> >>> different than making the claim, "I don't care about freedom of
> >>
> >>> speech because I have nothing to say." -- Edward Snowdon
> >>
> >

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System