Re: Problem building seq-2-2-1 on Windows

[Benjamin Franksen <benjamin.franksen@helmholtz-berlin.de>]

On Monday 18 May 2015 17:55:56 Mark Rivers wrote:
> > In particular, $(TOP)/../configure/ could theoretically collide with
> > some other module's configure directory.
>
On Tuesday 19 May 2015 04:01:40 Johnson, Andrew N. wrote:
> Yes, theoretically.  But the only files we look for there are
> EPICS_BASE.$(EPICS_HOST_ARCH) and SUPPORT.$(EPICS_HOST_ARCH) which
> are names we made up for synApps.  So a collision is unlikely.
 > That collision would only occur if a user were to embed (install) the
> sequencer as a subdirectory inside the other module, which IMHO is
> more likely to be the kind of case where they would /want/ to be able
> to inherit settings from the parent module, especially for the path
> to Base itself. The $(TOP)/../configure/ convention is synApps’
> fairly simple approach to reducing the amount of work for its users.
> If this breaks some other site’s installation then they can fix it by
> removing the -include line from the synApps RELEASE file (which they
> have to edit anyway) when they install it; they will have to do that
> for any other synApps modules too since most of them already have
> this line.

You are right that accidental collisions are pretty unlikely. And people
are used to modify configure/RELEASE anyway so indeed they can delete
this line if they want.

> > The tool then generates the configure RELEASE files according to the
> > application's specification
 about which versions of which modules
> > it depends on.
>
> Does it handle Linux and Windows builds in the same file tree?  If so,
> how does it do it?

Uh, no. The idea is that you specify once and for all for each version
of each module: its dependencies, where to get the source, what patches
to apply, etc. This "database" *is* supposed to be shared between build
hosts (with possibly different host archs). Actually compiling whatever
an application needs is automatic (one command, can even be integrated
into the EPICS build system). So, each build host has its own "cache" of
already compiled support modules; trying to share this cache between
build hosts would only complicate matters.

> > > -    Deleted this file.  Using this mechanism requires 4 separate
> >>
> >> RELEASE.$(EPICS_HOST_ARCH) files if one is building for Windows
> >> 32/64, static/dynamic.  The -include is a simpler way to do this.
> >> This is obviously your call on how you would like to do it.
>
> > Similar considerations apply here as for the previous point.
>
> I would really like to get rid of this file, even if you don't put the
> -include in configure/RELEASE.  I always get bit by this, because by
> adding the -include to configure/RELEASE it works on windows-x64, but
> not in win32-x86, because this file takes precedence and it does not
> automatically update.

I am convinced. Will apply.

On Tuesday 19 May 2015 04:01:40 Johnson, Andrew N. wrote:
> >> examples/demo/Makefile
> >>
> >> -    This needed to be changed to build on cygwin-x86.  All symbols
> >> that are imported because of a dbd file (i.e, function(),
> >> registrar(), etc.) must be in a library, they cannot be built
> >> directly into a main program.  Andrew Johnson can explain this
> >> further if you have questions.
> >>
> >> src/seq/Makefile
> >> src/seq_main.c
> >> src/seq_prim_types.c (new file)
> >> src/seq_prim_types.h
> >> src/snc/var_types.c
> >
> > Andrew, if you are reading this, I would be very interested to hear
> > more. Besides, I may have misunderstood this but didn't you write
> > that EPICS will stop supporting cygwin anyway?
>
> I did not write that we were dropping Cygwin,

Sorry.

> Ralph and Michael were
> discussing the possibility a few weeks ago, but I fixed the bug that
> was causing the problem, and the fact that Mark needs it tells me
> that we couldn’t afford to drop it anyway.

Ok.

> The issue is related to name decorations, which are described in
> comments in shareLib.h The ioc_registerRecordDeviceDriver.cpp file
> generated from an ioc.dbd file decorates the names of the external
> symbols read from the DBD file with epicsShareExtern;
> epicsExportSharedSymbols is not defined when these macros are used.
> Thus when compiling this file on Windows with SHARED_LIBRARIES=YES
> the epicsShareExtern macro expands to __declspec(dllimport) extern,
> telling the compiler these symbols will be imported from a DLL.
>
> The matching symbols must also have been decorated with
> epicsShareExtern but with epicsExportSharedSymbols defined, so the
> macro expands to __declspec(dllexport) extern. This is normally
> hidden by the epicsExport.h header and the epicsExportAddress()
> macro. If you try to define those symbols in the same executable as
> the registerRecDevDriver code, the linker will not be able to match
> up the decorated names and the link will fail.
>
> Now the Microsoft linker seems to have been made somewhat more
> forgiving in recent versions such that when it can’t match up the
> symbols it checks the names decorated differently and silently
> accepts incorrect decorations, but in Cygwin (and probably in MinGW
> builds too) the GNU linker doesn’t do that. Thus if you want your
> code to build on Cygwin you have to get your decorations right and
> you can’t mix-and-match between code compiled for a DLL and code
> meant for directly linking into an executable.

I think I understand. Thanks for explaining.

> >> -    The way that prim_types was being handled does not work on
> >> win32-x86, windows-x64, or cygwin-x86.  Following Andrew Johnson's
> >> suggestion I left only the structure definitions in
> >> seq_prim_types.h
> >> (with epicsShareExtern) and moved the value assignments into the
> >> new
> >> seq_prim_types.c file.  This works fine, and also cleans up
> >> seq_main.c and var_types.c.
> >
> > I have picked up this pattern from EPICS base sources where it is
> > used in several places and is documented in shareLib.h. I don't
> > understand why it works in base but not in the sequencer.
>
> Some older versions of Base would not build on Cygwin with
> SHARED_LIBRARIES=YES, we just ignored the errors and said “don’t try
> that”. I fixed the problems in Base the same way Mark is recommending
> here.

The pattern is still present in 3.14.12.5 (epicsTypes.h and
alarm.h/alarmString.h). Does this mean Cygwin with SHARED_LIBRARIES=YES
works only with 3.15.x?

Anyway, I will take a closer look at the code and probably apply this
1:1. Though I find it unfortunate that we no longer have the string
table and the enum definition in the same (header) file because this
made it easier to spot inconsistencies.

To make sure I understood this right: I can still use this pattern
inside a library, just not in the API of one?

> >> On win32-x86-static and windows-x64-static many of the tests
> >> succeeded, but in both cases it hung up in bittypeIoc.t.  I had to
> >> kill them with ^C.
> >>
> >> bittypesIoc.t .......... Starting iocInit
> >> iocRun: All initialization complete
> >> bittypesIoc.t .......... 1/24 ERROR: The process "softIoc.exe" not
> >> found. Terminating on signal SIGINT(2)
> >> Terminating on signal SIGINT(2)
> >> make[2]: *** [runtests.win32-x86-static] Error 130
> >> make[1]: *** [validate.runtests] Error 130
> >> make: *** [test.runtests] Error 130
> >>
> >> bittypesIoc.t .......... Starting iocInit
> >> bittypesIoc.t .......... 1/24 cas warning: Configured TCP port was
> >> unavailable. cas warning: Using dynamically assigned TCP port
> >> 65113,
> >> cas warning: but now two or more servers share the same UDP port.
> >> cas warning: Depending on your IP kernel this server may not be
> >> cas warning: reachable with UDP unicast (a host's IP in
> >> EPICS_CA_ADDR_LIST) iocRun: All initialization complete
> >> Terminating on signal SIGINT(2)
> >
> > This happens often when running the tests immediately after
> > building.
> > Normally it works fine when you run the tests again.
> >
> > I have not yet found what causes these hangups. There must be some
> > subtle interaction between the Perl/system/exec/kill stuff and what
> > the IOC in the background does.
>
> Seems vaguely familiar, but I can’t think why. I’ll let you know if I
> remember anything. Ah, you don’t use ‘exec’ in your test scripts to
> run other programs do you? If you do,

I do indeed: for the IOC that host the database and runs in the
background.

> GNUmake on Windows thinks the
> Perl script has finished executing as soon as the Perl program goes
> away even if it successfully started that other program. My
> makeTestfile.pl script now uses this code to pick the command in the
> .t files that run other programs:
>
>   # Use system on Windows, exec doesn't work the same there and
>   # GNUmake thinks the test has finished as soon as Perl exits.
>   my $exec = $^O eq 'MSWin32' ? "system('./$exe') == 0" : "exec
> './$exe'";

Note that the tests sometimes hang even on Linux. This always happens
for the first test that tries to start a background IOC (bittypesIoc.t).
Usually running them a second time (after Ctrl-C-ing the hanging tests)
succeeds.

I just tested whether using system instead of exec makes a difference on
Linux. Indeed it does, but the effect is the opposite of the intended
one: The tests now consistently hang in bittypesIoc.t, no matter how
often I restart them.

So, for Linux this makes things worse, but perhaps for Windows it makes
things better?

> >> The other 3 architectures (all dynamic) failed to run any tests
> >> because seq.dll was not found when trying to run the executables.
> >> The EPICS build system creates an IOC file called dllPath.bat,
> >> which
> >> can be used to define the PATH to find all of the required DLLs.
> >> This needs to be somehow incorporated into the test harness.
> >
> > OMG.
> >
> > Andrew, do you have any suggestions how to proceed? Is this
> > documented somewhere?
>
> The dllPath.bat file is only created in an iocBoot subdirectory, I
> don’t think it’s going to be the solution here. If these tests start
> the IOC from a Perl script you should be able to adjust $ENV{PATH} in
> Perl beforehand (on Windows targets) to include your own
> $(TOP)/lib/$(ARCH) directory.

This is easily done. Mark, could you test if adding

 $ENV{PATH} = "$top/lib/$host_arch:$ENV{PATH}";

after the line that says

 my $top = Cwd::abs_path($ENV{TOP});

in test/validate/makeTestfile.pl helps?

Cheers
Ben

________________________________

Helmholtz-Zentrum Berlin für Materialien und Energie GmbH

Mitglied der Hermann von Helmholtz-Gemeinschaft Deutscher Forschungszentren e.V.

Aufsichtsrat: Vorsitzender Prof. Dr. Dr. h.c. mult. Joachim Treusch, stv. Vorsitzende Dr. Beatrix Vierkorn-Rudolph
Geschäftsführung: Prof. Dr. Anke Rita Kaysser-Pyzalla, Thomas Frederking

Sitz Berlin, AG Charlottenburg, 89 HRB 5583

Postadresse:
Hahn-Meitner-Platz 1
D-14109 Berlin

http://www.helmholtz-berlin.de