FW: Strange channel access problem

["Hammonds, John P." <jphammonds@anl.gov>]

Sorry I typed the address for tech-talk wrong in my original send.

John Hammonds
Beamline and Data Acquisition Group
APS Engineering Support Division
Argonne National Laboratory
JPHammonds@anl.gov
(630)252-5317


-----Original Message-----
From: Hammonds, John P. 
Sent: Friday, April 20, 2007 11:01 AM
To: Jeff Hill
Cc: tech-talk@apns.anl.gov
Subject: RE: Strange channel access problem

Yes, this was R3.13.7.  Adding IOCs to hosts file definitely fixed the
problem that we were having.  Note that I was just trying to confirm
Mark's remembrance of problems with no DNS.  This was a real problem for
us but was easily fixed when we knew what was going on.  3.14 probably
would solve this.

John Hammonds
Beamline and Data Acquisition Group
APS Engineering Support Division
Argonne National Laboratory
JPHammonds@anl.gov
(630)252-5317


-----Original Message-----
From: Jeff Hill [mailto:johill@lanl.gov] 
Sent: Friday, April 20, 2007 9:49 AM
To: Hammonds, John P.
Subject: RE: Strange channel access problem


> We did have problems with caget (the C version) as well as MEDM not
> working on Linux unless the IOC could be found in DNS.  Since we were
> using Linux, we just added IOCs to /etc/hosts.

Was this R3.14 or R3.13 that caget and medm were built with? Probably
R3.13
if it was 2002?

Jeff

> -----Original Message-----
> From: Hammonds, John P. [mailto:jphammonds@anl.gov]
> Sent: Thursday, April 19, 2007 7:14 PM
> To: Rivers, Mark L.; tech-talk@aps.anl.gov
> Subject: Re: Strange channel access problem
> 
> We did have problems with caget (the C version) as well as MEDM not
> working on Linux unless the IOC could be found in DNS.  Since we were
> using Linux, we just added IOCs to /etc/hosts.
> See tech-talk messges between Jeff and Garrett Rinehart about Dec 5,
2002.
> 
> John Hammonds
> 
> ----- Original Message -----
> From: Mark Rivers <rivers@cars.uchicago.edu>
> To: Jeff Hill <johill@lanl.gov>; tech-talk@aps.anl.gov <tech-
> talk@aps.anl.gov>; beamline_controls@aps.anl.gov
> <beamline_controls@aps.anl.gov>
> Sent: Thu Apr 19 19:44:11 2007
> Subject: RE: Strange channel access problem
> 
> Jeff,
> 
> Thanks for the reply.
> 
> cygwin is not involved, this was all built for native windows using
the
> Microsoft compiler.  The dlls and the caget binary were all built in
> September-October 2001.
> 
> I am actually not sure which version of caget this is, if it is the
one
> from base, or the ezca extension, or another extension.  It was built
in
> late 2001.
> 
> Kay Kaemir sent the following to me, though not to tech-talk:
> Hi Mark:
> 
> I have of course no real clue here, but a vague guess would be that
this
> involves differences in the name server setup.
> Even though CA doesn't really need a name server, it uses one just for
> the purpose of giving nice error messages.
> Depending on the CA version, that works great or less well in case of
> name server access problems.
> 
> -Kay
> 
> I suspect he may be right.  I have a dim recollection of things not
> working when our IOCs were not in DNS.  The Australians are tracking
> down this possibility as I write.
> 
> Mark
> 
> 
> 
> > -----Original Message-----
> > From: Jeff Hill [mailto:johill@lanl.gov]
> > Sent: Thursday, April 19, 2007 7:33 PM
> > To: Mark Rivers; tech-talk@aps.anl.gov;
beamline_controls@aps.anl.gov
> > Subject: RE: Strange channel access problem
> >
> >
> > Mark,
> >
> > > - ezca doesn't connect
> > > - The command line "caget" program works OK (except see note
below).
> >
> > Which caget are you using? Is it the caget based only CA or
> > the caget based
> > on ezca?
> >
> > > Sometimes, but not always, they get the following error with
caget:
> > >
> > > C:\RSI\IDL63\misc_dlls>caget epics:calcExample
> > > Unexpected UDP failure WIN32 Socket Library Error 10054
> >
> > Looking in the R3.13 source code one could make the guess
> > that _WIN32 isn't
> > being defined by the compiler, or by the compiler's command line
(this
> > snippet is from the R3.13 iocinf.c) when ca.dll was built? The
> > SOCK_ECONNRESET constant is 10054 when building CA for winsock.
> >
> > #               ifdef _WIN32
> > 			        /*
> > 			         * Avoid ECONNRESET from
> > disconnected socket
> > bug
> > 			         * in win32
> > 			         */
> > 			        if (SOCKERRNO==SOCK_ECONNRESET) {
> > 				        UNLOCK;
> >        				        return;
> > 			        }
> > #               endif
> > 		ca_printf("Unexpected UDP failure %s\n", SOCKERRSTR);
> >
> >
> > Considering this further I seem to recall that you typically
> > build using
> > cygwin POSIX emulation on windows instead of using the native
> > port, and
> > therefore naturally _win32 wouldn't be defined. What version
> > of cygwin is
> > installed at the Australian Synchrotron? I remember that
> > earlier version of
> > cygwin were quite squirrelly. These versions might predate
> > the installations
> > on the Australian Synchrotron. Nevertheless, based on your
> > "Unexpected UDP
> > failure WIN32 Socket Library Error 10054" symptom , this
> > still seems like a
> > best guess.
> >
> > As for why EZCA doesn't connect and "caget" does that's
> > harder to guess at.
> > Till's suggestion is definitely a possibility, but maybe that
doesn't
> > explain why "caget" connects and ezca doesn't if you are
> > using the "caget"
> > that comes with ezca, and is based on ezca. Another related
> > possibility is
> > that the code is using a different cygwin dll than it was built to
be
> > executed with.
> >
> > > Another thing that they have seen is that caget to local
> > PVs on the PC
> > > return instantly, while caget to PVs on the network do not
> > return until
> > > after a 4 second delay.
> >
> > That 4 seconds is an oddly round number. I wonder if
> > something is squirrelly
> > with cygwin's "select" system call based notification for the
> > arrival of new
> > incoming network messages.
> >
> > Another possibility is something funny with the local
> > network. You might try
> > ping to the IOC. Also, looking at icmp traffic with a sniffer
> > can be helpful
> > when there are network issues.
> >
> > You might also try using R3.14
> >
> > Jeff
> >
> > > -----Original Message-----
> > > From: Mark Rivers [mailto:rivers@cars.uchicago.edu]
> > > Sent: Thursday, April 19, 2007 1:46 PM
> > > To: tech-talk@aps.anl.gov; beamline_controls@aps.anl.gov
> > > Subject: Strange channel access problem
> > >
> > > Folks,
> > >
> > > I am trying to help diagnose a very puzzling channel access
> > problem that
> > > is happening on Windows machines from IDL at the Australian
> > Synchrotron.
> > >
> > > Background:
> > >
> > > IDL uses the following DLLs to do channel access:
> > >
> > >    - ezcaIDL.dll  A thin wrapper on top of ezca.dll and
> > ezcaScan.dll to
> > > convert IDL call_external argument passing parameters to
> > those used by
> > > ezca and ezcaScan.
> > >
> > >    - ezca.dll and ezcaScan.dll.  These libraries on top of
> > ca.dll and
> > > com.dll that take care of looking up PVs by name, handling
> > callbacks,
> > > etc.
> > >
> > >    - ca.dll, com.dll.  The standard CA libraries from EPICS base.
> > >
> > > I built these DLLs in 2001 with EPICS 3.13.  I've used the same
DLLs
> > > since then with all versions of IDL (up to 6.3, the current
> > version) and
> > > with all versions of Windows (NT, 2000, XP).  I've never had any
> > > problems, and it is probably running on over 100 machines at APS
and
> > > NSLS, and other sites.
> > >
> > > At the Australian Synchrotron, however, they have run into a
strange
> > > problem.  They are using the exact same DLLs that we are,
> > standard XP
> > > Professional SP1, and IDL 6.3.  It has the following behavior:
> > >
> > > - ezca.dll is being called OK, because functions like
> > caGetTimeout and
> > > caGetRetryCount that return values from ezca are working fine.
> > >
> > > - EPICS channel access is always failing to find PVs,
> > returning failure
> > > after the expected timeout interval.
> > >
> > >
> > > This failure happens despite the following:
> > >
> > > - The command line "caget" program works OK (except see note
below).
> > >
> > > - All Windows firewalls are turned off.
> > >
> > > - The failure happens for PVs on the network, as well as
> > for PVs on the
> > > same PC running a soft IOC.
> > >
> > > There is some behavior on the "caget" command line program that
> > > indicates something is not quite right:
> > >
> > > Sometimes, but not always, they get the following error with
caget:
> > >
> > > C:\RSI\IDL63\misc_dlls>caget epics:calcExample
> > > Unexpected UDP failure WIN32 Socket Library Error 10054
> > > epics:calcExample             9
> > >
> > > Even when the UDP failure message appears caget still "works"
> > >
> > > Sometimes this error does not occur:
> > >
> > > C:\RSI\IDL63\misc_dlls>caget epics:calcExample
> > > epics:calcExample             0
> > >
> > > Another thing that they have seen is that caget to local
> > PVs on the PC
> > > return instantly, while caget to PVs on the network do not
> > return until
> > > after a 4 second delay.
> > >
> > > In principle we should be able to turn on the rich debugging in
ezca
> > > (Trace and Debug flags).  However, that debugging output
> > goes to stdout,
> > > which does not exist in the Windows version of IDL, because it is
a
> > > "Windowing" application, not a console application, and hence has
no
> > > place for stdout to go.  We could rewrite ezca to send the
> > output to a
> > > file, but that is more work than I'm in the mood for right now :-)
> > >
> > > Anyone have any idea what could be wrong?
> > >
> > > Thanks,
> > > Mark
> >
> >