EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  <20012002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  <20012002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: RE: bad UDP messages
From: "Jeff Hill" <[email protected]>
To: <[email protected]>, "Geoff Savage" <[email protected]>
Cc: <[email protected]>
Date: Mon, 19 Nov 2001 11:42:13 -0700
> Here is an error message about bad UDP messages.  We have 
> seen this from two different IOCs over the weekend.

This could be caused by a server that is not CA that might
be listening on UDP port 5064 and responding. Alternatively,
a network with an exceptionally high bit error rate might
damage many bits in a UDP frame until the odds become higher
that the CRC checksum filter will not detect the damage, and 
discard the message. 

> CAC: post_msg(): Corrupt cmd in UDP msg b
> ../iocinf.c: bad UDP msg from d0olctl50.fnal.gov:5064

If this is from an IOC then it probably has Tornado II with DNS
installed because it appears that DNS has been used to 
convert the IP address to a host name. Alternatively they might 
have used the hostAdd() interface to populate a host table, but
this approach is less commonly used.

> 1) This message, as pointed by Andrew, one of our terminal server 
> responds to the CA search UDP
> message at port 5064 and sends a NON-epics reply to the IOC which send 
> out the CA search UDP message.
> Although we have several terminal servers of same type, only one of them 
> causes this symptom.
> 
> 2) Usually, it indicates there is unresolved channel connection on this 
> IOC. So we have check channel
> connection status on this IOC.

That's interesting. We had a terminal server on LEDA that was causing trouble
that was not well understood. This device is currently disconnected from 
our network. Personally, I would be very hesitant to leave any device connected
to an EPICS network that was known to be responding to messages on UDP port 5064.

Alternatively, you could configure CA to use a different port.

> 
> 3) I believe the program output this message has memory leak. So if you 
> leave this message long enough, this would cause low memory situation, 
> according to my experience.
> 

I was looking at this carefully and I don't see a memory leak in this part 
of CA in R3.13. Nevertheless, since both Noboru and Geoff have seen this 
in a host that is low on memory I am concerned.

There are only two UDP response stubs in the CA client library that allocate
memory. 

1) The stub that receives a CA beacon message for an IP address that has not 
been seen before.

2) The stub that receives a CA search response for an IP address that has not 
been seen before.

After careful examination of this part of the code it appears
that a memory leak would only occur if the rogue server was replying to the
CA request with a response message that matched the above two response codes
and also was varying the IP address portion of the message enough that the
resource consumption was noticeable.

Otherwise, the EPICS routine epicsVprintf() and the vxWorks routine 
hostGetByAddr() are also used in this part of the code. Since these
routines are used quite frequently perhaps it less likely that they would
have a memory leak that has not been detected. Nevertheless, use of DNS in
the vxWorks environment is new in Tornado II so it might be interesting
to build a system without DNS and see if the behavior changes.

In what versions of EPICS R3.13 were these problems observed?

Jeff



References:
Re: bad UDP messages Noboru Yamamoto

Navigate by Date:
Prev: channel access john sinclair
Next: RE: Error message from sequencer Jeff Hill
Index: 1994  1995  1996  1997  1998  1999  2000  <20012002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: bad UDP messages Noboru Yamamoto
Next: RE: VxWorks on Linux Mark Rivers
Index: 1994  1995  1996  1997  1998  1999  2000  <20012002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  2021  2022  2023  2024 
ANJ, 10 Aug 2010 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·