> Here is an error message about bad UDP messages. We have
> seen this from two different IOCs over the weekend.
This could be caused by a server that is not CA that might
be listening on UDP port 5064 and responding. Alternatively,
a network with an exceptionally high bit error rate might
damage many bits in a UDP frame until the odds become higher
that the CRC checksum filter will not detect the damage, and
discard the message.
> CAC: post_msg(): Corrupt cmd in UDP msg b
> ../iocinf.c: bad UDP msg from d0olctl50.fnal.gov:5064
If this is from an IOC then it probably has Tornado II with DNS
installed because it appears that DNS has been used to
convert the IP address to a host name. Alternatively they might
have used the hostAdd() interface to populate a host table, but
this approach is less commonly used.
> 1) This message, as pointed by Andrew, one of our terminal server
> responds to the CA search UDP
> message at port 5064 and sends a NON-epics reply to the IOC which send
> out the CA search UDP message.
> Although we have several terminal servers of same type, only one of them
> causes this symptom.
>
> 2) Usually, it indicates there is unresolved channel connection on this
> IOC. So we have check channel
> connection status on this IOC.
That's interesting. We had a terminal server on LEDA that was causing trouble
that was not well understood. This device is currently disconnected from
our network. Personally, I would be very hesitant to leave any device connected
to an EPICS network that was known to be responding to messages on UDP port 5064.
Alternatively, you could configure CA to use a different port.
>
> 3) I believe the program output this message has memory leak. So if you
> leave this message long enough, this would cause low memory situation,
> according to my experience.
>
I was looking at this carefully and I don't see a memory leak in this part
of CA in R3.13. Nevertheless, since both Noboru and Geoff have seen this
in a host that is low on memory I am concerned.
There are only two UDP response stubs in the CA client library that allocate
memory.
1) The stub that receives a CA beacon message for an IP address that has not
been seen before.
2) The stub that receives a CA search response for an IP address that has not
been seen before.
After careful examination of this part of the code it appears
that a memory leak would only occur if the rogue server was replying to the
CA request with a response message that matched the above two response codes
and also was varying the IP address portion of the message enough that the
resource consumption was noticeable.
Otherwise, the EPICS routine epicsVprintf() and the vxWorks routine
hostGetByAddr() are also used in this part of the code. Since these
routines are used quite frequently perhaps it less likely that they would
have a memory leak that has not been detected. Nevertheless, use of DNS in
the vxWorks environment is new in Tornado II so it might be interesting
to build a system without DNS and see if the behavior changes.
In what versions of EPICS R3.13 were these problems observed?
Jeff
- References:
- Re: bad UDP messages Noboru Yamamoto
- Navigate by Date:
- Prev:
channel access john sinclair
- Next:
RE: Error message from sequencer Jeff Hill
- Index:
1994
1995
1996
1997
1998
1999
2000
<2001>
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: bad UDP messages Noboru Yamamoto
- Next:
RE: VxWorks on Linux Mark Rivers
- Index:
1994
1995
1996
1997
1998
1999
2000
<2001>
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|