Re: Memory Problem related to CA?

[Andrew Johnson <anj@aps.anl.gov>]

Jeff Hill wrote:
> > What does the first number in parenthesis mean? Something like 
> > an element count. For this sick connection it was 54! Please look at the
> attachment.
>
> This means that your client has subscribed (called ca_create_subscription or
> ca_add_event) 54 times! One option of course will be to find the misbehaved
> client (the IP address is in the casr report) and convince its author to
> recode his application.

One common bug that can cause such a large number of connections is if 
the channel connection callback routine creates a new subscription 
whenever a channel connection comes up, even if it already had a 
subscription from the last time it connected.  The application in 
question here appears to be the DESY archiver which is likely to have an 
effect on a large number of IOCs, so I would strongly recommend that 
this code be reviewed for incorrect channel subscription logic.  A 
strong indication that this kind of bug is present is if the number 
increases over an IOC reboot and decreases over an application restart.

> > The best
> > would be to disconnect all CA connections and give back (at least a part
> > of) the additionally allocated memory back to the system pool.
>
> We don't currently have an option to do this, but considering that vxWorks
> memory allocation is much better than in the past perhaps one option that
> should be considered for R3.15 is to use free lists dedicated to each client
> that are returned to system pool when the client disconnects.

Note that this IOC is running R3.13.10 on vxWorks 5.3.1; if my memory is 
correct, this is using the old (pre-SENS) TCP/IP stack, so its network 
buffer management is very different to the SENS stack which has fixed 
buffer sizes.  The earlier stack could (and did) call malloc() to get 
new buffers after initialization time.

Also, if the memory is being used up by a multiple subscription bug in a 
client this kind of memory pool rework would make no difference anyway.

- Andrew
--
When a distinguished but elderly scientist states that something is
possible, he is almost certainly right.  When he states that something
is impossible, he is very probably wrong.  -- Arthur C. Clarke