RE: excessive ioc memory utilization

["Jeff Hill" <johill@lanl.gov>]

Geoff,

Use memShow to obtain status of the system pool. A common cause
of system pool depletion will be system pool data structure
corruption, or of course some program continuously allocating
memory that it does not free. The former shows up instantaneously
while the latter generally can be predicted as a trend. If the
problem is easily reproducible you may find that selective use of
the task suspend command can be used to isolate the responsible
thread. Typically a binary search is used where you suspend one
half of the threads initially followed by half of the remaining
half etc to tighten the noose with minimized effort.

With your version of vxWorks the network stack pool shouldn't be
increasing past the limits set when vxWorks is built.

There is an integer level argument to casr that can be used to
get more information about what it has allocated. 

Jeff

> -----Original Message-----
> From: Geoff Savage [mailto:savage@fnal.gov]
> Sent: Tuesday, May 10, 2005 9:48 AM
> To: EPICS Tech-Talk
> Subject: excessive ioc memory utilization
> 
> Hi,
> 
> When I came in this morning our high voltage iocs are using
> excessive
> amounts of memory, around 95%.  They are all mv2301 processors
> with 16MB
> of memory running epics 3.14.6 built with vxworks 5.5.1.  The
> netStackSysPoolShow and netStackDataPoolShow outputs are
> reasonable.
> There are not an excessive number of channels connected.  What
> can I use
> to look and see where the memory is allocated?  Any other
> suggestions?
> 
> Thanks,  Geoff
> 
> -> netStackSysPoolShow
> type        number
> ---------   ------
> FREE    :    5991
> DATA    :      0
> HEADER  :      0
> SOCKET  :     21
> PCB     :     35
> RTABLE  :     91
> HTABLE  :      0
> ATABLE  :      0
> SONAME  :      0
> ZOMBIE  :      0
> SOOPTS  :      0
> FTABLE  :      0
> RIGHTS  :      0
> IFADDR  :      4
> CONTROL :      0
> OOBDATA :      0
> IPMOPTS :      0
> IPMADDR :      2
> IFMADDR :      0
> MRTABLE :      0
> TOTAL   :    6144
> number of mbufs: 6144
> number of times failed to find space: 0
> number of times waited for space: 0
> number of times drained protocols for space: 0
> __________________
> CLUSTER POOL TABLE
> _______________________________________________________________
> ________________
> size     clusters  free      usage
> ---------------------------------------------------------------
> ----------------
> 64       1024      965       15123
> 128      1024      996       1647
> 256      512       446       10893
> 512      512       512       0
> ---------------------------------------------------------------
> ----------------
> value = 80 = 0x50 = 'P'
> 
> -> netStackSysPoolShow
> type        number
> ---------   ------
> FREE    :    5991
> DATA    :      0
> HEADER  :      0
> SOCKET  :     21
> PCB     :     35
> RTABLE  :     91
> HTABLE  :      0
> ATABLE  :      0
> SONAME  :      0
> ZOMBIE  :      0
> SOOPTS  :      0
> FTABLE  :      0
> RIGHTS  :      0
> IFADDR  :      4
> CONTROL :      0
> OOBDATA :      0
> IPMOPTS :      0
> IPMADDR :      2
> IFMADDR :      0
> MRTABLE :      0
> TOTAL   :    6144
> number of mbufs: 6144
> number of times failed to find space: 0
> number of times waited for space: 0
> number of times drained protocols for space: 0
> __________________
> CLUSTER POOL TABLE
> _______________________________________________________________
> ________________
> size     clusters  free      usage
> ---------------------------------------------------------------
> ----------------
> 64       1024      965       15125
> 128      1024      996       1647
> 256      512       446       10894
> 512      512       512       0
> ---------------------------------------------------------------
> ----------------
> value = 80 = 0x50 = 'P'
> 
> 
> -> casr
> Channel Access Server V4.11
> Connected circuits:
> TCP 131.225.231.65:46261(d0olj.fnal.gov): User="d0cal", V4.8,
> 70
> Channels, Priority=0
> TCP 131.225.230.156:1026(d0olctl38): User="vxworks", V4.11, 6
> Channels,
> Priority=80
> TCP 131.225.231.39:45142(d0ol57.fnal.gov): User="d0fpd", V4.8,
> 6
> Channels, Priority=0
> TCP 131.225.231.39:45149(d0ol57.fnal.gov): User="d0fpd", V4.8,
> 4
> Channels, Priority=0
> TCP 131.225.231.252:60499(d0ol49.fnal.gov): User="d0fpd", V4.8,
> 4
> Channels, Priority=0
> TCP 131.225.231.252:49088(d0ol49.fnal.gov): User="d0fpd", V4.8,
> 30
> Channels, Priority=0
> TCP 131.225.231.39:49714(d0ol57.fnal.gov): User="d0fpd", V4.8,
> 6
> Channels, Priority=0
> TCP 131.225.231.247:42004(d0ol45.fnal.gov): User="d0cal", V4.8,
> 3
> Channels, Priority=0
> TCP 131.225.231.247:43286(d0ol45.fnal.gov): User="d0cal", V4.8,
> 128
> Channels, Priority=0
> value = 0 = 0x0