Re: gateway aborted by SIGBUS

[Bruce Hill <bhill@slac.stanford.edu>]

FYI
We've been running the 64 bit version of gateway 2.0.3.0 since Sept 2009,
compiled against base 3.14.11.    As I recall, we had some issues early on,
but I don't recall them being related to 32 vs 64 bit.    It's been running reliably
since then.

Current IOC count through the gateway machine is over 380, probably closer
to 400, and our ulimit stack size is the default 10240.

Regards,
- Bruce

On 04/26/2011 08:19 AM, Andrew Johnson wrote:

On Tuesday 26 April 2011 08:06:52 Martin L. Smith wrote:
  

If your gateways are running under Linux my understanding is that you may
 want to increase the stack size limit. I have a comment in my starter
 file:

# Limit stacksize for Linux to increase number of threads
# 1024 is 204 threads (default)
#  4096 is 512 threads
ulimit -s 4096

As you can see I use 4096 although I'm not sure that this will help
 especially if you are not running on Linux.
    

I want to add a little more information about what this actually does for 
future reference: Like most operating systems Linux has to reserve a fixed 
amount of virtual memory space for the program stack for each independent 
thread that gets started.  You can't move or extend a stack after the thread 
has started as each thread's stack is separated by the reserved distance from 
the next thread.

It depends on your particular distribution but that distance often defaults to 
around 10240KB=10MB.  If you're running on a 32-bit Intel CPU the OS usually 
only allows a process to use half of its virtual memory space, i.e. 2GB.  
2GB=2048MB divided by 10MB means that a single process such as the Gateway 
can't have more than 204 threads by default.  The CA client library starts two 
threads for each IOC it talks to, so that would limit the gateway or 
nameserver to connecting with at most about 100 IOCs.

Marty's "ulimit -s 4096" command sets the default stack size to 4096KB=4MB, 
thus permitting up to about 512 threads or 256 IOCs.  You can find the limit 
for a particular machine by running "ulimit -s" under Bash or "limit" under 
the C-shell.  In practice it should be possible to run the gateway code with a 
significantly smaller limit, but I don't know if anyone has tried to find out 
how small you can go — I would expect it to still work in 64K, although it 
seems unlikely that anyone would need a single gateway able to connect to 
32768 IOCs.

An alternative is to run the gateway on a 64-bit machine where the available 
virtual memory space is much larger, but I don't know whether the gateway code 
is 64-bit safe — has anyone tried that?

- Andrew
  

-- 
Bruce Hill
Member Technical Staff
SLAC National Accelerator Lab
2575 Sand Hill Road M/S 10
Menlo Park, CA  94025