Argonne National Laboratory

Experimental Physics and
Industrial Control System

2002  2003  2004  <20052006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020  Index 2002  2003  2004  <20052006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020 
<== Date ==> <== Thread ==>

Subject: RE: memory management
From: Jeff Hill <johill@lanl.gov>
To: 'Eric Norum' <norume@aps.anl.gov>
Cc: 'Ralph Lange' <Ralph.Lange@bessy.de>, 'Matej Sekoranja' <matej.sekoranja@ijs.si>, 'Marty Kraimer' <mrk@aps.anl.gov>, 'Andrew Johnson' <anj@aps.anl.gov>, 'Ken Evans' <evans@aps.anl.gov>, 'Benjamin Franksen' <benjamin.franksen@bessy.de>, 'Bob Dalesio' <bdalesio1@comcast.net>, "'Kasemir, Kay'" <kasemirk@ornl.gov>
Date: Thu, 24 Feb 2005 16:54:43 -0700
>> The executive summary is that, with the C RTL, continuously
>> allocating random sized blocks has a very bad impact on memory
>> fragmentation, and therefore CPU overhead. 

> Just curious.  What's the basis for the above statement?  
> Is this problem present everywhere or only on vxWorks?

In my experience malloc and free are very high overhead calls
compared to free list based memory management. It is well known
that the overhead associated with calls to malloc and free goes
up when the number of fragments in pool increases. When designing
the C RTL it is important to avoid fragmenting memory in the
first place. There are many different approaches {best fit, first
fit etc} and considerable research has gone into finding an
optimal approach. Nevertheless, it is still possible to fragment
memory. Naturally, fragmentation will be worse with certain usage
patterns. For example it will be worse when you allocate in
between every block that you keep a small block that you will
free. The particular choice of {best fit, first fit etc} isn't
going to eliminate the problem. However, if you are willing to
add some additional latency and overhead to malloc and free, you
can decrease the probability of noticeable pool fragmentation.

We did experience memory fragmentation issues on vxWorks in the
90's. I vaguely recall that at the time, after someone had a
quick look at the WRS source code, it was stated that the WRS
memory allocation algorithms wasn't main stream. As I recall the
issue became somewhat more apparent when the locally adapted
LEX/YACC parser code was used in the IOC for the first time to
load ASCII databases. The solution applied was to modify that
code (and others) to make more use of free lists.

Of course, Java has other ways of approaching this issue which,
according to conventional wisdom, tend to increase non
deterministic behavior, but I understand that considerable
research is being devoted to that issue also.

Memory management is typically a pivotal part of the design for
IP stacks, and no surprise from my vantage point, they are not
basing the high throughput stuff on continuous use of malloc and
free.

It's important to keep in mind the type of software that we are
writing. Reliability is important, and systems are expected to
run without a restart for months or years. Limited memory
situations are likely. Proper efficiency and scaling behavior are
also an important part of the EPICS niche.

Jeff




Replies:
Re: memory management Benjamin Franksen

Navigate by Date:
Prev: memory management Jeff Hill
Next: Re: EPICS base V4 Ralph Lange
Index: 2002  2003  2004  <20052006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020 
Navigate by Thread:
Prev: memory management Jeff Hill
Next: Re: memory management Benjamin Franksen
Index: 2002  2003  2004  <20052006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  2018  2019  2020 
ANJ, 02 Feb 2012 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·