Re: Standard String

[Benjamin Franksen <benjamin.franksen@bessy.de>]

On Wednesday 20 July 2005 22:02, Andrew Johnson wrote:
> Benjamin Franksen wrote:
> >>how you would implement this record type if all strings become
> >>read-only once created.
> >
> > You already answered your question: The string gets regenerated
> > every time. It's a new string, different from the old one. No
> > destructive update of the string is necessary to achieve that.
>
> So every time you process the record, you free() the old string
> buffer and malloc() at least one new one, if not several as we extend
> the string?  I don't see how that helps us at all, so I'm probably
> not understanding what you meant.

We can use reference counting for the string data. This still means 
allocating a new block each time you construct a new string. On the 
other hand, some of the data can be shared; for instance between 
records where data is transfered from one to the other with a db link; 
or in case of communication with the CA server task. Or for record name 
and property name lookup, etc., whenever the string is only passed 
around and read but not changed.

> You have a record (a struct) that contains a string field as a data
> member.  If you say I can't modify the string, and I say I can't move
> the record in memory, how do I change the value of that string field?

Assignment, overloaded for the string class.

> The current EPICS code only allocates memory for database storage at
> dbLoadRecords() and iocInit() time, for a very good reason: continual
> allocation and freeing of memory *will* fragment the IOC memory pool
> into smaller and smaller chunks (we know that from experience, and
> take some special precautions inside iocCore which you may not be
> aware of). The vxWorks memory allocator is *particularly* bad when it
> comes to fragmentation, but even using a better allocator it's not
> going to be possible to run an IOC for months or even years without
> rebooting if it's continually churning its memory.

Then you will not be able to reliably run an EPICS V4 IOC on such a 
system, however much low-level memory re-use you put in place. Do you 
propose that memory used by a record instance will be re-used for new 
records when the old one has been deleted? Or else, how often am I 
allowed to change/update a running database before all memory is 
fragmented and the IOC crashes? What good will the new V4 EPICS version 
do me, if I must not use the new features, or else risk running out of 
memory sooner or later?

Your epicsString interface has a method 'reserve', that presumably calls 
re-alloc underneath. Now, as a programmer of, say, a record support or 
a device support, how hard should I try to avoid calling 'reserve', 
assuming some new version of a string is shorter than the previous one. 
Never? Or is this in fact a no-op for arguments smaller than the 
current capacity? But that would be a memory leak, too, wouldn't it? 
Because maybe after an initially very large string, all following might 
be quite small and all the memory is wasted.

OTOH, run-away fragmentation of the kind you describe above is normally 
observed only with allocators that cannot merge adjacent free blocks. 
If the VxWorks allocators is that bad, we will have to replace it 
anyway. Otherwise fragmentation for real-world programs is typically 
around 10% and does not exceed 50% of live memory. At least that is 
what studies say.

> Please remember that we have to be able to run on small footprint
> systems with 8MB or maybe 16MB of RAM (I think we said 8 at the
> meeting last week), and there's no virtual memory so no swapping or
> paging possible on these small IOCs.

I am quite aware of these constraints.

Ben