Re: CA server message buffers

[hill@luke.atdiv.lanl.gov (Jeff Hill)]

Perry,

> 	I'm getting R3.12.1 to run on Sparc targets and need some insight
> into how the buffers are managed in the server code.
> 
> 	My Sparcs were generating memory alignment exceptions when 
> getEnumDouble() attempted to write a double into the send buffer address.
> I traced this back into cas_alloc_msg() (rsrv/caserverio.c) and found that
> although the message sizes were aligned to 8 bytes, the base address of the
> send buffer wasn't - so the buffer addresses weren't - hence the error.
> 

Sorry about this but we have only recently began porting the server code
to run on multiple architectures.

I see one possible cause of problems. If the message buffer starts
at a proper boundary then all protocol items will be properly
aligned because each item's length is an integer multiple of the 
largest data item passed - a double. The structure below has 3 long's 
before the protocol buffer (which is an array of characters). Long's are 
4 bytes on the SPARC so we end up with the protocol buffer potentially
on a 4 byte boundary. This is a problem.

struct message_buffer{
        unsigned long           stk;
        unsigned long           maxstk;
       	long           		cnt;
        char                    buf[MAX_MSG_SIZE];
};

Try making the following change to the structure:

/*
 * !! align and buf must be next to each other !!
 *
 * The dbr_double_t pad guarantees buf will have
 * proper alignment because dbr_double_t is currently 
 * the largest atomic data type passed in the CA protocol 
 * (I really should malloc the buffer)
 */
struct message_buffer{
	const dbr_double_t	align;
        char                    buf[MAX_MSG_SIZE];
        unsigned long           stk;
        unsigned long           maxstk;
       	long           		cnt;
};

With a double immediately preceding the buffer the compiler
will force proper alignment. Of course this is ugly and I 
should malloc the buffer however I am looking for a simple
reliable fix that will not require extensive changes on your part.

> 	The simplest solution seemed to be to add the following line to
> the start of cas_message_align() to force the alignment of the send buffer:
> 	pclient->send.stk += 
> 		CA_MESSAGE_ALIGN((long)EXTMSGPTR(pclient)) -
> 		(long)EXTMSGPTR(pclient);
> 
> 	While this had the desired effect of forcing the message buffer
> addresses to be 8-byte aligned it had the following side-effects:
> 	1) The stack index (pclient->send.stk) no longer changes, and
> 	2) The client (in this case caget) complaines about bad UDP
> 	messages.
> 	

This does not have the desired effect because it inserts garbage 
bytes in between or on top of individual protocol items.

Let me know if the above fix has the desired effect. If not
I will take a closer look.

Jeff


______________________________________________________________________
Jeffrey O. Hill			Internet	johill@lanl.gov
LANL MS H820			Voice		505 665 1831
Los Alamos, NM 87545 USA 	FAX		505 665 5107