Re: calcout record precision

[Tim Mooney <mooney@aps.anl.gov>]

Andrew,  I agree with this.
Tim


On 5/19/2010 4:38 PM, Andrew Johnson wrote:
> Hi Tim,
>
> On Wednesday 19 May 2010 15:25:26 Tim Mooney wrote:
> > Andrew,  I just noticed that the calcout record displays
> > all numeric fields except VAL as though PREC were 0.  It
> > looks like code changed in 3.14.10, or maybe earlier, to
> > let recGblGetPrec() choose a precision based on the field
> > type, but recGblGetPrec() doesn't do this.  recGblGetPrec()
> > expects the caller to have chosen a precision for floats,
> > and merely restricts it to a reasonable range.
>  From what I can see, that dates back to before R3.14.9, to a change I made in
> October 2006, which I agree was wrong (I broke the calc record similarly).
> However I don't believe the way recGblGetPrec() is used in the other record
> types really makes much sense either; they actually implement this:
>
>   * For VAL or some other small list of fields, the precision used
>     is read directly from PREC.
>   * For any other double or float fields, the precision used will be
>     the value of PREC but clipped to the range 0-15.
>   * For any integer fields, the precision is zero.
>
> As you said above, recGblGetPrec() is merely doing the clipping to the range
> relevant for the field type (but it ought to be more like 0-7 for floats).
> The name is wrong, it should be more like recGblLimitPrec().  It doesn't seem
> to make much sense to only call it for the non-VAL fields though, we should be
> calling it for all fields.
>
> I suggest that for all record types that have one, the get_precision() routine
> should really look like this (modulo the type name and cast for prec):
>
> static long get_precision(DBADDR *paddr, long *precision)
> {
>      aoRecord	*prec = (aoRecord *)paddr->precord;
>
>      *precision = prec->prec;
>      recGblGetPrec(paddr, precision);
>      return 0;
> }
>
> The only reason to have conditionals in that routine would be if your record
> type had more than one PREC field, which would be completely legal although I
> don't know of any that do.
>
> Comments?
>
> - Andrew
--
Tim Mooney (mooney@aps.anl.gov) (630)252-5417
Software Services Group, Advanced Photon Source, Argonne National Lab.