Re: ao record convert overflow

[Andrew Johnson <anj@aps.anl.gov>]

Hi Bernd,

On 03/20/2015 06:01 AM, Schoeneburg, Bernd wrote:
> Epics R3.14.11 (same as 3.14.12)  vxWorks Pentium
> I have encountered a problem when using the aoRecord. The convert
> routine produces an "overlow" when the double is casted to long. When
> the value is > (double)LONG_MAX the resulting value of RVAL becomes
> LONG_MIN instead LONG_MAX. This jump is not healthy for output devices.
> Possibly this happens only in my environment of compiler or Pentium
> target. 

Your Pentium target might be why you're the first person to report this,
the behaviour on PowerPC might be more sensible (I haven't checked
though). Setting DRVH might have prevented the problem, but the existing
code triggers Undefined Behaviour when the converted raw value falls
outside of the epicsInt32 range, so we do have to fix the code even if
that results in some subtle behaviour change to an existing IOC.

I've just committed the attached changes to the 3.14 branch, which
clamps both the +ve and -ve ends of the conversion. Note that a long is
64 bits wide on some target architectures, so I use explicit literal
values for the 32-bit range limits.

Thanks,

- Andrew
-- 
Light thinks it travels faster than anything but it is wrong.
No matter how fast light travels, it finds the darkness has
always got there first, and is waiting for it.
    -- Terry Pratchett, Reaper Man

=== modified file 'src/rec/aiRecord.dbd'
--- src/rec/aiRecord.dbd	2005-11-15 23:35:34 +0000
+++ src/rec/aiRecord.dbd	2015-03-20 18:34:35 +0000
@@ -176,8 +176,8 @@
 		pp(TRUE)
 		interest(2)
 	}
-	field(ROFF,DBF_LONG) {
-		prompt("Raw Offset, obsolete")
+	field(ROFF,DBF_ULONG) {
+		prompt("Raw Offset")
 		pp(TRUE)
 		interest(2)
 	}

=== modified file 'src/rec/aoRecord.c'
--- src/rec/aoRecord.c	2013-11-20 22:21:10 +0000
+++ src/rec/aoRecord.c	2015-03-20 19:10:43 +0000
@@ -469,10 +469,20 @@
     }
     value -= prec->aoff;
     if (prec->aslo != 0) value /= prec->aslo;
-    if (value >= 0.0)
-        prec->rval = (epicsInt32)(value + 0.5) - prec->roff;
-    else
-        prec->rval = (epicsInt32)(value - 0.5) - prec->roff;
+
+    /* Apply raw offset and limits, round to 32-bit integer */
+    value -= prec->roff;
+    if (value >= 0.0) {
+        if (value >= (0x7fffffff - 0.5))
+            prec->rval = 0x7fffffff;
+        else
+            prec->rval = (epicsInt32)(value + 0.5);
+    } else {
+        if (value > (0.5 - 0x80000000))
+            prec->rval = (epicsInt32)(value - 0.5);
+        else
+            prec->rval = 0x80000000;
+    }
 }
 
 

=== modified file 'src/rec/aoRecord.dbd'
--- src/rec/aoRecord.dbd	2002-07-12 21:35:43 +0000
+++ src/rec/aoRecord.dbd	2015-03-20 17:58:18 +0000
@@ -82,8 +82,8 @@
 		interest(1)
 		size(16)
 	}
-	field(ROFF,DBF_LONG) {
-		prompt("Raw Offset, obsolete")
+	field(ROFF,DBF_ULONG) {
+		prompt("Raw Offset")
 		pp(TRUE)
 		interest(2)
 	}