Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019 
<== Date ==> <== Thread ==>

Subject: Re: compress record FIFI/LIFO not working
From: Daniel Schick <Daniel.Schick@mbi-berlin.de>
To: Heinz Junkes <junkes@fhi-berlin.mpg.de>, Talk EPICS Tech <tech-talk@aps.anl.gov>
Date: Fri, 13 Apr 2018 11:08:54 +0200
Hello Heinz,

I guess that this bug is independent of the averaging, since it happens already for a simple ring-buffer-type of compress record.

From simply comparing the 3.15 and 3.16 source code for the compress record: epics-base/plain/src/std/rec/compressRecord.c

There is an obvious difference in how the values are put into the record.

For 3.16:


static void put_value(compressRecord *prec, double *psource, int n){ int fifo = (prec->balg == bufferingALG_FIFO); epicsUInt32 offset = prec->off; epicsUInt32 nuse = prec->nuse; epicsUInt32 nsam = prec->nsam; nuse += n; if (nuse > nsam) nuse = nsam; while (n--) { /* for LIFO, decrement before */ if (!fifo) offset = (offset - 1) % nsam; prec->bptr[offset] = *psource++; /* for FIFO, increment after */ if (fifo) offset = (offset + 1) % nsam; } prec->off = offset; prec->nuse = nuse;}

and for 3.15

static void put_value(compressRecord *prec, double *psource, int n)

{ epicsInt32 offset = prec->off; epicsInt32 nuse = prec->nuse; epicsInt32 nsam = prec->nsam; double *pdest = prec->bptr + offset; int i; for (i = 0; i < n; i++) { *pdest = *psource++; if (++offset >= nsam) { pdest = prec->bptr; offset = 0; } else pdest++; } nuse += n; if (nuse > nsam) nuse = nsam; prec->off = offset; prec->nuse = nuse; return;}

There is also already a bug report: https://bugs.launchpad.net/epics-base/+bug/1743795

Best

Daniel

Am 13.04.18 10:19 schrieb Heinz Junkes <junkes@fhi-berlin.mpg.de>:
> 
> Same with 
> 
> ############################################################################
> ## EPICS R7.0.1.2-DEV
> ## EPICS Base built Apr 9 2018
> ############################################################################
> 
> compress with ALG "N to 1 Average”.
> 
> db-file:
> 
> record(waveform, "$(P):TimeScale") {
> field(FTVL, "DOUBLE")
> field(NELM, "2500")
> field(EGU, "ns")
> field(FLNK, "$(P):TimeScale:mid")
> field(TPRO, "1")
> }
> 
> record(compress, "$(P):TimeScale:mid") {
> field(DESC, "Time scale (mid res)")
> field(INP, "$(P):TimeScale")
> field(ALG, "N to 1 Average")
> field(N, "5")
> field(NSAM, "500")
> field(EGU, "ns")
> field(TPRO, "1")
> }
> 
> 
> caget dumm:TimeScale 2500 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360 370 380 390 400 410 420 430 440 450 460 470 480 490 500 510 520 530 540 550 560 570 580 590 600 610 620 630 640 650 660 670 680 690 700 710 720 730 740 750 760 770 780 790 800 810 820 830 840 850 860 870 880 890 900 910 920 930 940 950 960 970 980 990 1000 1010 1020 1030 1040 1050 1060 1070 1080 1090 1100 1110 1120 1130 1140 1150 1160 1170 1180 1190 1200 1210 1220 1230 1240 1250 1260 1270 1280 1290 1300 1310 1320 1330 1340 … 24960 24970 24980 24990
> 
> 
> caget dumm:TimeScale:mid 500 14820 14870 14920 14970 15020 15070 15120 15170 15220 15270 15320 15370 15420 15470 15520 15570 15620 15670 15720 15770 15820 15870 15920 15970 16020 16070 16120 16170 16220 16270 16320 16370 16420 16470 16520 16570 16620 16670 16720 16770 16820 16870 16920 16970 17020 17070 17120 17170 17220 17270 17320 17370 17420 17470 17520 17570 17620 17670 17720 17770 17820 17870 17920 17970 18020 18070 18120 18170 18220 18270 18320 18370 18420 18470 18520 18570 18620 18670 18720 18770 18820 18870 18920 18970 19020 19070 19120 19170 19220 19270 19320 19370 19420 19470 19520 19570 19620 19670 19720 19770 19820 19870 19920 19970 20020 20070 20120 20170 20220 20270 20320 20370 20420 20470 20520 20570 20620 20670 20720 20770 20820 20870 20920 20970 21020 21070 21120 21170 21220 21270 21320 21370 21420 21470 21520 21570 21620 21670 21720 21770 21820 21870 21920 21970 22020 22070 22120 22170 22220 22270 22320 22370 22420 22470 22520 22570 22620 22670 22720 22770 22820 22870 22920 22970 23020 23070 23120 23170 23220 23270 23320 23370 23420 23470 23520 23570 23620 23670 23720 23770 23820 23870 23920 23970 24020 24070 24120 24170 24220 24270 24320 24370 24420 24470 24520 24570 24620 24670 24720 24770 24820 24870 24920 24970 20 70 120 170 220 270 320 370 420 470 520 570 620 670 720 770 820 870 920 970 1020 1070 1120 1170 1220 1270 1320 1370 1420 1470 1520 1570 1620 1670 1720 1770 1820 1870 1920 1970 2020 2070 2120 2170 2220 2270 2320 2370 2420 2470 2520 2570 2620 2670 2720 2770 2820 2870 2920 2970 3020 3070 3120 3170 3220 3270 3320 3370 3420 3470 3520 3570 3620 3670 3720 3770 3820 3870 3920 3970 4020 4070 4120 4170 4220 4270 4320 4370 4420 4470 4520 4570 4620 4670 4720 4770 4820 4870 4920 4970 5020 5070 5120 5170 5220 5270 5320 5370 5420 5470 5520 5570 5620 5670 5720 5770 5820 5870 5920 5970 6020 6070 6120 6170 6220 6270 6320 6370 6420 6470 6520 6570 6620 6670 6720 6770 6820 6870 6920 6970 7020 7070 7120 7170 7220 7270 7320 7370 7420 7470 7520 7570 7620 7670 7720 7770 7820 7870 7920 7970 8020 8070 8120 8170 8220 8270 8320 8370 8420 8470 8520 8570 8620 8670 8720 8770 8820 8870 8920 8970 9020 9070 9120 9170 9220 9270 9320 9370 9420 9470 9520 9570 9620 9670 9720 9770 9820 9870 9920 9970 10020 10070 10120 10170 10220 10270 10320 10370 10420 10470 10520 10570 10620 10670 10720 10770 10820 10870 10920 10970 11020 11070 11120 11170 11220 11270 11320 11370 11420 11470 11520 11570 11620 11670 11720 11770 11820 11870 11920 11970 12020 12070 12120 12170 12220 12270 12320 12370 12420 12470 12520 12570 12620 12670 12720 12770 12820 12870 12920 12970 13020 13070 13120 13170 13220 13270 13320 13370 13420 13470 13520 13570 13620 13670 13720 13770 13820 13870 13920 13970 14020 14070 14120 14170 14220 14270 14320 14370 14420 14470 14520 14570 14620 14670 14720 14770
> 
> 
> Setting NSAM in TimeScale:mid to 2500 same as TimeScale … then It works.
> 
> caget dumm:TimeScale:mid 2500 20 70 120 170 220 270 320 370 420 470 520 570 620 670 720 770 820 870 920 970 1020 1070 1120 1170 1220 1270 1320 1370 1420 1470 1520 1570 1620 1670 1720 1770 1820 1870 1920 1970 2020 2070 2120 2170 2220 2270 2320 2370 2420 2470 2520 2570 2620 2670 2720 2770 2820 2870 2920 2970 3020 3070 3120 3170 3220 3270 3320 3370 3420 3470 3520 3570 3620 3670 3720 3770 3820 3870 3920 3970 4020 4070 4120 4170 4220 4270 4320 4370 4420 4470 4520 4570 4620 4670 4720 4770 4820 4870 4920 4970 5020 5070 5120 5170 5220 5270 5320 5370 5420 5470 5520 5570 5620 5670 5720 5770 5820 5870 5920 5970 6020 6070 6120 6170 6220 6270 6320 6370 6420 6470 6520 6570 6620 6670 6720 6770 6820 6870 6920 6970 7020 7070 7120 7170 7220 7270 7320 7370 7420 7470 7520 7570 7620 7670 7720 7770 7820 7870 7920 7970 8020 8070 8120 8170 8220 8270 8320 8370 8420 8470 8520 8570 8620 8670 8720 8770 8820 8870 8920 8970 9020 9070 9120 9170 9220 9270 9320 9370 9420 9470 9520 9570 9620 9670 9720 9770 9820 9870 9920 9970 10020 10070 10120 10170 10220 10270 10320 10370 10420 10470 10520 10570 10620 10670 10720 10770 10820 10870 10920 10970 11020 11070 11120 11170 11220 11270 11320 11370 11420 11470 11520 11570 11620 11670 11720 11770 11820 11870 11920 11970 12020 12070 12120 12170 12220 12270 12320 12370 12420 12470 12520 12570 12620 12670 12720 12770 12820 12870 12920 12970 13020 13070 13120 13170 13220 13270 13320 13370 13420 13470 13520 13570 13620 13670 13720 13770 13820 13870 13920 13970 14020 14070 14120 14170 14220 14270 14320 14370 14420 14470 14520 14570 14620 14670 14720 14770 14820 14870 14920 14970 15020 15070 15120 15170 15220 15270 15320 15370 15420 15470 15520 15570 15620 15670 15720 15770 15820 15870 15920 15970 16020 16070 16120 16170 16220 16270 16320 16370 16420 16470 16520 16570 16620 16670 16720 16770 16820 16870 16920 16970 17020 17070 17120 17170 17220 17270 17320 17370 17420 17470 17520 17570 17620 17670 17720 17770 17820 17870 17920 17970 18020 18070 18120 18170 18220 18270 18320 18370 18420 18470 18520 18570 18620 18670 18720 18770 18820 18870 18920 18970 19020 19070 19120 19170 19220 19270 19320 19370 19420 19470 19520 19570 19620 19670 19720 19770 19820 19870 19920 19970 20020 20070 20120 20170 20220 20270 20320 20370 20420 20470 20520 20570 20620 20670 20720 20770 20820 20870 20920 20970 21020 21070 21120 21170 21220 21270 21320 21370 21420 21470 21520 21570 21620 21670 21720 21770 21820 21870 21920 21970 22020 22070 22120 22170 22220 22270 22320 22370 22420 22470 22520 22570 22620 22670 22720 22770 22820 22870 22920 22970 23020 23070 23120 23170 23220 23270 23320 23370 23420 23470 23520 23570 23620 23670 23720 23770 23820 23870 23920 23970 24020 24070 24120 24170 24220 24270 24320 24370 24420 24470 24520 24570 24620 24670 24720 24770 24820 24870 24920 24970 0 0 0 0 0 0 0 0 0 0 0 0 0 …
> 
> 
> One might read this behaviour in the documentation:
> 
> from: https://wiki-ext.aps.anl.gov/epics/index.php/RRM_3-14
> 
> ! The retrieved array is truncated to be of length NSAM. ! Maybe that happens in the wrong place :-(
> 
> "Average takes an average of every element of the array obtained from INP over time; that is, the entire array referenced by INP is retrieved, and for each element the new average is calculated and placed in the corresponding element of the value buffer. The retrieved array is truncated to be of length NSAM. N successive arrays are averaged and placed in the buffer. Thus, VAL[0] holds the average of the first element of INP over N samples, VAL[1] holds the average of the next element of INP over N samples, and so on.”
> 
> 
> 
> 
> > On 9. Jan 2018, at 04:57, Michael Davidsaver  wrote:
> > 
> > Confirmed. Seems like the NSAM=4 case being tested with src/std/rec/test/compressTest.c
> > is somehow special.
> > 
> > FIFO w/ NSAM=4 is as expected
> > 
> >> ring 2018-01-08 19:44:53.628938 2 1 2 
> >> ring 2018-01-08 19:44:54.628815 3 1 2 3 
> >> ring 2018-01-08 19:44:55.628853 4 1 2 3 4 
> >> ring 2018-01-08 19:44:56.628797 4 2 3 4 5 
> >> ring 2018-01-08 19:44:57.628762 4 3 4 5 6 
> >> ring 2018-01-08 19:44:58.628891 4 4 5 6 7 
> >> ring 2018-01-08 19:44:59.628730 4 5 6 7 8 
> >> ring 2018-01-08 19:45:00.628934 4 6 7 8 9 
> >> ring 2018-01-08 19:45:01.628738 4 7 8 9 10
> > 
> > But FIFO w/ NSAM=5 isn't
> > 
> >> ring 2018-01-08 19:45:23.105160 2 1 2 
> >> ring 2018-01-08 19:45:24.105240 3 1 2 3 
> >> ring 2018-01-08 19:45:25.105008 4 1 2 3 4 
> >> ring 2018-01-08 19:45:26.105076 5 2 3 4 5 1 
> >> ring 2018-01-08 19:45:27.105066 5 3 4 5 6 2 
> >> ring 2018-01-08 19:45:28.105133 5 4 5 6 7 3 
> >> ring 2018-01-08 19:45:29.105158 5 5 6 7 8 4 
> >> ring 2018-01-08 19:45:30.105079 5 6 7 8 9 5
> > 
> > 
> > 
> > On 01/08/2018 05:37 AM, Daniel Schick wrote:
> >> Hi all,
> >> 
> >> I am currently using epics base 3.16.1 and I have a problem with the compress record used as ring buffer.
> >> In both, Epics 3.16.1 and 7.0 the update of the ring buffer is not FIFO or LIFO but happens somewhere in the middle of the array.
> >> When I use the latest 3.15 branch everything works fine.
> >> 
> >> Here is an example of my test db-file and the output from camonitor:
> >> 
> >> record(calc,"counter") {
> >> field(CALC,"VAL+1")
> >> field(SCAN,"1 second")
> >> }
> >> 
> >> record(compress,"ring") {
> >> field(DESC, "test")
> >> field(NSAM,"10")
> >> field(ALG,"Circular Buffer")
> >> field(INP,"counter")
> >> field(SCAN,"1 second")
> >> field(BALG,"LIFO")
> >> field(PREC,"1")
> >> }
> >> 
> >> with epics 3.15 it is fine:
> >> 
> >> ~ $ camonitor ring
> >> ring *** Not connected (PV not found)
> >> ring 2018-01-08 12:55:16.628324 8 0 1 2 3 4 5 6 7 
> >> ring 2018-01-08 12:55:17.628331 9 0 1 2 3 4 5 6 7 8 
> >> ring 2018-01-08 12:55:18.628323 10 0 1 2 3 4 5 6 7 8 9 
> >> ring 2018-01-08 12:55:19.628330 10 1 2 3 4 5 6 7 8 9 10 
> >> ring 2018-01-08 12:55:20.628326 10 2 3 4 5 6 7 8 9 10 11 
> >> ring 2018-01-08 12:55:21.628331 10 3 4 5 6 7 8 9 10 11 12 
> >> ring 2018-01-08 12:55:22.628327 10 4 5 6 7 8 9 10 11 12 13 
> >> ring 2018-01-08 12:55:23.628324 10 5 6 7 8 9 10 11 12 13 14 
> >> ring 2018-01-08 12:55:24.628325 10 6 7 8 9 10 11 12 13 14 15 
> >> ring 2018-01-08 12:55:25.628332 10 7 8 9 10 11 12 13 14 15 16 
> >> ring 2018-01-08 12:55:26.628326 10 8 9 10 11 12 13 14 15 16 17 
> >> ring 2018-01-08 12:55:27.628330 10 9 10 11 12 13 14 15 16 17 18 
> >> ring 2018-01-08 12:55:28.628323 10 10 11 12 13 14 15 16 17 18 19 
> >> ring 2018-01-08 12:55:29.628325 10 11 12 13 14 15 16 17 18 19 20 
> >> ring 2018-01-08 12:55:30.628327 10 12 13 14 15 16 17 18 19 20 21 
> >> ring 2018-01-08 12:55:31.628331 10 13 14 15 16 17 18 19 20 21 22 
> >> ring 2018-01-08 12:55:32.628327 10 14 15 16 17 18 19 20 21 22 23
> >> 
> >> 
> >> with epics 3.16 and 7.0 I get the wired update behaviour:
> >> 
> >> ~ $ camonitor ring
> >> ring *** Not connected (PV not found)
> >> ring 2018-01-08 14:23:50.889595 2 0 1 
> >> ring 2018-01-08 14:23:51.889598 3 0 1 2 
> >> ring 2018-01-08 14:23:52.889594 4 0 1 2 3 
> >> ring 2018-01-08 14:23:53.889596 5 0 1 2 3 4 
> >> ring 2018-01-08 14:23:54.889603 6 0 1 2 3 4 5 
> >> ring 2018-01-08 14:23:55.889601 7 0 1 2 3 4 5 6 
> >> ring 2018-01-08 14:23:56.889594 8 0 1 2 3 4 5 6 7 
> >> ring 2018-01-08 14:23:57.889598 9 0 1 2 3 4 5 6 7 8 
> >> ring 2018-01-08 14:23:58.889592 10 6 7 8 9 0 1 2 3 4 5 
> >> ring 2018-01-08 14:23:59.889607 10 7 8 9 10 1 2 3 4 5 6 
> >> ring 2018-01-08 14:24:00.889598 10 8 9 10 11 2 3 4 5 6 7 
> >> ring 2018-01-08 14:24:01.889599 10 9 10 11 12 3 4 5 6 7 8 
> >> ring 2018-01-08 14:24:02.889596 10 10 11 12 13 4 5 6 7 8 9 
> >> ring 2018-01-08 14:24:03.889594 10 11 12 13 14 5 6 7 8 9 10 
> >> ring 2018-01-08 14:24:04.889598 10 12 13 14 15 6 7 8 9 10 11 
> >> ring 2018-01-08 14:24:05.889596 10 13 14 15 16 7 8 9 10 11 12 
> >> ring 2018-01-08 14:24:06.889597 10 14 15 16 17 8 9 10 11 12 13 
> >> ring 2018-01-08 14:24:07.889598 10 15 16 17 18 9 10 11 12 13 14 
> >> ring 2018-01-08 14:24:08.889595 10 16 17 18 19 10 11 12 13 14 15 
> >> ring 2018-01-08 14:24:09.889616 10 17 18 19 20 11 12 13 14 15 16 
> >> ring 2018-01-08 14:24:10.889598 10 18 19 20 21 12 13 14 15 16 17 
> >> ring 2018-01-08 14:24:11.889599 10 19 20 21 22 13 14 15 16 17 18 
> >> ring 2018-01-08 14:24:12.889595 10 20 21 22 23 14 15 16 17 18 19 
> >> 
> >> thanks and best wishes
> >> 
> >> Daniel
> >> ____________________
> >> 
> >> Dr. Daniel Schick
> >> 
> >> Max-Born-Institute
> >> for nonlinear optics
> >> and short pulse spectroscopy
> >> im Forschungsverbund Berlin e.V.
> >> 
> >> Max-Born-Str. 2a
> >> 12489 Berlin, Germany
> >> Room: House B, 3.3
> >> Tel.: +49 30 6392 1311
> >> 
> >
-- 
____________________

Dr. Daniel Schick

Max-Born-Institute
for nonlinear optics
and short pulse spectroscopy
im Forschungsverbund Berlin e.V.

Max-Born-Str. 2a
12489 Berlin, Germany
Room: House B, 3.3
Tel.: +49 30 6392 1311

References:
compress record FIFI/LIFO not working Daniel Schick
Re: compress record FIFI/LIFO not working Michael Davidsaver
Re: compress record FIFI/LIFO not working Heinz Junkes

Navigate by Date:
Prev: Re: EPICS Record to SCADA data exchange Heinz Junkes
Next: RE: EPICS 7 Compilation in windows-64 Freddie Akeroyd - UKRI STFC
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019 
Navigate by Thread:
Prev: Re: compress record FIFI/LIFO not working Heinz Junkes
Next: EPICS data extraction with mysql and compressed data Ghribi Adnan
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019 
ANJ, 13 Apr 2018 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·