RE: time series circular buffer

[Hinko Kocevar <Hinko.Kocevar@esss.se>]

Thanks again Mark. I'll give it a run.

BTW, I noticed that in OPIs from ADCore 3-3-2, NDPluginBase.opi entry field next to 'Array address' wants to be of type String while it should be Decimal.

Best,
Hinko
 

-- 

Hinko Kocevar

Beam Diagnostics Engineer

European Spallation Source ERIC

P.O Box 176, SE-221 00 Lund, Sweden

Visiting address: Odarslövsvägen 113, 224 84 Lund

Mobile: +46-721-79 22 58

Phone:  +46-46-888 32 58

E-mail: hinko.kocevar@esss.se

Web:    www.esss.se

 

On 2018-09-20 18:32:51+02:00 Mark Rivers wrote:

I have made 2 more changes:

 

Added the NDPluginTimeSeries TSTimePerPointLink.DOL to autosave.

 

Changed all of the statistics time series plots  (NDStatsTimeSeriesPlot.adl, NDStatsTimeSeriesBasicAll.adl, NDStatsTimeSeriesCentroidAll.adl) so that they use the TSTimeAxis waveform for the horizontal axis.  The horizontal axis is thus now in units of seconds rather than point number.

 

I did the autoconversion of these screens to edm, caQtDM, and CSS-BOY.  Everything is on the master branch on Github.

 

Mark

 

 

From: Hinko Kocevar <Hinko.Kocevar@esss.se>
Sent: Thursday, September 20, 2018 10:19 AM
To: Mark Rivers <rivers@cars.uchicago.edu>; tom.cobb@diamond.ac.uk; tech-talk@aps.anl.gov
Subject: RE: time series circular buffer

 

Hi Mark,

thanks for fixing the issues!

Best,
Hinko
 

-- 

Hinko Kocevar

Beam Diagnostics Engineer

European Spallation Source ERIC

P.O Box 176, SE-221 00 Lund, Sweden

Visiting address: Odarslövsvägen 113, 224 84 Lund

Mobile: +46-721-79 22 58

Phone:  +46-46-888 32 58

E-mail: hinko.kocevar@esss.se

Web:    www.esss.se

 

On 2018-09-20 16:15:33+02:00 Mark Rivers wrote:

Hi Hinko,

> I get only two distinct timestamps displayed in NDPluginTimeseries plugin window. The Unique ID also toggles between two values. I have a test case with the following startup script:

I have fixed this.  The problem was that the NDPluginStats was not setting the uniqueId, timeStamp, and epicsTS of the time series NDArrays that it was passing to the NDPluginTimeSeries plugin.

The fix is in the master branch on Github.

I also found that the time series TimePerPointLink record is not autosave.  I will fix that today.

Mark

________________________________

From: Mark Rivers

Sent: Thursday, September 20, 2018 7:45 AM

To: 'Hinko Kocevar'; tom.cobb@diamond.ac.uk; tech-talk@aps.anl.gov

Subject: RE: time series circular buffer

Hi Hinko,

Ø  I get only two distinct timestamps displayed in NDPluginTimeseries plugin window. The Unique ID also toggles between two values. I have a test case with the following startup script:

I see something similar, although I see a larger number of values they are toggling between.  These are clearly the stale values of the UniqueID and timestamps from the NDArray it got from the NDArrayPool.  I will fix this.  The questions is what values should it use?  The values from last input array before the average was computed?

Ø  When changing the Time link field value (sets DOL), the Time/point RBV does not update. I have to use caput $(P)$(R):TSTimePerPointLink.PROC 1 to have it process.

If you want this value to update when the external value changes you need to use the CP attribute on the PV you enter into the link.  I just tested this using the following link and it worked fine: “13SIM1:cam1:AcquirePeriod_RBV CP”

TSNumAverage in the time-series plugin updated as it should when the simDetector AcquirePeriod was changed.

Mark

From: Hinko Kocevar <hinko.kocevar@esss.se>

Sent: Thursday, September 20, 2018 4:06 AM

To: Mark Rivers <rivers@cars.uchicago.edu>; tom.cobb@diamond.ac.uk; tech-talk@aps.anl.gov

Subject: RE: time series circular buffer

Hi Mark,

I'm trying to use the new stats and time series plugins. It got it to work, with two comments.

a)

I get only two distinct timestamps displayed in NDPluginTimeseries plugin window. The Unique ID also toggles between two values. I have a test case with the following startup script:

NDStatsConfigure("ST.LCR.CH$(C)", $(QSIZE), 0, "$(PORT)", $(C), 0, 0, 0, 0, $(MAX_THREADS=5))

dbLoadRecords("NDStats.template",       "P=$(PREFIX),R=STLCRCH$(N):,  PORT=ST.LCR.CH$(C),ADDR=0,TIMEOUT=1,HIST_SIZE=256,XSIZE=$(XSIZE),YSIZE=$(YSIZE),NCHANS=$(NCHANS),NDARRAY_PORT=$(PORT),NDARRAY_ADDR=$(C),TS_PORT=STTS.LCR.CH$(C)")

NDTimeSeriesConfigure("STTS.LCR.CH$(C)", $(QSIZE), 0, "ST.LCR.CH$(C)", 1, 23)

dbLoadRecords("NDTimeSeries.template",  "P=$(PREFIX),R=STLCRCH$(N):TS:, PORT=STTS.LCR.CH$(C),ADDR=0,TIMEOUT=1,NDARRAY_PORT=ST.LCR.CH$(C),NDARRAY_ADDR=1,NCHANS=$(NCHANS),TIME_LINK=$(PREFIX)TickValueR,ENABLED=1")

dbLoadRecords("NDTimeSeriesN.template", "P=$(PREFIX),R=STLCRCH$(N):TS:Min:, PORT=STTS.LCR.CH$(C),ADDR=0,TIMEOUT=1,NCHANS=$(NCHANS),ENABLED=1,NAME=Min")

dbLoadRecords("NDTimeSeriesN.template", "P=$(PREFIX),R=STLCRCH$(N):TS:Max:, PORT=STTS.LCR.CH$(C),ADDR=3,TIMEOUT=1,NCHANS=$(NCHANS),ENABLED=1,NAME=Max")

dbLoadRecords("NDTimeSeriesN.template", "P=$(PREFIX),R=STLCRCH$(N):TS:Ave:, PORT=STTS.LCR.CH$(C),ADDR=6,TIMEOUT=1,NCHANS=$(NCHANS),ENABLED=1,NAME=Average")

The stats plugin shows the proper Uniques ID and timestamp, same as the driver.

b)

When changing the Time link field value (sets DOL), the Time/point RBV does not update. I have to use caput $(P)$(R):TSTimePerPointLink.PROC 1 to have it process.

Thanks,

Hinko

--

Hinko Kocevar

Beam Diagnostics Engineer

European Spallation Source ERIC

P.O Box 176, SE-221 00 Lund, Sweden

Visiting address: Odarslövsvägen 113, 224 84 Lund

Mobile: +46-721-79 22 58

Phone:  +46-46-888 32 58

E-mail: hinko.kocevar@esss.se<mailto:hinko.kocevar@esss.se>

Web:    www.esss.se<http: www.esss.se="">

On 2018-06-26 10:16:24+02:00 tech-talk-bounces@aps.anl.gov<mailto:tech-talk-bounces@aps.anl.gov> wrote:

Hi Mark,

thank you for the fast solution! I'll give it run in the next weeks and report!

Best,

Hinko

________________________________________

From: Mark Rivers <rivers@cars.uchicago.edu>

Sent: Monday, June 25, 2018 10:30:10 PM

To: Hinko Kocevar; tech-talk@aps.anl.gov<mailto:tech-talk@aps.anl.gov>; tom.cobb@diamond.ac.uk<mailto:tom.cobb@diamond.ac.uk>

Subject: RE: time series circular buffer

Hi Hinko,

I have implemented the change to the NDPluginStats plugin using the second method I suggested:

- Move the time series code out of the statistics plugin and have the statistics plugin send the time points as NDArrays to time series plugins.

This has the benefit of reducing the code by 240 lines while adding the functionality of the Circular Buffer mode.

The only downside is that it breaks backwards compatibility.  The names of the PVs for the time-series arrays are unchanged, but the PVs to control the time-series (start/stop, number of time points, etc.) are slightly changed.  I think this is probably OK because I doubt if these are widely used in client scripts.

I have put the new version in this branch of ADCore on Github: https://github.com/areaDetector/ADCore/tree/stats_time_series.  There is a pending pull-request for it.  If you (or anyone else) wants to test and comment that would be welcome.

Thanks,

Mark

From: Hinko Kocevar <hinko.kocevar@esss.se>

Sent: Tuesday, June 19, 2018 8:28 AM

To: tech-talk@aps.anl.gov<mailto:tech-talk@aps.anl.gov>; tom.cobb@diamond.ac.uk<mailto:tom.cobb@diamond.ac.uk>; Mark Rivers &lt;rivers@cars.uchicago.edu&gt;

Subject: RE: time series circular buffer

Hi Mark,

sounds like both approaches might be acceptable. I guess it is about code duplication vs. number of different plugins one needs to get this feature, when it comes to deciding where to place this.

thanks,

hinko

--

Hinko Kocevar

Beam Diagnostics Engineer

European Spallation Source ERIC

P.O Box 176, SE-221 00 Lund, Sweden

Visiting address: Odarslövsvägen 113, 224 84 Lund

Mobile: +46-721-79 22 58

Phone:  +46-46-888 32 58

E-mail: hinko.kocevar@esss.se&lt;mailto:hinko.kocevar@esss.se&gt;<mailto:hinko.kocevar@esss.se&lt;mailto:hinko.kocevar@esss.se&gt>;

Web:    www.esss.se&lt;http<http: www.esss.se&lt;http="">: www.esss.se=""&gt;<http: www.esss.se="%22%22>">;

On 2018-06-16 01:07:11+02:00 Mark Rivers wrote:

Hi Hinko,

I think it would be useful to be able to run the time series in circular buffer mode.  There are 2 ways this might be done:

- Add the code for circular buffer mode to the statistics plugin

- Move the time series code out of the statistics plugin and have the statistics plugin send the time points as NDArrays to time series plugins.

I'm not sure which is best.

I have created an issue for this in ADCore.

Thanks,

Mark

-----Original Message-----

From: tech-talk-bounces@aps.anl.gov&lt;mailto:tech-talk-bounces@aps.anl.gov&gt;<mailto:tech-talk-bounces@aps.anl.gov&lt;mailto:tech-talk-bounces@aps.anl.gov&gt>; &lt;tech-talk-bounces@aps.anl.gov&lt;mailto:tech-talk-bounces@aps.anl.gov&gt;&amp;gt; On Behalf Of Hinko Kocevar

Sent: Friday, June 15, 2018 3:55 AM

To: tom.cobb@diamond.ac.uk&lt;mailto:tom.cobb@diamond.ac.uk&gt;<mailto:tom.cobb@diamond.ac.uk&lt;mailto:tom.cobb@diamond.ac.uk&gt>;; Tech-talk &lt;tech-talk@aps.anl.gov&lt;mailto:tech-talk@aps.anl.gov&gt;&amp;gt;

Subject: Re: time series circular buffer

Hi Tom,

thanks for the suggestion, I have not used this particular plugin so far. I'll give it a try!

Cheers,

Hinko

________________________________________

From: tom.cobb@diamond.ac.uk&lt;mailto:tom.cobb@diamond.ac.uk&gt;<mailto:tom.cobb@diamond.ac.uk&lt;mailto:tom.cobb@diamond.ac.uk&gt>; &lt;tom.cobb@diamond.ac.uk&lt;mailto:tom.cobb@diamond.ac.uk&gt;&amp;gt;

Sent: Friday, June 15, 2018 9:44:57 AM

To: Hinko Kocevar; Tech-talk

Subject: RE: time series circular buffer

Hi,

There is the NDPluginAttrPlot plugin which takes any numeric stamped NDAttributes (like the ones the NDPluginStats produces) and puts them in a circular buffer, periodically pushing the last N samples to a set of waveforms. We use it to plot arbitrary attributes against each other.

http://cars9.uchicago.edu/software/epics/NDPluginAttrPlot.html

Thanks,

Tom

> -----Original Message-----

&amp;gt; From: tech-talk-bounces@aps.anl.gov&lt;mailto:tech-talk-bounces@aps.anl.gov&gt;<mailto:tech-talk-bounces@aps.anl.gov&lt;mailto:tech-talk-bounces@aps.anl.gov&gt>; &lt;tech-talk-bounces@aps.anl.gov&lt;mailto:tech-talk-bounces@aps.anl.gov&gt;&amp;gt; On

> Behalf Of Hinko Kocevar

> Sent: 15 June 2018 07:38

> To: Tech-talk <tech-talk@aps.anl.gov<mailto:tech-talk@aps.anl.gov>>

> Subject: time series circular buffer

>

> Hi,

>

> in areaDetector NDPluginStats plugin I can get nice timeseries plots

> for max, mean, min.

> After the number of predefined points are collected the storing stops.

> Is there a way to configure timeseries buffer as a circular buffer?

> I've noticed that NDTimeSeries has this feature, but I'm failing to

> find one in NDPluginStats.

>

> Thanks,

>

> Hinko Kocevar

>

> Beam Diagnostics Engineer

> European Spallation Source ERIC

> P.O Box 176, SE-221 00 Lund, Sweden

> Visiting address: Odarslövsvägen 113, SE-224 84 Lund , Sweden

>

> Mobile: +46-721-79 22 58

> Phone:  +46-46-888 32 58

>

&amp;gt; E-mail: hinko.kocevar@esss.se&lt;mailto:hinko.kocevar@esss.se&gt;<mailto:hinko.kocevar@esss.se&lt;mailto:hinko.kocevar@esss.se&gt>;

&amp;gt; Web:    www.esss.se&lt;http<http: www.esss.se&lt;http="">: www.esss.se=""&gt;<http: www.esss.se="%22%22>">;

--

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.

Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.

Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.

Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

>>>>>

</http:></mailto:hinko.kocevar@esss.se></tech-talk@aps.anl.gov<mailto:tech-talk@aps.anl.gov></tech-talk-bounces@aps.anl.gov<mailto:tech-talk-bounces@aps.anl.gov></mailto:tech-talk-bounces@aps.anl.gov></tom.cobb@diamond.ac.uk<mailto:tom.cobb@diamond.ac.uk></mailto:tom.cobb@diamond.ac.uk></tech-talk@aps.anl.gov<mailto:tech-talk@aps.anl.gov></mailto:tom.cobb@diamond.ac.uk></tech-talk-bounces@aps.anl.gov<mailto:tech-talk-bounces@aps.anl.gov></mailto:tech-talk-bounces@aps.anl.gov></http:></mailto:hinko.kocevar@esss.se></rivers@cars.uchicago.edu></hinko.kocevar@esss.se></rivers@cars.uchicago.edu>

</http:></http:></mailto:hinko.kocevar@esss.se&lt;mailto:hinko.kocevar@esss.se&gt></mailto:tech-talk-bounces@aps.anl.gov&lt;mailto:tech-talk-bounces@aps.anl.gov&gt></mailto:tom.cobb@diamond.ac.uk&lt;mailto:tom.cobb@diamond.ac.uk&gt></mailto:tom.cobb@diamond.ac.uk&lt;mailto:tom.cobb@diamond.ac.uk&gt></mailto:tech-talk-bounces@aps.anl.gov&lt;mailto:tech-talk-bounces@aps.anl.gov&gt></http:></http:></mailto:hinko.kocevar@esss.se&lt;mailto:hinko.kocevar@esss.se&gt></mailto:tom.cobb@diamond.ac.uk></mailto:tech-talk@aps.anl.gov></mailto:tom.cobb@diamond.ac.uk></mailto:tech-talk@aps.anl.gov></mailto:tech-talk-bounces@aps.anl.gov></http:></mailto:hinko.kocevar@esss.se></rivers@cars.uchicago.edu></hinko.kocevar@esss.se>