Hi Michael,
Thanks for the explanation.
Another way to handle this could be for the areaDetector IOC to stream to an HDF5 file using the new Single Writer Multiple Reader (SWMR) support. Then your client could read from the file while the IOC is still writing to it. This is just a possible alternative to using Channel Access as the inter-process communication mechanism.
Mark
-----Original Message-----
From: Michael Huth [mailto:[email protected]]
Sent: Tuesday, June 06, 2017 1:47 PM
To: Mark Rivers; [email protected]
Subject: Re: Channel Access client sees only partial update.
Hello Mark,
thanks for the hint.
The instrument streams via USB3 (in theory up to 400 MB/s) into it's own
buffer. So the data set is of variable size and could grow fast.
Probably ~100 MB minimum and may be up to 10 GB+.
I wanted to split it into blocks to be able to find an optimal size for
fastest transfer to the channel access client. It might also prevent
long pauses from large memory allocations.
Best regards,
Michael
On 06.06.2017 20:28, Mark Rivers wrote:
> Hi Michael,
>
> By default the devAsynInt8Array callback does not buffer the array it receives, it copies it directly into the waveform record. If your driver callbacks are happening faster than the record processing and Channel Access transfer to the client then what you are seeing is expected.
>
> devAsynInt8Array can optionally use a ring-buffer to buffer the arrays passed to the callback function. This should eliminate the problem you are seeing. The ring buffer is disabled by default, but you can enable it by adding this line to your waveform record database file:
>
> info(asyn:FIFO, "20")
>
> where 20 is the maximum number of blocks you want to be able to buffer.
>
> This is documented here (search for "Buffering of Driver Callbacks"):
> http://www.aps.anl.gov/epics/modules/soft/asyn/R4-31/asynDriver.html
>
> How large is the data set you are trying to transfer? Are you breaking it into blocks because of its size, or is your source actually delivering it in sub-blocks?
>
> Mark
>
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of Michael Huth
> Sent: Tuesday, June 06, 2017 1:15 PM
> To: [email protected]
> Subject: Channel Access client sees only partial update.
>
> Hello,
>
> I'am quite new to Epics and ran into an issue where I could use a hint
> how to solve it:
>
> I have an IOC driver based on the area detector driver that should be
> extended for the transfer of a large data set from another source.
>
> So I setup a Record for it like this:
> record(waveform, "$(P)$(R)Data")
> {
> field(PINI, "0")
> field(DTYP, "asynInt8ArrayIn")
> field(INP, "@asyn($(PORT),$(ADDR),$(TIMEOUT))DATA")
> field(FTVL, "CHAR")
> field(NELM, "$(SIZE_DATA_BLOCK)")
> field(SCAN, "I/O Intr")
> }
>
> In the main thread of the IOC I split the large data set into smaller
> blocks and have a loop that calls
>
> doCallbacksInt8Array(blockPtr , block_size, Data, 0);
>
> for each block to transfer the blocks to the data base.
>
> In a second process a channel access client is subscribed with
> ca_create_subscription to 'DATA'.
>
> The issue is that the client receives only the last block from each loop
> of the IOC driver thread.
>
> I guess I have to wait somehow until the database processing has
> finished after each block?!?
>
> Best regards,
> Michael Huth
>
--
() byte physics
Dr. Michael Huth
Software-Developer
Company CEO: Thomas Braun
Address: Schwarzastraße 9, 12055 Berlin, Germany
Web: www.byte-physics.de
Telephone: +49 (0)30/52 68 23 22
Fax: +49 (0)30/52 68 43 26
USt-IdNr: DE276717536
- References:
- Channel Access client sees only partial update. Michael Huth
- RE: Channel Access client sees only partial update. Mark Rivers
- Re: Channel Access client sees only partial update. Michael Huth
- Navigate by Date:
- Prev:
Re: Channel Access client sees only partial update. Michael Huth
- Next:
Re: Channel Access client sees only partial update. Michael Huth
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
<2017>
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: Channel Access client sees only partial update. Michael Huth
- Next:
Re: Channel Access client sees only partial update. Michael Huth
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
<2017>
2018
2019
2020
2021
2022
2023
2024
|