Chris,
My only comment is that similar functionality is planned in CA V4
(ala my paper/poster from ICALEPCS 99 and my talk at the last
two collaboration meetings), and so there is some potential for
redundant code in the IOC. Perhaps there is some way that we can
combine our efforts?
Jeff
> -----Original Message-----
> From: [email protected] [mailto:[email protected]]
> Sent: Wednesday, November 17, 1999 5:43 PM
> Cc: [email protected]
> Subject: Re: Channel Archiving
>
>
> Hi all,
>
> [the following is fairly long and perhaps tedious,
> so here's the executive summary]
>
> I'm working on a data logger which will run on an
> ioc, compress channel data in real-time, and stream
> it over the network to a spooler. I hope that the
> resulting system can be generally useful to other
> sites, and so need some small amount of information
> and/or comments.
>
> [end executive summary, but there's some more good
> stuff at the bottom]
>
> (1) What is the fastest rate at which people want to
> log data from an ioc?
>
> (2) How much of this data is there?
>
> My preliminary investigation of how data behaves on
> the ioc's here at JLAB indicates that, if we consider
> each byte which comprises a signal's value as an
> independent quantity, the probability that any particular
> one of these will change in a ten minute period ranges
> between 0.005 and 0.05 (this is sampling at 1 hz).
> Futhermore, when a byte does change, it does so in a
> remarkably structured manner (see
> http://www.sura.org/~larrieu/Info.html).
> From an information theory standpoint, this means that
>
> (1) the observation that a signal has not changed
> carries very little information and can therefore
> be conveyed with minimal overhead.
>
> (2) when a byte does change, it will probably change
> to one of the predictable values, so this too can
> be encoded using very low bit rates.
>
> (3) when a byte changes to an unlikely value, then
> we need more bits to indicate such, though this
> happens relatively infrequently.
>
> Of course, when you know that a certain byte belongs
> to a certain signal, then you know more about its
> behavior over time. By monitoring this behavior we
> can build a fairly accurate statistical model of it
> so that we can further reduce the amount of information
> required to indicate a change in value.
>
> The whole point here is that I believe we can achieve
> dramatic results by compressing the channel data on the
> ioc, then sending the result over the wire to a simple
> spooling process.
>
> My intent is to:
>
> (1) implement this logging system
> (2) implement a client library for accessing the logged
> data, using Kay's C++ API.
> (3) build a new data extraction tool using (2)
>
> The benefits should be:
>
> (1) ability to store more information with less data
> (2) significantly reduced network traffic
> (3) circumvent CA size limit
> (4) compatible high-level interface, facilitating
> sharing of tools between participating labs.
>
> Trade-offs:
>
> (1) increased memory requirements on ioc
> (2) possibly significant CPU time on ioc
> (3) increased access time to archived data (decompress)
> (4) annoyance: another archiver?!
>
> Chris
> --
> Christopher A. Larrieu
> Jefferson Laboratory
> (757) 269-5097
>
- Replies:
- Re: Channel Archiving Christopher A. Larrieu
- References:
- Re: Channel Archiving Christopher A. Larrieu
- Navigate by Date:
- Prev:
Re: capfast symbol for transform and ANL motor record? Rozelle Wright
- Next:
R3.13.1 RISC Arch IOC BUG Report (assertion failure in dbEvent.c) Jeff Hill
- Index:
1994
1995
1996
1997
1998
<1999>
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: Channel Archiving Christopher A. Larrieu
- Next:
Re: Channel Archiving Christopher A. Larrieu
- Index:
1994
1995
1996
1997
1998
<1999>
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|