Jeff Hill wrote:
>
> Chris,
>
> My only comment is that similar functionality is planned in CA V4
> (ala my paper/poster from ICALEPCS 99 and my talk at the last
> two collaboration meetings), and so there is some potential for
> redundant code in the IOC. Perhaps there is some way that we can
> combine our efforts?
Certainly! I hadn't realized that you were actually working
on this already, and thought that it was a "gee it would be nice
if someone would do this" sort of deal. As I am just starting,
perhaps we should discuss our ideas and try to synthesize some-
thing from them. Because this work is precipitated by course-
related research, my immediate goals are a precise understanding
of the problem and a formulation of the solution, with the
implementation being secondary; once I've arrived at this under-
standing, though, the implementation should follow in a fairly
straightforward fashion (really, I have no naive expectation that
it will be *easy*)...
Chris
>
> Jeff
>
> > -----Original Message-----
> > From: [email protected] [mailto:[email protected]]
> > Sent: Wednesday, November 17, 1999 5:43 PM
> > Cc: [email protected]
> > Subject: Re: Channel Archiving
> >
> >
> > Hi all,
> >
> > [the following is fairly long and perhaps tedious,
> > so here's the executive summary]
> >
> > I'm working on a data logger which will run on an
> > ioc, compress channel data in real-time, and stream
> > it over the network to a spooler. I hope that the
> > resulting system can be generally useful to other
> > sites, and so need some small amount of information
> > and/or comments.
> >
> > [end executive summary, but there's some more good
> > stuff at the bottom]
> >
> > (1) What is the fastest rate at which people want to
> > log data from an ioc?
> >
> > (2) How much of this data is there?
> >
> > My preliminary investigation of how data behaves on
> > the ioc's here at JLAB indicates that, if we consider
> > each byte which comprises a signal's value as an
> > independent quantity, the probability that any particular
> > one of these will change in a ten minute period ranges
> > between 0.005 and 0.05 (this is sampling at 1 hz).
> > Futhermore, when a byte does change, it does so in a
> > remarkably structured manner (see
> > http://www.sura.org/~larrieu/Info.html).
> > From an information theory standpoint, this means that
> >
> > (1) the observation that a signal has not changed
> > carries very little information and can therefore
> > be conveyed with minimal overhead.
> >
> > (2) when a byte does change, it will probably change
> > to one of the predictable values, so this too can
> > be encoded using very low bit rates.
> >
> > (3) when a byte changes to an unlikely value, then
> > we need more bits to indicate such, though this
> > happens relatively infrequently.
> >
> > Of course, when you know that a certain byte belongs
> > to a certain signal, then you know more about its
> > behavior over time. By monitoring this behavior we
> > can build a fairly accurate statistical model of it
> > so that we can further reduce the amount of information
> > required to indicate a change in value.
> >
> > The whole point here is that I believe we can achieve
> > dramatic results by compressing the channel data on the
> > ioc, then sending the result over the wire to a simple
> > spooling process.
> >
> > My intent is to:
> >
> > (1) implement this logging system
> > (2) implement a client library for accessing the logged
> > data, using Kay's C++ API.
> > (3) build a new data extraction tool using (2)
> >
> > The benefits should be:
> >
> > (1) ability to store more information with less data
> > (2) significantly reduced network traffic
> > (3) circumvent CA size limit
> > (4) compatible high-level interface, facilitating
> > sharing of tools between participating labs.
> >
> > Trade-offs:
> >
> > (1) increased memory requirements on ioc
> > (2) possibly significant CPU time on ioc
> > (3) increased access time to archived data (decompress)
> > (4) annoyance: another archiver?!
> >
> > Chris
> > --
> > Christopher A. Larrieu
> > Jefferson Laboratory
> > (757) 269-5097
> >
--
Christopher A. Larrieu
Jefferson Laboratory
(757) 269-5097
- References:
- RE: Channel Archiving Jeff Hill
- Navigate by Date:
- Prev:
R3.13.1 RISC Arch IOC BUG Report (assertion failure in dbEvent.c) Jeff Hill
- Next:
Re: capfast symbol for transform and ANL motor record? Benjamin Franksen
- Index:
1994
1995
1996
1997
1998
<1999>
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
RE: Channel Archiving Jeff Hill
- Next:
Re: Channel Archiving Garrett D. Rinehart
- Index:
1994
1995
1996
1997
1998
<1999>
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|