Re: Maximum archival rate

[Maren Purves <m.purves@jach.hawaii.edu>]

Kay,

(only responding to the data volume problem)

On Thu, 29 Mar 2007, Kay-Uwe Kasemir wrote:
> On Mar 29, 2007, at 02:04 , Terry Cornall wrote:
> > I'm trying to dimension the archive requirements for all the 14 beamlines 
> > for the Australian Synchrotron and am wondering if I can do it
> > with one PC or if I need many. Of course, sampling or monitoring rates will 
> > play a big part, but also I need to know what is possible per archive 
> > engine.
>
> Problem 2:
> The amount of data. One 'double' sample uses a little over
> 20 bytes for the timestamp, status, sev, value.
> For many values, the data file structure and index add
> relatively little to that, but you'll still get about 20GB
> per day. How do you intend to back that up?

[...]

> If somebody tells you that disks are cheap, please ask
> that person to take care of your archiving, then run
> as fast as you can.

you got a point, but I'm not sure that's what Terry asked.

And besides, with our wide field camera (motor control is
EPICS, array control and data acquisition isn't) that has
4 arrays we get over 50GB per array in a good night with
short integrations. With all 4 arrays we can get well over
a Terabyte in a week.

Yes, it's an archiving problem, but it's entirely doable.
LTO 2 drives in our case generally hold about 300 GB of
compressed data (depends on how compressible your data
are, that's why 'in our case').

Aloha,
Maren