Dear Kay,
Thank you very much for answering my question!
When I tried to archive only 90 channels, sometimes the "Write State" shows
"Write Error". When the state is ok, the write count is about 900 samples
and the write duration is only 0.4 sec.
But when I tried to archive more channels, the "Write State" always shows
"Write Error".
The Memory is about 6.2%, so I guess it is not memory issue.
Any ideas/suggestions would be deeply appreciated.
Caizhen
-----Original Message-----
From: Kasemir, Kay [mailto:[email protected]]
Sent: Friday, October 14, 2016 8:23 AM
To: Caizhen Liu; [email protected]
Subject: Re: archive engine
Hi:
>We have about 4000 PVs being archived at the moment.
> It seems that the archiving have some issue starting from PV
?OPS-EPICS:R:CCG1?.
> The queue is full and overruns happens. And there is no data record for PV
OPS-EPICS:R:CCG1 in the postgres database.
> When I reduced the number of channels to be 90, everything seems ok.
More important than the number of PVs or the specific PV names is the number
of samples that these channels produce.
The main web page of the archive engine will show you a "write count" and a
"write duration".
From watching that for a while, you can compute the number of values that
the engine can write to your relational database.
Bottom line is that the RDB approach for archiving is very reliable, but not
fast.
You may simply reach some maximum supported number of samples/sec that you
can write to your database.
When you build CSS from sources, you can execute the RDBArchiveWriterTest
(https://github.com/ControlSystemStudio/cs-studio/blob/master/applications/a
rchive/archive-plugins/org.csstudio.archive.writer.test/test/org/csstudio/ar
chive/writer/rdb/RDBArchiveWriterTest.java).
Way back, I got these types of performance readings:
PostgreSQL: ~12000 samples/sec
MySQL: ~21000 samples/sec
Those are write speeds when the database is on the same computer that's also
running the test, and nothing else is using the RDB.
In the real world, when you connect to a remote database that's also doing
its day job, the numbers a lower.
The EPICS Archiver Appliance,
https://slacmshankar.github.io/epicsarchiver_docs/index.html, is faster
because it uses its own set of files instead of a relational database.
-Kay
- Replies:
- Re: archive engine Kasemir, Kay
- References:
- FW: archive engine Caizhen Liu
- Re: archive engine Kasemir, Kay
- Navigate by Date:
- Prev:
RE: Question about epics and pcaspy running in one computer Mark Rivers
- Next:
Re: archive engine Kasemir, Kay
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
<2016>
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: archive engine Kasemir, Kay
- Next:
Re: archive engine Kasemir, Kay
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
<2016>
2017
2018
2019
2020
2021
2022
2023
2024
|