Hi Patrick,
> Would it be possible to extend this, or something similar, to monitor and
> record changes for tens or hundreds of thousands of PVs in a mySQL database?
I think it is possible. I'm currently archiving ~5000 PVs, but that's
not limited by what the system can handle, but just how many PVs we
actually want to monitor. But to be honest, the majority of the PVs
we monitor change very slowly.
My concern for scaling up would be that you might really want to use
multiple threads or processes. Currently, the EpicsArchiver code uses
the older EpicsCA python interface, which is single-threaded and uses
non-preemptive callbacks. I have not yet converted this to pyepics3
(which lifts both restrictions), though that's on my ToDo list. I
believe that allowing multiple collection threads could be be
important for handling a "huge" number of PVs, though I'm not sure
what "huge" is.
I think that whether mysql can handle the load is mostly a question of
hardware and mysql configuration, but I doubt that mysql would be the
limiting factor.
Cheers,
--Matt Newville <newville at cars.uchicago.edu> 630-252-0431
- References:
- archive server discussion james.rowland
- Re: archive server discussion Matt Newville
- Re: archive server discussion Patrick Thomas
- Navigate by Date:
- Prev:
medm Mezger Anton-Chr.
- Next:
caPutLog Performance Gary Carr
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
<2010>
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
- Navigate by Thread:
- Prev:
Re: archive server discussion Patrick Thomas
- Next:
archive server discussion Loris Giovannini
- Index:
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
<2010>
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
|