| 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 <2025> | Index | 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 <2025> |
| <== Date ==> | <== Thread ==> |
|---|
| Subject: | Link error in MRF event system due to reference clock glitch |
| From: | "Hong, Ran via Tech-talk" <tech-talk at aps.anl.gov> |
| To: | "tech-talk at aps.anl.gov" <tech-talk at aps.anl.gov> |
| Date: | Fri, 14 Mar 2025 05:54:25 +0000 |
|
Hello Everyone, After commissioned the MRF event system at Advanced Photon Source at ANL for a year, we recently experienced some instabilities.
Occasionally, clients lose important events like the machine clock (P0) event, fast beam abort event, injection events, etc. The “Link Error” count of all event receivers incremented by 1. Further investigation showed that the event link
in the entire system was down for a few milliseconds, and then recover by itself.
In my previous studies, I found that such errors can be caused by sudden changes of the reference clock phase. When investigating these recent errors, I triggered a scope with the fast beam abort signal from the machine protection system
and found a glitch happened about 30 microseconds before the beam abort signal. Normally the clock period jitters about 40ps, while the glitch the clock period changed by +/-300 ps. I think it is very likely that such changes caused the event link error. Questions to other facilities that are using MRF event systems: have you seen similar glitches in your reference clock? If so, is there a way to make the MRF event master more resilient to such glitches? Thank you! Ran Hong Argonne National Lab, Advanced Photon Source |