Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
<== Date ==> <== Thread ==>

Subject: which time/timestamp format to use for an EPICS facility?
From: "Konrad, Martin" <konrad@frib.msu.edu>
To: EPICS Tech Talk <tech-talk@aps.anl.gov>
Date: Tue, 3 Dec 2013 18:34:44 +0000
Hi,
I'm working on the Global Timing System for FRIB. In this context I'm 
wondering which time we should use for the whole facility. This question 
  is about two things:

1. the time we want to use (something like local time, UTC, GPS time, TAI)
2. the representation of this time in a computer/FPGA/PLC/IOC/...

We still have a few years until FRIB will be in operation so we have the 
opportunity to do things right from the beginning. I'd appreciate to 
hear your thoughts about this topic since this seems to be a critical 
design decision.


Regarding 1.:

In my eyes using daylight savings time is calling for trouble. Think 
about an archiver that has to deal with gaps and getting samples for the 
same hour twice. This is also about all the clients, scripts, and 
operators that need to keep this in mind.

Using local (winter) time might be ok for the EPICS environment itself. 
But it will be difficult to tell which of two firmware files generated 
in different time zones is newer. And we probably do want to use the 
same time for everything...

UTC solves these problems but has leap seconds. Unfortunately they are 
only known a few months in advance which makes it difficult to build 
them into computers not to mention a distributed control system.

If you want to get rid of leap seconds you can go for GPS time [1] or 
International Atomic Time (TAI, [2]). Both do not have leap seconds and 
thus slowly diverge from UTC (right now GPST = UTC + 16 s, TAI = UTC + 
35 s).

TAI and GPST seem to be ideal to me for an accelerator control system. 
They can be easily derived with high precision from GPS clocks. The 
downside is that operators have to work with different times on their 
GUI and their watch.


Regarding 2:

It seems like most of our current systems use POSIX timestamps or EPICS 
timestamps for internal representation of time. Unfortunately they both 
come with several issues:

1. 32 bit POSIX timestamps will overflow in 2038 [3] which is within the 
planned life time of FRIB. EPICS timestamps are very similar but use a 
different epoch and thus can be used until 2058.

2. POSIX as well as EPICS timestamps do not provide a representation for 
leap seconds (although the clock is running on UTC by definition). Thus 
one of the following ugly work-arounds is needed:

a) Wait for the clock to be off one second and compensate for that using 
NTP after it happened. Not acceptable since every machine corrects at 
its own speed.
b) Make the preceding second last two seconds by stopping the clock for 
one second. This is done by systems running recent NTP daemons and 
kernels. Since these systems do not have a correct time during leap 
seconds they cheat by simply providing _some_ time that they think 
_might_ make sense. There is no guarantee that you will get the same 
timestamp at the same time on multiple machines (even if they run the 
same OS!). We also have to make sure all our software can handle twice 
as much samples per second etc. Probably not acceptable.
c) Start some time earlier and stretch seconds a little ("smear" out the 
leap second). This approach is used by Google for their servers [4]. If 
implemented in the right way this should happen synchronously on all 
machines. This probably cannot be done with PLCs.

3. Leap seconds make it much more difficult to calculate the time 
elapsed between two timestamps (which is what nuclear physicists usually 
do when working with their data). We need to make sure we have correct 
leap-second data available for all software that calculates time 
differences. This seems to be practically impossible to me.

It seems to be a _really_ big task to determine if all system on our 
accelerator network handle leap seconds correctly or even the same way.


My proposal/solution:

Use TAI for the whole accelerator network including all EPICS IOCs, 
PLCs, control room PCs etc. Force operators, users, logbooks etc. to use 
TAI. Keep using local time for office PCs and data center equipment.

Use TAI64 timestamps [5] where possible. Convert to POSIX/EPICS 
timestamps where needed. The latter should work smoothly until 2038 
since TAI has no leap seconds. Control System Studio's PVManager already 
uses 64 bit timestamps internally. Since most state-of-the-art CSS 
applications use PVManager I don't see any fundamental problems on this end.

To use TAI we have to run our own PTP and NTP servers and prevent the 
latter from announcing leap seconds. We need to block access to NTP 
servers outside of the accelerator network.

There are still some open questions:

What effects do different timestamps have on network connections that 
cross the office/accelerator network boundary? Some things that come to 
my mind:

1. Encrypted connections might cause problems because system time is 
used to protect against replay attacks. Usually a time difference of one 
or two minutes should be acceptable, though.
2. Functions of GUI software sending the current system time via the 
network might result in undesired behavior if used from the office 
network. E.g. Control System Studio Archive Viewer's "-10 minutes to 
now" probably will not work as expected.

What else do we need to keep in mind? Can anyone provide some experience 
with running a control system on a time that is different from local 
time/UTC?

Thanks for reading this long one!

Martin

[1] 
http://en.wikipedia.org/w/index.php?title=Global_Positioning_System&oldid=584232968#Timekeeping
[2] 
http://en.wikipedia.org/w/index.php?title=International_Atomic_Time&oldid=581789205
[3] http://en.wikipedia.org/w/index.php?title=Unix_time&oldid=583201834
[4] http://www.aps.anl.gov/epics/tech-talk/2011/msg01579.php
[5] http://cr.yp.to/libtai/tai64.html

-- 
Martin Konrad
Control System Engineer
Facility for Rare Isotope Beams
Michigan State University
640 South Shaw Lane
East Lansing, MI 48824-1321, USA
Tel. 517-908-7253
Email: konrad@frib.msu.edu


Replies:
RE: which time/timestamp format to use for an EPICS facility? Emmanuel Mayssat

Navigate by Date:
Prev: RE: EpicsSharp issues Christopher J. Pendleton
Next: RE: EpicsSharp issues Christopher J. Pendleton
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
Navigate by Thread:
Prev: RE: Microlab 500 Syringe Pump Jemian, Pete R.
Next: RE: which time/timestamp format to use for an EPICS facility? Emmanuel Mayssat
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  <20132014  2015  2016  2017  2018  2019 
ANJ, 20 Apr 2015 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·