EPICS Controls Argonne National Laboratory

Experimental Physics and
Industrial Control System

1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020  2021  2022  2023  2024  Index 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020  2021  2022  2023  2024 
<== Date ==> <== Thread ==>

Subject: RE: Weird IOC behavior with NTP time shift
From: Abdalla Ahmad via Tech-talk <[email protected]>
To: Michael Davidsaver <[email protected]>
Cc: "[email protected]" <[email protected]>
Date: Mon, 24 Dec 2018 08:18:21 +0000
Hi Michael

Thank you that was quite helpful. I still did not understand the relation that time counter need to be monotonic. You mentioned you are working on a true monotonic time function, doesn't that will have the same behavior? Or in other words, what do you mean by true monotonic time source?

Best Regards,
Abdalla.

-----Original Message-----
From: Michael Davidsaver [mailto:[email protected]] 
Sent: Monday, December 24, 2018 6:08 AM
To: Abdalla Ahmad <[email protected]>
Cc: [email protected]
Subject: Re: Weird IOC behavior with NTP time shift

On 12/23/18 5:55 AM, Abdalla Ahmad via Tech-talk wrote:
> Hi
> 
>  
> 
> This issue happened with base 3.14.12.3 and 3.15.6. What happened is that the IT group upgraded their MS windows active directory servers, one of the included services in the update is the NTP. When all the servers rebooted, the NTP service did not start and caused a 3-hour time shift forward in time on all connected PCs and hosts.
> 
>  
> 
> On the IOC side, when the shift happened the IOC date command returns the host's date/time. But when the problem was resolved by the IT group, the time got synchronized and went 3-hour backward in time. At this moment the date command returns the same timestamp, i.e., a frozen time stamp. We had to reboot every IOC to get the machine working.
> 
>  
> 
> My question is, is there any explanation for this behavior and any 
> potential way to avoid it? Also on daylight saving times, we can't 
> recall this issue happening on a DST switch before most probably 
> because the machine was OFF, will DST switch give the same effect? The 
> issue is reproducible with any IOC, whenever the IOC's timestamp is 
> ahead of the host's time stamp, the IOC's timestamp will get frozen,

What you describe is a known issue with IOC time handling which arises from the (imo contradictory) requirement that epicsTimeGetCurrent() be monotonic.
The result is that after real time jumps backwards* calls to
epicsTimeGetCurrent() will return the same value until the OS clock "catches up" with it's previous value.

https://github.com/epics-base/epics-base/blob/5f46d6dceed6c10c43ed5db0543979d696f1becd/src/libCom/osi/epicsGeneralTime.c#L162

As you've found, epicsTimeGetCurrent() is used internally in several places, most notably in timer queues.  So the effects are quite noticeable...

https://bugs.launchpad.net/epics-base/+bug/1392516

I have a (very) long term plan to switch the timer queue and similar to a true monotonic time source.  Such a time source is available as of Base 7.0.1 (cf. epicsMonotonicGet() ), so the next step is to replace usages of epicsTimeGetCurrent() and epicsTime::getCurrent().


* Daylight savings time isn't implemented as a jump of the seconds counter.
  Rather it is a change in the mapping between the seconds counter and
  calendar/time of day.

Replies:
Re: Weird IOC behavior with NTP time shift Michael Davidsaver via Tech-talk
References:
Weird IOC behavior with NTP time shift Abdalla Ahmad via Tech-talk
Re: Weird IOC behavior with NTP time shift Michael Davidsaver via Tech-talk

Navigate by Date:
Prev: Re: Weird IOC behavior with NTP time shift Michael Davidsaver via Tech-talk
Next: Re: Weird IOC behavior with NTP time shift Michael Davidsaver via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020  2021  2022  2023  2024 
Navigate by Thread:
Prev: Re: Weird IOC behavior with NTP time shift Michael Davidsaver via Tech-talk
Next: Re: Weird IOC behavior with NTP time shift Michael Davidsaver via Tech-talk
Index: 1994  1995  1996  1997  1998  1999  2000  2001  2002  2003  2004  2005  2006  2007  2008  2009  2010  2011  2012  2013  2014  2015  2016  2017  <20182019  2020  2021  2022  2023  2024 
ANJ, 24 Dec 2018 Valid HTML 4.01! · Home · News · About · Base · Modules · Extensions · Distributions · Download ·
· Search · EPICS V4 · IRMIS · Talk · Bugs · Documents · Links · Licensing ·