FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 07-02-2012, 04:50 PM
Michael Cronenworth
 
Default Leap Second

Steven Stern wrote:
> No problem on two Fedora systems, two Centos 5 systems and two Centos 6
> systems. What application went wacky for you?

Desktop:
Firefox/Thunderbird

Server:
VirtualBox
(8 vm systems on 8 core server, had >13.00 load!)
Bind

Everything worked like nothing was wrong, but they were using all of the
system's CPU.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 07-02-2012, 05:21 PM
David
 
Default Leap Second

On 7/2/2012 11:25 AM, Michael Cronenworth wrote:
> Hi all,
>
> I recommend that anyone not familiar with the term "leap second" check
> out all of their Linux systems. Most likely a piece of software is
> running in an infinite loop due to the added second on July 1st. Your
> system may also appear to be running normally but double-check your
> system load to make sure it is less than 1.00. I had several affected
> systems so Fedora was not ready (and I didn't bother to ready my systems).
>
> If you have high system load there are two solutions:
> 1. Reboot, or...
> 2. Manually set the date with "date". Ex: "date 07021025" for July 2nd,
> 10:25 AM.
>
> FYI,
> Michael
>


"Wait just a (leap) second"

<http://blogs.discovermagazine.com/badastronomy/2012/01/23/wait-just-a-leap-second/>

--

David
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 07-02-2012, 05:37 PM
Peter Eckel
 
Default leap second

Hi Les,

> Interesting, but I thought that ntp clients always advanced the clock
> by small fractions of a second anyway even when the master source
> differs by more.

they do. But the leap second is quite a different thing: Actually the time doesn't really diverge from the server's, but the stratum 1 server deliveres a totally unexpected 01:59:60, and the stratum 2 server follows.

The Google approach is not to use that time at all, but slow the clock down a bit on the stratum 2 server so that the stratum 1 (that has the 'genuine' time and jumps to the :60 time stamp after :59) is, after the time window is over, about one second ahead of the stratum 2. So approximately the instant when the stratum 1 server jumps from :59 to :60, the stratum 2 server jumps from :58 to :59, and at the next second tick, they will both jump to 02:00:00 and be in synch again. The same approach works with a negative leap second, which was never needed yet, however.

The disadvantage of this method is that you have to know in advance when the leap second will happen, which requires tables that regularly have to be updated since it is fairly unpredictable in the long run when a leap second will be necessary. I don't know why they didn't simply use the 'LI' bit in the NTP protocol to determine when to start 'smearing' - at least the article doesn't say they did:

<http://www.networksorcery.com/enp/protocol/ntp.htm>

Maybe 24 hours notification in advance did not seem long enough for the smear interval. I doubt it, because I would not really like the time to differ from the real time for more than a day.

Best regards,

Peter.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 07-02-2012, 05:44 PM
M A Young
 
Default Leap Second

On Mon, 2 Jul 2012, Michael Cronenworth wrote:


Hi all,

I recommend that anyone not familiar with the term "leap second" check
out all of their Linux systems. Most likely a piece of software is
running in an infinite loop due to the added second on July 1st. Your
system may also appear to be running normally but double-check your
system load to make sure it is less than 1.00. I had several affected
systems so Fedora was not ready (and I didn't bother to ready my systems).

If you have high system load there are two solutions:
1. Reboot, or...
2. Manually set the date with "date". Ex: "date 07021025" for July 2nd,
10:25 AM.


Running
date `date +%m%d%H%M.%S`
should reset the date without changing the time too much.

I had trouble on RHEL6 servers, which were mostly running java, with
sluggish reponse and high cpu loads. I ended up rebooting them to make
sure I cleared the problem.


Michael Young
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org
 
Old 07-02-2012, 05:45 PM
Stephen Harris
 
Default leap second

On Mon, Jul 02, 2012 at 07:37:46PM +0200, Peter Eckel wrote:
> Maybe 24 hours notification in advance did not seem long enough for
> the smear interval. I doubt it, because I would not really like the time
> to differ from the real time for more than a day.

Yeah, there are some regularity requirements in some industries that the
server clocks are within 100ms of UTC (or, at least, that's how internal
audit have interpreted the regulations where I work).

Allowing the clock to drift by a second would normally be bad, but I
guess it wouldn't matter on a non-business day. (Well, non-business
for 95% of the company where this matters - some areas were still open).

--

rgds
Stephen
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 07-02-2012, 06:32 PM
Gordon Messmer
 
Default leap second

On 07/02/2012 09:24 AM, Peter Eckel wrote:
> On the other hand I'm a bit surprised that the problems were
> comparably few - actually there is a time '01:59:60' for one second,
> and any plausibility check I've ever seen assumes that minutes and
> seconds are in the range from 0..59. Wrongly, it seems.

As far as I've been able to understand it, the problem had nothing to do
with validity checks or other date handling code. The problem was
simply a bug in the API provided by the Linux kernel for notification of
leap seconds. The kernel messed up some internal data that led to
futexes going nuts. The affected programs weren't handling dates
poorly, they were just threaded applications.

> Apparently Google uses an approach that looks much less risky to me -
> they use a time window over which they 'smear' the leap second by
> making their time servers lie about the time for a while, making it
> pass a little bit slower. That way they avoid the unlucky 61st second
> and still advance the clocks within a reasonable time.

Google's approach was reliable by chance. They used a different kernel
API to adjust the clock, and that one didn't break futexes.

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 07-02-2012, 06:45 PM
Les Mikesell
 
Default leap second

On Mon, Jul 2, 2012 at 1:32 PM, Gordon Messmer <yinyang@eburg.com> wrote:
> On 07/02/2012 09:24 AM, Peter Eckel wrote:
>> On the other hand I'm a bit surprised that the problems were
>> comparably few - actually there is a time '01:59:60' for one second,
>> and any plausibility check I've ever seen assumes that minutes and
>> seconds are in the range from 0..59. Wrongly, it seems.
>
> As far as I've been able to understand it, the problem had nothing to do
> with validity checks or other date handling code. The problem was
> simply a bug in the API provided by the Linux kernel for notification of
> leap seconds. The kernel messed up some internal data that led to
> futexes going nuts. The affected programs weren't handling dates
> poorly, they were just threaded applications.
>
>> Apparently Google uses an approach that looks much less risky to me -
>> they use a time window over which they 'smear' the leap second by
>> making their time servers lie about the time for a while, making it
>> pass a little bit slower. That way they avoid the unlucky 61st second
>> and still advance the clocks within a reasonable time.
>
> Google's approach was reliable by chance. They used a different kernel
> API to adjust the clock, and that one didn't break futexes.

So it wasn't anything special about java? I did find one one
not-very-busy instance of a Centos 6.x with a java application still
running that did not appear to have a problem.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 07-02-2012, 06:55 PM
Gordon Messmer
 
Default leap second

On 07/02/2012 10:37 AM, Peter Eckel wrote:
>
> they do. But the leap second is quite a different thing: Actually the
> time doesn't really diverge from the server's, but the stratum 1
> server deliveres a totally unexpected 01:59:60, and the stratum 2
> server follows.

That's not quite correct. The NTP protocol (as you mentioned later)
actually indicates that the current day should include a leap second,
the NTP server notifies the kernel that the day should include a leap
second, and the kernel inserts the leap second at the end of the day by
extending the duration of one of the system clock's seconds.

The "60" second doesn't exist in NTP or in the POSIX system clock, both
of which are counters from their respective epochs. The "60" second is
present only in time representations that are converted from the system
clock or NTP clock.

http://www.eecis.udel.edu/~mills/leap.html

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 07-02-2012, 07:10 PM
Gordon Messmer
 
Default leap second

On 07/02/2012 11:45 AM, Les Mikesell wrote:
> So it wasn't anything special about java? I did find one one
> not-very-busy instance of a Centos 6.x with a java application still
> running that did not appear to have a problem.

Only that java applications tend to be threaded, and threaded
applications were the ones likely to be affected by the bug.

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 07-02-2012, 07:54 PM
Les Mikesell
 
Default leap second

On Mon, Jul 2, 2012 at 2:10 PM, Gordon Messmer <yinyang@eburg.com> wrote:
> On 07/02/2012 11:45 AM, Les Mikesell wrote:
>> So it wasn't anything special about java? I did find one one
>> not-very-busy instance of a Centos 6.x with a java application still
>> running that did not appear to have a problem.
>
> Only that java applications tend to be threaded, and threaded
> applications were the ones likely to be affected by the bug.
>

Sooo... Are the 6.x boxes that did not exhibit a problem yet still
likely to have it if you start a threaded program or did it have to
happen in the 1 second window?

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 07:41 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org