A fix for the Java Leap Second bug

Mattias Geniar, Monday, July 2, 2012 - last modified: Monday, June 29, 2015

Java + the leap second that occurs every 18 months = pain. Not that Debian Squeeze is that much better, though. The patch was provided a few months in advance, but most vendors package a custom java version that doesn't include that fix.

It starts with entries in your logs that are similar to these.

Clock: inserting leap second 23:59:60 UTC

And after that, your Java processes eat up more and more CPU causing a continued rise of system load, usually until things just come to a halt.

The fix is something like this.

$ /etc/init.d/ntpd stop
$ date -s "`date`"

Why does this work, you may wonder?

It seems that the bug in the leap second code caused a glitch in how the time is to be changed on the server. It's missing a vital function call that 'unblocks' the time.

By setting the date again with the command above, even to the very same value as it already is, the correct routine is executed and the 'lock' is released -- causing all programs depending on that to continue working at their normal processing load.

The discussion at lkml.org is also interesting to follow. You may want to start your ntpd the next day or so, just to be sure.



Hi! My name is Mattias Geniar. I'm a Support Manager at Nucleus Hosting in Belgium, a general web geek & public speaker. Currently working on DNS Spy & Oh Dear!. Follow me on Twitter as @mattiasgeniar.

Share this post

Did you like this post? Will you help me share it on social media? Thanks!

Comments

John Tuesday, July 10, 2012 at 22:42 - Reply

Thanks for your posts on the leap second issue. We run Hudson and JIRA (via Tomcat) and have recently been scratching our heads over their sustained drain on CPU. Resetting the date as you suggest fixed the problem.


Leave a Reply

Your email address will not be published. Required fields are marked *

Inbound links