A fix for the Java Leap Second bug

Want to help support this blog? Try out Oh Dear, the best all-in-one monitoring tool for your entire website, co-founded by me (the guy that wrote this blogpost). Start with a 10-day trial, no strings attached.

We offer uptime monitoring, SSL checks, broken links checking, performance & cronjob monitoring, branded status pages & so much more. Try us out today!

Profile image of Mattias Geniar

Mattias Geniar, July 02, 2012

Follow me on Twitter as @mattiasgeniar

Java + the leap second that occurs every 18 months = pain. Not that Debian Squeeze is that much better, though. The patch was provided a few months in advance, but most vendors package a custom java version that doesn’t include that fix.

It starts with entries in your logs that are similar to these.

Clock: inserting leap second 23:59:60 UTC

And after that, your Java processes eat up more and more CPU causing a continued rise of system load, usually until things just come to a halt.

The fix is something like this.

$ /etc/init.d/ntpd stop
$ date -s "`date`"

Why does this work, you may wonder?

It seems that the bug in the leap second code caused a glitch in how the time is to be changed on the server. It’s missing a vital function call that ‘unblocks’ the time.

By setting the date again with the command above, even to the very same value as it already is, the correct routine is executed and the ‘lock’ is released – causing all programs depending on that to continue working at their normal processing load.

The discussion at lkml.org is also interesting to follow. You may want to start your ntpd the next day or so, just to be sure.



Want to subscribe to the cron.weekly newsletter?

I write a weekly-ish newsletter on Linux, open source & webdevelopment called cron.weekly.

It features the latest news, guides & tutorials and new open source projects. You can sign up via email below.

No spam. Just some good, practical Linux & open source content.