GHOST: critical glibc update (CVE-2015-0235) in gethostbyname() calls

Mattias Geniar, Tuesday, January 27, 2015 - last modified: Thursday, January 29, 2015

A very serious security issue has been found and patched: CVE-2015-0235 nicknamed "Ghost".

The security bug

A heap-based buffer overflow was found in glibc's __nss_hostname_digits_dots() function, which is used by the gethostbyname() and gethostbyname2() glibc function calls. A remote attacker able to make an application call either of these functions could use this flaw to execute arbitrary code with the permissions of the user running the application.

This is major. The gethostbyname() calls can often be triggered remotely for applications that do any kind of DNS resolving within the code.

GHOST is a 'buffer overflow' bug affecting the gethostbyname() and gethostbyname2() function calls in the glibc library. This vulnerability allows a remote attacker to execute arbitrary code with the permissions of the user running the application.

The gethostbyname() function calls are used for DNS resolving, which is a very common event. To exploit this vulnerability, an attacker must trigger a buffer overflow by supplying an invalid hostname argument to an application that then calls gethostbyname().

Red Hat announcement of Ghost

The patch shows the updated files in the nss library (Name Service Switch). The bug was first disclosed on a French mailing list, but it may have been an accident --- the bug probably wasn't meant to be disclosed already, as no distro's had updated packages available.

This bug is present in all versions of Red Hat Enterprise Linux and variants (CentOS etc.) as well as debian systems.

Qualys, who discovered the bug during a code audit, wrote a mailing-list entry with more details, including a more in-depth analysis and exploit vectors.

Fixing CVE-2015-0235

Just like the recent OpenSSL heartbleed bug, this will be an annoying one to fix. The update is in the glibc package, but that's a set of libraries that are being used by a lot of running services. After the update, each of these services needs to be restarted ...

To find all the services that rely on the glibc libraries, run the following command. It will list all open files (lsof) and find the files that refer to the glibc libraries.

$ lsof | grep libc | awk '{print $1}' | sort | uniq

The updates are now available for RHEL 5, 6 and 7 as well as CentOS 5, 6 and 7 (all architectures).

Debian and Ubuntu have the updated packages available already so you can upgrade those.

Once the packages are updated and available for your distro, update your box.

For CentOS, Red Hat, Fedora, Scientific Linux, ...

$ yum clean all && yum update

For Debian, Ubuntu and derivatives:

$ apt-get clean && apt-get update && apt-get upgrade

Afterwards, restart every service you found with the lsof command above. It's probably easiest to just reboot your entire server, since pretty much everything depends on glibc ... If you can't reboot the entire system, at least restart all public-facing services like webservers, mailservers, etc.

Until the updates are available to all distributions, it's a waiting game. And until that time, every DNS name being resolved is a potential security threat ...

Possible attack vectors

The gethostbyname() call is probably among the most used ones on a server. That means any kind of DNS resolve can be used to trigger the CVE. The only catch is, you need to control whatever DNS is being resolved.

That could mean;

  • Mailservers using reverse DNS lookups on connecting IPs (DNS Blacklisting, SPF checks, ...)
  • Form submits that allow user content which results in a DNS lookup, think URLs, WordPress XML-RPC pingbacks, ...
  • MySQL servers doing authentication checks based on hostnames (in MySQL privileges)
  • SSH servers that perform DNS lookups for authentication allow/deny rules
  • ...

For a more in-depth look, including code examples, have a look at the Qualys mailing list entry which covers the situation more in-depth.

Any kind of DNS lookup can potentially trigger this. The only "positive" thing is that the exploit doesn't immediately escalate privileges, you're still the same user that ran the command. But there are ways of doing privilege escalation of course. And non-privileged users are still valuable assets for DDoS attacks, making server inventories, ...

Update: I've been thinking about ways to automate these patches using config management, and I'd love to hear thoughts!

If you're looking for scripts to test GHOST / CVE-2015-0235 vulnerability, check out this post.

Hi! My name is Mattias Geniar. 👋 I'm an independent software developer ⌨️ & Linux sysadmin 👨‍💻, a general web geek & public speaker. Currently working on DNS Spy & Oh Dear! Follow me on Twitter as @mattiasgeniar 🐦.

🔥 If you're stuck with a technical problem, I'm available for hire to help you fix it!

Share this post

Did you like this post? Help me share it on social media! Thanks. 🤗

Have feedback?

New comments have been disabled on this blog, existing comments will remain as-is. Want to give feedback? Is there a mistake in the post?

Send me a tweet on @mattiasgeniar!


Arthur de Jong Tuesday, January 27, 2015 at 18:18 -

NSS in Glibc refers to Name Service Switch.

On Debian systems there is a checkrestart command in the debian-goodies package which is very useful.

Denis Tuesday, January 27, 2015 at 18:36 -

Debian and Ubuntu have the updated packages available already so you can upgrade those.

I can’t find any updated Ubuntu packages. Can you post a link?

Denis Tuesday, January 27, 2015 at 18:45 -

Thanks, I found the Ubuntu ones in the meantime. They seem to be pushed as we speak:

    Nobody of Import Wednesday, January 28, 2015 at 01:54 -

    It should be noted that eglibc’s no longer maintained upstream. Ubuntu’s moved with the latest versions back to glibc. So there’s two upstreams there.

Florian Heigl Tuesday, January 27, 2015 at 18:50 -

Good time to turn off HostNameLookups in Apache and UseDNS / VerifyHostKeyDNS in SSH, at least for the moment.
How fancy this comes around when people start adopting DNSSEC + DANE.

thatjuan Wednesday, January 28, 2015 at 00:36 -

Thanks for posting this ;)

FYI, your bio says “Works at Nucleus” twice.. maybe a typo?

Thanks again!

Nobody of Import Wednesday, January 28, 2015 at 01:53 -

Rather than restarting services, wouldn’t this one be more of a smack-start the whole box story this time- be easier and less painful to schedule a small downtime window and do it. If you’ve got hardware that you’re worried about doing this to…you’ve got a ticking timebomb waiting to blow up in your face anyhow.

RMC Wednesday, January 28, 2015 at 03:09 -

I’m guessing you never managed servers in the thousands, right?

Wade Mealing Wednesday, January 28, 2015 at 03:38 -

The Red Hat versions have been released on the 27th.

Thomas Rumbaut Wednesday, January 28, 2015 at 11:12 -

Concerning automating the update process: we used the following one liner to check and restart all services that are using glibc:

# servicelist=""; for problemservice in `lsof 2> /dev/null | grep libc | awk '{print $1}' | sort | uniq`; do for service in `ls /etc/init.d/* | awk -F "/etc/init.d/" '{print $2}'`; do if [ "$problemservice" == "$service" ]; then if [ -n "`service $problemservice status | grep running`" ]; then servicelist+=" $problemservice"; else echo "$problemservice found but service is not running"; fi; fi; done; done; count=`tr -dc ' ' <<<"$servicelist" | wc -c`; servicelist=`echo $servicelist | xargs`; echo -n "$count services have to be restarted ($servicelist): continue (y/N)? "; read continue; if [ $continue == "y" ]; then for service in $servicelist; do /etc/init.d/$service restart; done; else echo "Leaving without restarting services"; fi
12 services have to be restarted (atd crond httpd mysqld named nginx php-fpm saslauthd snmpd sshd sw-engine xinetd): continue (y/N)? y
Stopping atd:                                              [  OK  ]
Starting atd:                                              [  OK  ]
Stopping crond:                                            [  OK  ]
Starting crond:                                            [  OK  ]
Stopping httpd:                                            [  OK  ]
Starting httpd:                                            [  OK  ]
Stopping mysqld:                                           [  OK  ]
Starting mysqld:                                           [  OK  ]
Stopping named: .                                          [  OK  ]
Starting named:                                            [  OK  ]
Starting nginx:                                            [  OK  ]
Stopping php-fpm:                                          [  OK  ]
Starting php-fpm:                                          [  OK  ]
Stopping saslauthd:                                        [  OK  ]
Starting saslauthd:                                        [  OK  ]
Stopping snmpd:                                            [  OK  ]
Starting snmpd:                                            [  OK  ]
Stopping sshd:                                             [  OK  ]
Starting sshd:                                             [  OK  ]
Stopping sw-engine-fpm:                                    [  OK  ]
Starting sw-engine-fpm:                                    [  OK  ]
Stopping xinetd:                                           [  OK  ]
Starting xinetd:                                           [  OK  ]

Matteo Wednesday, January 28, 2015 at 12:13 -

the command

lsof | grep libc | awk '{print $1}' | sort | uniq

contains a double UUOC (useless use of cat / fork)
a way better command is:

lsof | awk '/libc/{print $1}' | sort -u

Mattias Geniar Wednesday, January 28, 2015 at 15:47 -

For anyone that ‘s look for a one-liner to see if you’re vulnerable, I found the following script that works.

/usr/sbin/clockdiff `python -c "print '0' * $((0x10000-16*1-2*4-1-4))" `

If it throws a Segmentation Fault ($? == 139, exit code should be 139), then your system is vulnerable.

If you get a help-message about missing arguments, you’re safe and the buffer overflow failed.

Jesús Huerta Wednesday, January 28, 2015 at 22:25 -

A small FAQ in Spanish to help to detect and patch:

Patricia Wednesday, January 28, 2015 at 22:47 -

How do I get into the command line to run the test? I’m using an Apple Mac. We have websites on a dedicated CentOS server

Aron Roberts Thursday, January 29, 2015 at 00:41 -

This comment suggests that it might be necessary to run lsof as root (e.g. via sudo), in order to view a complete list of glibc-dependent processes:

(Reposting with – hopefully – corrected formatting …)

Paul Thursday, January 29, 2015 at 13:16 -

Tested my CentOS yesterday, and the upgrade can do the fix with no restart of OS or service needed as far as the ghost bug goes. However there will still be pending updates to install on reboot. So i say do the upgrade as it will remove the immediate threat and reboot the server at a regular risk time :-)

Inbound links