HTTPd: Cannot load mod_status.so into server: undefined symbol: ap_copy_scoreboard_worker

Red Hat Enterprise Linux has published a new advisory on July 23rd, RHSA-2014:0920-1 . This involves a security update for the Apache2 webserver for configurations that use mod_status in combination with ITK. CentOS then mirrored this update to their repositories as well.

However, if you execute the update, your previous configs may be broken due to the following error.

~# service httpd start
Starting httpd: httpd.itk: Syntax error on line 60 of /etc/httpd/conf/httpd.conf:
Cannot load /etc/httpd/modules/mod_status.so into server:
/etc/httpd/modules/mod_status.so: undefined symbol: ap_copy_scoreboard_worker

[FAILED]

The following bug reports have been reported upstream;

At the time of writing there was no known fix yet, so the two options below were the way out: downgrade the HTTPd version to the previous (unsafe) version, or disable the mod_status configurations in order to start your Apache server.

Update: the proper fix landed shortly after. The undefined symbol came from httpd-itk bundling its own httpd that was missing the new ap_copy_scoreboard_worker symbol that the patched stock httpd introduced. That ABI mismatch was fixed in httpd-itk-2.2.22-7.el6 (pushed to EPEL 6 in September 2014), so on a current system you just update httpd-itk to a -7.el6 or newer build instead of using either workaround below.

Solution #1: downgrade HTTPd / Apache to previous version#

First, remove the old versions. Note, this may remove dependencies, such as mod_fastcgi, webalizer, … whatever you may have. Keep that in mind.

~# yum remove httpd httpd-itk httpd-tools mod_ssl

Then, reinstall with the previous version number.

~# yum install mod_ssl-2.2.15-30.el6.centos.x86_64 httpd-tools-2.2.15-30.el6.centos.x86_64
 httpd-2.2.15-30.el6.centos.x86_64 httpd-itk-2.2.22-6.el6.x86_64

Obviously this is not ideal, but one can argue that a website is worth more online than offline. Keep monitoring the upstream updates for fixes.

Solution #2: remove the mod_status configurations from HTTPd / Apache#

Since it’s only the mod_status module that is having issues, you can disable that in your Apache configs.

First, comment out the line that loads the mod_status.so file.

$ grep 'mod_status.so' /etc/httpd/* -R
/etc/httpd/conf/httpd.conf:LoadModule status_module modules/mod_status.so

Second, comment out the lines that enable the ExtendedStatus and /server-status URLs.

$ grep -P '(ExtendedStatus|server-status)' /etc/httpd/* -R
/etc/httpd/conf/httpd.conf:ExtendedStatus       On
/etc/httpd/conf.d/monitor.conf:  <Location /server-status>
/etc/httpd/conf.d/monitor.conf:      SetHandler server-status
/etc/httpd/conf.d/monitor.conf:      ...
/etc/httpd/conf.d/monitor.conf:  </Location>

In the example above, that’s the “ExtendedStatus On” config and the “SetHandler server-status” part. Comment them both out and restart your Apache.