A minor update to the Apache httpd project on CentOS 6 had an unexpected consequence. The update from 2.2.15-59 to 2.2.15-60, as advised because of a small security issue, started respecting RFC 1123 and as a result, stops allowing underscores in hostnames.
I spent a while debugging a similar problem with IE dropping cookies on hostnames with underscores, because it turns out that's not a valid "hostname" as per the definition in RFC 1123.
Long story short, the minor update to Apache broke these kind of URLs;
These worked fine before, but now started throwing these errors;
Bad Request Your browser sent a request that this server could not understand. Additionally, a 400 Bad Request error was encountered while trying to use an ErrorDocument to handle the request.
The error logs showed the message as such;
[error] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23)
The short-term workaround was to downgrade Apache again.
$ yum downgrade httpd-tools httpd mod_ssl
And that allowed the underscores again, long-term plan is to migrate those (sub)domains to versions without underscores.