Why Internet Explorer Won’t Allow Cookies On (sub)domains With Underscores

Mattias Geniar, Wednesday, July 8, 2015 - last modified: Thursday, April 6, 2017

Debugging this issue cost me some time today. Enough that I'll never forget how IE handles cookies on (sub)domains that contain underscores.

In hindsight, it seems obvious. In fact, there's even an Internet Explorer FAQ that describes how IE should react when it's presented with a domain or subdomain that contains underscores. Except at the time, I had no idea this was even related.

My Problem

My problem was that session cookies in PHP would work in Chrome and Firefox, but just refuse to work in Internet Explorer. Even the very latest version of IE, Internet Explorer 11. It's the kind of bug that appears to be by design and will stick around until the end of IE times.

Maybe Project Spartan aka Microsoft Edge will change this ancient behaviour?

What Is This Bug Of Which You Speak?

If the domain or subdomain your web application is running on contains an underscore, Internet Explorer will refuse to store cookies. Any kind of cookie. From session cookies to persistent cookies. Your webserver will reply with a Set-Cookie header and the client will happily ignore it.

This kind of domain name works: something.domain.tld
This kind of domain does not: some_thing.domain.tld

If you're working with sessions and session cookies, that's a problem. Every page refresh, the client responds with an empty Cookie: header so the server generates a new Set-Cookie header on every request.

Cookies just don't work in IE if your (sub)domain contains an underscore.

What's The Cause?

According to Microsoft, this behaviour was introduced with kb316112. It's a Windows patch designed to resolve CVE MS01-055, which dates back to 2001.

It's a cookie vulnerability. From 2001. For which we still experience the consequences.

The original CVE fixed this particular problem:

This patch eliminates three vulnerabilities affecting Internet Explorer. The first involves how IE handles URLs that include dotless IP addresses.

If a web site were specified using a dotless IP format (e.g., http://031713501415 rather than http://207.46.131.13), and the request were malformed in a particular way, IE would not recognize that the site was an Internet site. Instead, it would treat the site as an intranet site, and open pages on the site in the Intranet Zone rather than the correct zone.

This would allow the site to run with fewer security restrictions than appropriate.
MS01-051

So why does the fix for that CVE still affect us today?

As part of the fix in kb316112, Microsoft introduced stricter validation for domain names in DNS. That essentially means all domain names must follow the DNS RFC. Its origin dates back to RFC606 (1973) and RFC608 (1974).

Guess what the original DNS syntax does not contain? That's right: underscores.

So Microsoft started preventing cookies on anything that contains invalid DNS characters.

Security patch MS01-055 prevents servers with improper name syntax from setting cookies names. Domains that use cookies must use only alphanumeric characters ("-" or ".") in the domain name and the server name.

Internet Explorer blocks cookies from a server if the server name contains other characters, such as an underscore character ("_").
Security Patch MS01-055

Here's where I think they went wrong, though.

Underscores are indeed not allowed in host names, but they are allowed in domain names. The difference is the interpretation of a "host name" vs. a "domain name".

RFC2181, published in 1997, clearly states this.

The DNS itself places only one restriction on the particular labels that can be used to identify resource records. That one restriction relates to the length of the label and the full name. [...]

Implementations of the DNS protocols must not place any restrictions on the labels that can be used. In particular, DNS servers must not refuse to serve a zone because it contains labels that might not be acceptable to some DNS client programs.

RFC2181

To me, it seems like Microsoft introduced a wrong kind of validation and mixed host names with domain names.

So How Do I Fix It?

Just send a Pull Request to the IE11 codebase with the fix!

Well, since that's obviously not an option, there really is no alternative but to avoid underscores in your (sub)domains on the internet. This can be especially annoying for auto-generated subdomains (where I experienced it), where underscores could accidentally be introduced and break things in unexpected ways, for IE users.

For the future, I hope Microsoft reviews this policy of setting cookies on domains that include an underscore. Maybe they're correct in following the RFC & standards (although this is debatable), but they're the only browser that appears to be doing so.

At what point should you abandon principle and instead follow the masses in adopting non-standard practices?



Hi! My name is Mattias Geniar. I'm a Support Manager at Nucleus Hosting in Belgium, a general web geek & public speaker. Currently working on DNS Spy & Oh Dear!. Follow me on Twitter as @mattiasgeniar.

Share this post

Did you like this post? Will you help me share it on social media? Thanks!

Comments

Trouble Thursday, July 9, 2015 at 02:13 - Reply

I am blissfully ignorant about webby things and cookies, but I do know a little bit about DNS. As far as I can tell, Microsoft’s interpretation looks correct.

Unless you’re using a very strange web client, you’re resolving the names of your webservers to AAAA (and legacy A) records. In other words: to “hostnames”.

In the context of DNS, the names of address records (AAAA and legacy A), the names of nameservers (NS records and the SOA mname field), and the names of mailservers (MX records) are “hostnames”. For historical reasons “hostnames” cannot contain underscores. (See RFC 1035, 2.3.1).

I agree that this historical limitation is awkward but … it’s DNS. Live with it.


    Trouble Thursday, July 9, 2015 at 02:18 - Reply

    For completeness: if you were resolving to a SRV record, you would not be resolving a “hostname” and its name could be anything you wanted it to be.


    Mattias Geniar Thursday, July 9, 2015 at 08:40 - Reply

    I agree that this historical limitation is awkward but … it’s DNS. Live with it.

    That may indeed be the RFC, but reality is: underscores work perfectly in DNS. A wildcard *.something.tld will happily match client_a.something.tld.

    I agree that IE is following the standards to the letter, which could be argued as a good thing or a bad. But my final thought still remains though: if everyone else is allowing it, and the internet has pretty much accepted it “as is”, should IE just allow underscores as well then?


      Trouble Thursday, July 9, 2015 at 10:23 - Reply

      A wildcard will resolve anything. But a wildcard still is not a valid “hostname”. Try pointing an NS record at a wildcard, see what happens. ;) Or try telling your laptop that its hostname is “*” (or _!).

      I’m not sure which internet you think has accepted underscores in hostnames. As far as I can tell, the RFC 1035 limitation that hostnames must follow ARPANET rules is still valid. I seem to remember this came up a couple of years ago in discussions on punycode for IDN. Punycode uses hyphens.

      Underscores do “work perfectly” in DNS. The DNS does not care about the names of things (though I’m sure more than a few implementations will break on underscores in hostnames, among other things). But the rules are that underscores are not allowed in hostnames. If you want to use underscores, you’ll need to convince your application to use SRV records, or come up with your own RR type (good luck with that … see what happened to SPF).

      The bottom line is that it’s best to be conservative with DNS. A surprising number of things rely on behaviour specified decades ago. While the reasons for that behaviour may be lost in the mists of time, gratuitously breaking things by expecting that “everyone” has forgotten is not how standards work.


        Mattias Geniar Thursday, July 9, 2015 at 13:09 - Reply

        While the reasons for that behaviour may be lost in the mists of time, gratuitously breaking things by expecting that “everyone” has forgotten is not how standards work.

        You make a very valid point, indeed.


Will Thursday, July 9, 2015 at 15:17 - Reply

Trouble is right; Microsoft’s interpretation is correct. The [host] portion of an HTTP (or HTTPS) URL will always contain a hostname and thus be subject to the character set restrictions placed on hostnames. The DNS is a red herring here: it is only one method among many for resolving hostnames to IP addresses (hence the existence of nsswitch.conf), and hostnames are only one sort of data that can be represented in the DNS.


    Will Thursday, July 9, 2015 at 15:28 - Reply

    That was meant to read “the [host] portion of” a URL, but the HTML parser ate my angle brackets.


    Will Thursday, July 9, 2015 at 15:34 - Reply

    …and section 3.2.2 of RFC 3986 is authority for my assertion about the [host] portion:

    Such a name consists of a sequence of domain labels separated by “.”,
    each domain label starting and ending with an alphanumeric character
    and possibly also containing “-” characters.


Bogdan Sunday, October 11, 2015 at 21:32 - Reply

“Maybe Project Spartan aka Microsoft Edge will change this ancient behaviour?”

Well, I can confirm that this isn’t the case :). I’ve wasted about an hour today until I landed on your page due to this behaviour.

Thanks for writing this!


Brad Friday, June 17, 2016 at 07:47 - Reply

Thanks soooo much for the post … This explains why Firefox and Chrome had no issue and IE did. The problem manifested itself in IE with the message of “Please enable cookies in your browser before using this version of Citrix Receiver” …. argh … who would have guessed that it was an underscore in the host name of the url that was the cause of this message? Thanks again.


Shishir raven Saturday, January 7, 2017 at 10:49 - Reply

So, So happy that you wrote this article.
Really. Really happy.

I was able to find out the issue and fix it.


Stef Wednesday, February 22, 2017 at 15:29 - Reply

Same for me!
Your article is very useful and also very interesting!!
Thanks to it, I was also able to solve my inexplicable problem that i had in IE but not in Firefox nor in Chrome!

So a lot of thanks!


Chris Lienert Thursday, April 6, 2017 at 10:26 - Reply

Thanks for posting this having run into the same problem just now.

Irrespective of whether the behaviour is correct or not, the killer is the lack of error thrown. The utter silence turns an oddity into a nightmare.


André Morales Monday, August 28, 2017 at 22:33 - Reply

Thanks for posting this. Helped immensely.

Microsoft like as always making our lives as developers more difficult.


Adam S Thursday, March 28, 2019 at 16:56 - Reply

Someone on my team was able to solve an IE cookie issue we were having due to this post. So, thank you! :)


Leave a Reply

Your email address will not be published. Required fields are marked *

Inbound links