I use Netlify’s generous free tier to host this blog. When I set it up, it gave 2 options: either Netlify could be the DNS or to CNAME to the Netlify subdomain from my existing DNS. At that time, since I wasn’t using my domain for anything else I let Netlify be the DNS.

Recently I made a decision to use my own domain for emails too as it would be easy to jump services without losing much. As part of this, I also started using Cloudflare as the DNS for my domain & setup CNAME records to point my blog subdomain to the netlify subdomain.

CNAME setup

While doing this, I wanted to check if it would be possible to compartmentalize & keep Netlify in the dark about my custom domain. This didn’t work out as expected as you can see from the following screenshot. Note that we are setting the Host header in this request, the reason for which will become apparent in the next paragraph:

Blog not found
404s (not found) without Netlify also knowing about the custom domain

Wondering if it could be for security purposes to prevent one from passing off someone else’s content as one’s own, I read more about how CNAME records work under the hood. I couldn’t find anything supporting this theory but came across a few posts (1, 2) that drove home the importance of the Host header. It is possible for a single server at an IP address to serve multiple sites. The server ends up using the Host header to figure out what the user entered in the address bar regardless of any CNAME based redirection 1, 2.

If the incoming Host value is not one of the recognized values, servers handle this in a few different ways. A few examples:

DuckDuckGo redirects
DuckDuckGo 301s (permanent redirection) to https://duckduckgo.com/
Netflix 404s
Netlix 404s (not found) which is the same as how Netlify handles it

All I had to do to make my blog resolve corretly was to inform Netlify about the custom domain. Netlify’s docs already talks about this but I wanted to check if it was an absolute requirement & the answer seems to be a resounding “yes”. Once I did that, things started working as I expected:

Blog resolves
Netlify successfully serves my blog on informing about the custom domain

It would be great if Netlify recognizes this case & does some of the following:

  • note in docs about how not informing Netlify about the custom domain would lead to a 404
  • note the importance of the Host header (although this could become a little too technical)
  • fail with a more helpful error message like Are you trying to access your blog through a custom domain? If so, setup CNAME & let Netlify also know about this custom domain

Notes

1. Host header is not the only way, SNI can also be used 2. In case of redirection because of 301 or 302, the Host header will be rewritten. This can be tested by visiting duck.com which redirects to duckduckgo.com