Comcast Releases Detailed Analysis of NASA.gov DNSSEC Validation Failure

Comcast dnssecDid you see all the tweets and mentions on the web last week about Comcast apparently blocking NASA’s website on their network? While it made for great headlines, the truth was that there was an error with the DNSSEC signing of the NASA.gov domain and now that Comcast has deployed DNSSEC-validating DNS servers those DNS servers were correctly responding with a failure and blocking access to the site.

Comcast alerted the administrators of the NASA.gov site and worked with them to resolve the issue. Comcast’s DNS Engineering team has now issued a very helpful analysis of the NASA.gov DNSSEC validation failure. As they outline in part of the executive summary:

On January 18, 2012, the NASA.GOV domain had a DNS Security Extensions (DNSSEC) signing error that blocked access to all NASA.GOV sites when using DNS recursive resolvers performing DNSSEC validation. As one of the largest ISPs in the world utilizing DNSSEC validation, users of Comcast noticed a problem when attempting to connect to the website. This caused some people to incorrectly interpret this as Comcast purposely blocking access to NASA.GOV and recommending users switch from Comcast security-aware DNS resolvers to resolvers not performing DNSSEC validation … Instead, the administrators of the NASA.GOV domain had enabled DNSSEC signing for their domain, and the security signatures in their domain were no longer valid. The Comcast DNS resolvers correctly identified the DNSSEC signature errors and responded with a failure to Comcast customers. This is the expected result when a domain can no longer be validated, and this protects users from a potential security threat.

The document then goes on at some length to explain how Comcast monitors for DNSSEC validation failures, how the NASA.gov domain failed, how the issue was resolved and how users responded to the issue. The document includes a number of charts from DNSViz showing the precise error in the DNSSEC chain-of-trust and then interestingly includes examples of the web and Twitter reaction to the problem.

In a message to the dnssec-deployment mailing list announcing the release of this document, Comcast’s Jason Livingood stated:

Since we feel that the entire Internet community has room for improvement on signing processes (not to single out NASA), we decided to start doing failure analyses here and there – and share them with the community in the hope that it will help bring greater operational scrutiny and maturity to DNSSEC signing processes.

Kudos to the Comcast DNS Engineering team for releasing this analysis as it is indeed quite helpful to understand how DNSSEC validation failed, what the user experience was and how the issue was resolved. We look forward to seeing more of these types of analysis documents as the global DNSSEC rollout continues. Sharing this level of detail will definitely help others who encounter similar situations and will only make the DNSSEC system that much stronger.

Want to learn more about DNSSEC? View our DNSSEC-related resources, including tutorials, videos and more…

UPDATE: Dark Reading also put up a good post today on this topic: DNSSEC Error Caused NASA Website To Be Blocked

January 25th, 2012 by | Posted in DNSSEC | Tags: , | 4 Comments

4 Responses to Comcast Releases Detailed Analysis of NASA.gov DNSSEC Validation Failure

  1. Mark Fritz says:

    So this is now April 20, 2012 and Comcast is still blocking NASA.
    http://wwt.nasa.gov/wwt/p/mars_base_map/2/0/0.png

    Should show a nice png of Mars, but It’s not to be found if you’re a Comcast customer. When are they going to get around to fixing the mistake?

    • Dan York says:

      Mark,

      The issue with NASA.gov having incorrect DNSSEC keys was fixed way back in January. The report from Comcast mentioned in this blog post was the report from after the incident relaying what happened and how it was fixed. There have been no reports of issues since that time that I’ve seen.

      If you are unable to get to an image on NASA’s site, I would contact Comcast technical support or send them a Twitter message to @ComcastCares. I don’t see any messages on their DNSSEC info center indicating there are any new problems: http://www.dnssec.comcast.net/

      Hopefully if there is an issue it will be quickly corrected.

      Dan

  2. […] was the issue with NASA.GOV, which turned out to be a problem with the changing of DNSSEC keys.  Comcast and NASA provided a detailed explanation of what happened then.  (And in another case of spectacularly bad timing, the outage occurred on the day of the […]

  3. […] quality to the level of conversation. For instance, much of the conversation lately has been around the Comcast / NASA DNSSEC validation error and the great thing is that discussion includes people from both NASA and […]

Leave a Reply

Your email address will not be published. Required fields are marked *