System Outage
Incident Report for

Public root cause analysis of domain failure on 12.04.17

On the morning of December 4, 2017 our site and email processing systems went down. After some investigation, we realized that another party had taken our domain, though our account was up to date, and we had initiated no changes. After contacting our DNS registrar, we discovered that the .ai NIC (which is based in Anguilla) had mistakenly assigned our domain to a Chinese nameserver. We also learned that Anguilla updates the top level registry manually and that recent hurricanes have severely damaged their infrastructure.

Fortunately, after contacting our registrar and the errant owner of our domain, we recovered it and put our systems back online. Once we were able to get the domain owner to reverse the error, we experienced residual delays as we waited for our DNS change to propagate through the internet (a process that can take up to 24 hours).

Posted over 1 year ago. Dec 12, 2017 - 13:51 EST

System outage has been resolved and is slowly coming back to full functionality.
Posted over 1 year ago. Dec 04, 2017 - 17:51 EST
The root cause of the outage (which was external to has been identified and fixed. We are now waiting for all systems to become fully operational.
Posted over 1 year ago. Dec 04, 2017 - 15:46 EST
We're experiencing a system outage that’s affecting emails to and from Amy & Andrew as well as We’re on it and working to fix ASAP.
Posted over 1 year ago. Dec 04, 2017 - 13:34 EST