For three days in January, the Government thought it may have been the target of a malicious cyber attack, Marc Daalder reports
When a wide range of government websites went down for 76 minutes on a Friday in mid-January, officials grew concerned it might have been the result of a massive hack.
For more than an hour on January 15, about 60 percent of web addresses with a .govt.nz, .mil.nz, .parliament.nz or .health.nz domain name were affected by an outage of the Government's Domain Name Service (DNS). Some sites were unavailable for the duration of the outage while others may have gone down partway through. Emails may also "have been delayed during all or part of the outage," a spokesperson from the Department of Internal Affairs told Newsroom.
Among the websites which may have been affected were 79 private health organisations.
The following Monday, officials determined that the outage was the result of a confluence of unfortunate events: A cyber-security test run by the Public Service Commission at the same time as business-as-usual maintenance and a new upgrade to a firewall meant to deter cyber attacks spiralled out of control, taking the entire DNS system down.
A "malicious attack was initially suspected", according to a briefing to Digital Economy and Communications Minister David Clark, released under the Official Information Act. The Internal Affairs spokesperson was more circumspect, saying only that "during the outage and subsequent investigation, a malicious attack was one of a number of possible root causes considered".
The outage and potential attack weren't publicly notified in any way.
"In this case the outage was advised to government agencies in order that they advise their staff and customers as required in case of inconvenience," the spokesperson said.
"There is no requirement to publicly notify this type of technical issue, and indeed until a cause is identified it can be prudent to limit detail in case of malicious intent."
The DNS system is spread across five servers - two in Wellington, two in Auckland and one in Sydney – to protect against cyber attacks. All five servers were "overwhelmed" and taken offline by the outage. The Wellington and Sydney data centres restarted at 2.15pm on the same day, while the Auckland servers needed to be manually reset the next day.
The briefing to Clark said officials "believe" the same test wouldn't have the same effect again. However, they were looking into ways to communicate with other agencies and the public in the event of a future widespread email and web outage.
Officials also implied that the event was well out of the norm. Hour-long outages for individual government websites were "rare but should be expected". As a whole, the DNS system is expected to be up for 99.999 percent of the time - allowing about five minutes of unavailability each year. There were no outages in the 11 months prior to January.
The January 15 event, therefore, saw an outage 15 times longer than the expected annual availability.
