A recurring trope in the Road Runner cartoons is Wile E Coyote chasing the Road Runner only to run off a cliff and taking a few protracted moments to realise before succumbing to the effects of gravity. In a way, the entire census farce has been much like our beloved Acme Coyote overestimating its competence and then finding itself at odds with forces far more powerful than itself. Unlike Wile E, gravity took months to catch up with the Australian Bureau of Statistics, but unlike Wile E, this isn’t a cartoon and coming crashing down to earth will have serious consequences.
This is a standard technology stuff-up story: the brash claims before of “extreme cost efficiencies and savings”; the herculean boasts about system performance and it being “impossible” for the system to crash; how the system is the eTitanic and will stand up to incredible loads; it’s a fantastic system, you’re going to love it, it’s going to be online and we’re going to collect all your data and get the taxpayers to pay for it. It’s wonderful. It’s incredible.
Anyone who works in technology knows you don’t spite the Production Gods and spite the Production Gods the ABS did, from statements made beforehand to apparently a substantive lack of testing. The ABS’s public statements indicate they expected about 1,000,000 or so form submissions an hour. That’s key, that doesn’t mean they had the capacity for 1,000,000 concurrent users, just that people would submit 1,000,000 censuses, distributed over that hour, so about 17,000 a minute. If we estimate that 12 or so million households would fill it out online (this is a generous estimation) that’s 12 hours to get the whole kit and caboodle in without a single issue or error. Technology simply doesn’t work that way, there are always weird ghosts in the machine and a 100% perfect strike rate is unheard of. Something can always go wrong.
Heck, even a bit of mild spelunking into the system shows they most likely didn’t build a system with servers all over the country. A system with servers across the country is pretty trivial to set up these days: you’d have a data centre in Sydney, one in Perth and maybe one in Adelaide. If the internet to the Sydney data centre was cut you’d be able to fail people over to one of the other data centres, rather than having the whole census taken offline. Sure, there are security and other reasons to initially beware about multiple data centres but this was a seriously long product development cycle – they could have proposed and audited multiple data centres as part of that system. It appears they did not.
This isn’t even taking into consideration the myriad privacy issues brought up with this census that were flatly hand-waved away by the ABS. If they’d done a proper privacy impact assessment with external stakeholders, members of the public and NGOs, rather than doing an in-house assessment dropped just before Christmas, the response would have been drastically different. In no uncertain terms, the comms management from initial handling of public concerns (such as threatening to fine Senators, a very smart move in the long run) to handling the aftermath of the system’s crash, have been a fantastic example in what not to do.
Not to mention the “we was hacked” claims which came out in response to the system falling over. No existing network traffic maps show large amounts of DDoS traffic into the census network, but even if there was and through some mechanism it wasn’t recorded, DDoS mitigation should absolutely have been part of the initial requirements scoping. The census was having a big technology launch on a big day and it would obviously be an attractive target for those wanting to put egg on the government’s face.
All in all this looks like serious structural organisational failure. The ABS, a government body that by all accounts had substantial respect and esteem in the community, hand-waved away concerns before, during and after the eCensus debacle. They seemed wholly unconcerned that Australian citizens were rightly worried about what their data was being collected for and why the scope of data had changed.
In the post-Snowden world, citizens have a right to know how their data will be collected and how it will be stored to protect it from both state and non-state actors. This is a serious honeypot of valuable data. Malcolm Turnbull’s proposal that the audit of the aftermath be performed by the ABS – the same organisation that has failed time and time again – is on the face of it, laughable.
The ABS has absolutely proven beyond any reasonable doubt that we cannot just “take their word for it”. The ABS is too important and vital an institution to let a few individuals tank it to save face. The accountability process must be open so these mistakes do not happen again. The census is too important and public trust in public institutions shouldn’t be so easily frittered away without consequences.
Time will tell whether the census will even be able to recover from the series of systemic failures that led to Tuesday night’s failure. Needless to say, the next Senate Estimates will be extraordinarily interesting.
Keep in mind this is merely an eCensus, running a system that has less throughput and uptime than Facebook handles in an hour from Australia. If the government can’t build and deliver secure and reliable IT systems then we should resist letting it fail in this field over and over again. If we can’t take the census online securely then for goodness sake we should definitely not consider taking voting online, unless we want Sun Yang to win every single seat.
