More than 200 of Optus’s customers were unable to call triple zero during the 14-hour outage, the company’s chief executive, Kelly Bayer Rosmarin, revealed to a Senate inquiry on Friday.
Bayer Rosmarin and the company’s head of networks, Lambo Kanagaratnam, faced a two-hour grilling before a Senate committee on Friday, chaired by the Greens communications spokesperson, Sarah Hanson-Young. Questions explored the outage itself as well as the company’s communication to customers and what will be done to prevent it happening again.
Decades-long rules mean mobile phones should be able to dial triple zero even when their mobile provider’s network is down by connecting via another network.
But Bayer Rosmarin said on Friday that 228 of the Optus customers who attempted to call triple zero during the outage on Wednesday last week were unable to connect their call.
The company had since conducted a welfare check on those customers and “thankfully everybody is OK”, she said.
The issue sparked an Australian Communications and Media Authority (Acma) investigation which Bayer Rosmarin should happen expeditiously.
“We absolutely believe that the triple zero system should have worked and it’s critical for all Australians that that system can be relied upon,” she said. “We don’t manage the triple zero system. It’s a very complex system that involves all the carriers.”
Bayer Rosmarin told the hearing she now carries both a Telstra and a Vodafone sim card with her in case of another outage like the one that took down Optus last week.
Bayer Rosmarin said she previously carried a spare Vodafone sim in case of an outage, but now also carries a spare Telstra sim.
“I now have all three. I used to have Vodafone and I have Telstra now too,” she said.
Bayer Rosmarin also said she did not speak to the communications minister, Michelle Rowland, until four hours into the outage last week.
“When I got up in the morning, I could see that the phone wasn’t working. And so I immediately decided to head into the office.”
She said she arrived at the office at 7.35am and had a crisis meeting from 7.45am until 8.30am. She said she spoke to the minister after that.
Bayer Rosmarin said her team had been in contact with the minister’s office before then, and that she assured Rowland Optus was working to restore the network as quickly as possible.
Optus initially claimed on Monday the outage was caused by changes to routing information supplied from an international peering network after a routine software upgrade. The company did not reveal until late on Wednesday that the peering network in question was Optus’s parent company’s Singtel Internet Exchange (STiX).
However, Singtel moved to distance itself from responsibility, saying Optus had been informed in advance about the upgrade and that it had not caused the outage. Kanagaratnam said on Friday it was ultimately an issue with Optus’s Cisco routers not being configured correctly.
“The outage was as a result of our defences for the change in our routing information not working as it should have been and that’s something obviously we’ve addressed,” he told the Senate committee. “And we’ve worked around the clock to ensure that we address all the issues that we found.”
During the outage about 90 of the company’s routers went offline due to a failsafe mechanism after receiving an overload in routing information, triggering the network shutdown. About 150 engineers were required to visit 14 sites across the country to reactivate them.
Bayer Rosmarin said the company had to perform a “hard reboot of the network” at about 10.30am on the Wednesday – a “brute force resuscitation of the network”. While the crisis was over for customers at 4pm, it took the team several days to determine what caused it.
The company had initially wondered whether it was a result of another cyber-attack, with Bayer Rosmarin noting there were “strange coincidences” between last week’s incident and last year’s cyber-attack, most notably that the Singtel board was in Australia for both. That idea was quickly ruled out, she said.
The company had conducted an outage exercise in October but did not account for the entire network going down. Kanagaratnam said Optus had now put in place protections so a similar event could not occur in future.
So far, 8,500 customers and small businesses have contacted Optus about compensation for losses amounting to $430,000 in compensation. The company had already applied $36,000 in compensation.
Bayer Rosmarin said Optus was keen to have a “broad” discussion about consequential losses as a result of the outage, noting that it would have wider implications for the NBN and other sectors going forward.
“We felt that this was an issue that’s much more broad and that should government choose to look into this we’d love to be part of that conversation. But there is no precedent for telcos or other essential providers covering consequential loss,” she said.
“We are very conscious that this would have far-reaching implications, not just for Optus, not just for all telcos, including the NBN, also for other essential services, utilities, government services, and that this needs to be a much broader conversation.”
She also said that measures allowing Optus customers to roam on to rival networks were not in place, and would take some discussion with other industry players and investment in capacity before it could happen in future.
The CEO also faced criticism from senators over the company’s communications with customers during the outage. She acknowledged that the company had only provided select media interviews, and said she would consider holding press conferences in the future, but it was “unusual for a CEO” to front the public during an outage.
Bayer Rosmarin did not directly respond to reports suggesting she could resign as early as next week, instead saying her focus had been on responding to the outage.
