Four years after a disastrous census, internal reports raise major concerns about next year’s population snapshot.
A $33 million contingency for next year’s census is expected to be exhausted, leaving nothing in reserve for Covid disruptions.
According to internal Stats NZ documents, released under the Official Information Act, the overall cost is now expected to rise to $250 million – almost double the $126m cost of the bungled 2018 census, and $40m more than was originally allocated by the Government.
Last year, Ministers agreed to release $15.4m in contingency money. “It is anticipated the remaining contingency ... will be spent,” the documents state.
A budget note from a census status report in October states: “The programme does not hold any contingency for high-impact, low-risk events, for example: natural disasters and Covid-19 resurgences.”
Concerns are also being raised within Stats NZ about the programme’s preparations, including staff recruitment issues, which add pressure to existing staff, and delays in creating back-end software systems. There’s “high” concern the expected response rate for Māori won’t be achieved, although management controls are said to be “effective”.
Worryingly, some initiatives to cut spending, to offset rising costs, seem aimed at on-the-ground operations.
For a census that is about re-building trust, could this be an early sign another mess is looming?
Newsroom asked Statistics Minister David Clark if he’s comfortable with preparations, and confident the census will achieve a higher response rate for Māori.
“I’m advised that the census programme is tracking according to plan, and that Stats NZ has appropriate plans to deal with the impact of Covid-19,” he said via email on Wednesday morning. “Stats NZ has thorough contingency planning in place and has been working to deliver a census within a Covid-19 environment for two years now. It has adapted many processes to work in this new environment.”
Clark parrots a line sent by Stats NZ the previous day: “A draw down of contingency funding was anticipated and is standard practice for programmes of this size.”
Assurance processes are in place, the Minister assures – including regular independent quality assurance assessments, quarterly monitoring of progress against the 2018 External Data Quality Panel recommendations, and programme gateway reviews with Treasury.
As for the likelihood of escalating costs affecting on-the-ground field operations next year, Clark says: “This is an operational matter and is best answered by Stats NZ.”
He doesn’t say if he’s confident the census will meet its response rate expectations for Māori. He simply lists the work Stats NZ is doing to “deliver for Māori”, including the development of a dedicated iwi-led data collection and analytics pilot.
“I understand the external Taumata reference group meets regularly to inform the programme’s response to the needs of Māori. Community engagement teams are already in place and engaging with iwi and Māori on the 2023 Census.”
Stats NZ’s annual reports show salaries and wages have increased 26 percent in two years, to $109m, for 1410 staff. Permanent staff paid more than $100,000 rose from 154 in 2015/16 to 263 four years later. (You can add another 23 fixed term staff to that latter figure.)
Michael Woodhouse, National’s statistics spokesman, says considering Stats NZ is employing more people on higher salaries, it’s extraordinary it is failing to meet census programme milestones, and can’t stay within the original budget.
“That needs to be explained,” he says. “David Clark has to be completely all over this, otherwise the 2023 census will go the way of the last one, and we can't afford that.”
Newsroom asked Stats NZ to list the biggest risks to the census right now, its largest areas of cost over-runs, and the biggest cuts to the programme to reduce costs. It was also asked if the public should be concerned the $33m contingency was expected to be used, with nothing left in case of Covid disruptions.
It refused to provide comment from a named spokesperson.
Senior communications adviser Stephanie Robertson said via email: “When attributing comments to individual employees of Stats NZ, our preference is to have some knowledge and context of the article to ensure their comments are appropriate to that context.”
The anonymised statement, which says the census programme is on track, can be found at the bottom of this story.
Why is the census important?
The census is a five-yearly snapshot of the country which governments use to plan – such as deciding the location of new schools, and the size of hospitals. (The first census was held in 1851. The Christchurch quakes delayed the 2011 edition until 2013.)
Data is publicly available, and used by a range of bodies from local government to private companies and academic researchers.
The census was meant to be modernised in 2018 – a shift to responses being done primarily online. More use would be made of administrative data – data collected for other purposes, such as income records from IRD, Ministry of Education data on school attendance, and birth registrations with Internal Affairs. The field collection workforce was smaller.
That would lead to better data and lower costs. At least, that’s how the theory went.
What actually happened was what Stats NZ characterises a “major failure”. It cost chief statistician, Liz MacPherson, her job.
There weren’t enough field staff and people couldn’t easily get their hands on paper forms. Public frustration led to the uncomfortably rare situation of the census making national news for the wrong reasons.
Lower-than-expected response rates, especially by Māori, prompted an independent review – which published its report before the first census data was released. (Māori response rates were 68.2 percent, down from 88.5 percent in 2013.)
The programme’s leadership “lacked strategic direction and effective programme management”, reviewers Murray Jack and Connie Graziadei said. Governance was “overly complex and ultimately ineffective”. The tone of reporting “had an optimism bias at times”.
Recommendations included establishing a single census programme board, with external advisors, and strengthening the programme’s leadership.
“Given the importance of the census and the role it plays in informing decisions that affect the lives of New Zealanders, it is critical that trust and confidence in future censuses are maintained and enhanced.”
Problems with the 2023 census would likely throw doubt on Stats NZ’s ability to rebuild public trust.
Contingency to be swallowed
Stats NZ provided 11 months of PowerPoint presentations from census programme board meetings, after an Official Information Act request by Newsroom.
A year ago, the programme was rated “on track”. However, its five-year overspend was forecast to be $5.1m – for a total of $223m. (It could be argued the overspend was higher, as it included a proposed $8.3m swap from capital expenditure to operational.) “The forecast overspend is within the $33 million contingency held by Treasury,” the January status report said.
Yet, a few pages on, there was warning of added potential costs: for additional bilingual forms, personal protective equipment for field staff, more staff needed to assist households to fill out forms, procurement costs, and postal overspend. The total was an eye-watering $18.4m, pushing the total dangerously close to the upper bound of $251m – comprising the budget forecast, including the capital spending swap, plus the added potential contingency.
(An estimated $2.3m in sixth-year costs were to be slipped into the the 2028 census budget. Borrowing into a future budget seems par for the course, as there’s a $1.6m item for “census 2018 overspend”.)
By June, the programme was rated “generally on track”, with some “problematic risks and issues”.
In budget terms, that meant “the estimate to completion now includes the likely results of key procurements, which will exceed the budget, and has confirmed that all of the contingency held by join ministers will be required”.
Documents for last year’s Budget show Stats NZ got an extra $14.1m to “support its Treaty partners in building their data collection capability by assisting iwi to collect responses to the 2023 census in two geographic areas”, for roughly 20,000 households. An extra $2.15m was added from the main census funding pot.
Ministers also agreed to release $15.4m of the contingency, it was noted in October. “It is anticipated the remaining contingency of $17.1m will be spent,” November’s report said.
Census programme managers demanded a “very very low risk” contingency of 10 percent be put aside, to cover “known unknowns” and “unknown unknowns”. That figure totalled $18.8m, which unfortunately left a shortfall of $1.7m.
Meanwhile, costs were still going up. Increases from October, totalling $1.7m, included five roles approved “not previously forecast nor budgeted”, and a Salesforce licence cost increase of $1m. “Increase funded from contingency,” November’s report notes.
On the other side, budget trimming to offset cost increases were focused on “optimisation of paper form quantities and collection staff ie. field design optimisation”, according to the August PowerPoint.
Late last year, as the programme team grappled with multiple issues behind the scenes, the census’s high-level design were released publicly. The goal is to get the highest response rates possible to produce quality data, deputy government statistician, and census and collection operations boss, Simon Mason said.
“To achieve this, we are prioritising solutions to address barriers to participation for population groups with historically lower response rates, and we will make it easier for all to participate in the census and complete census forms.”
Daily ‘burn rate’ to rise
More widely, problems were flagged through status updates littered with red ink – marking risk levels of “high”, meaning they were “major” and either “likely” or “possible”.
A year ago, when the programme was “on track”, recruitment was behind schedule, with 12 vacancies outstanding and 31 roles to be filled by July.
Some initial staffing expectations seemed unrealistic. An extra $1m had to be found because year-two payrises averaged 1.5 percent, whereas the budget was an average of 0.5 percent.
Later, as costs mounted, recruiting for some positions was frozen.
An additional recruitment consultant, Kelly Services, was engaged through a tender – with an approved budget up to $39.5m – asking for a firm to “recruit, engage, retain and support approximately 4900 field staff across New Zealand on a temporary basis”.
Other requests for proposals included the “next level design phase”, which was won by consultancy Deloitte.
The firm proved useful. It provided an additional 15 to 20 people, and its services contract was extended to December, with discussions about an extension to June 2023.
Unsurprisingly, Deloitte was also costly. While the programme budget had anticipated spending on Deloitte “pittering out” in August, it was noted in September the spend continued at roughly $450,000 a month.
To put that in context, the daily “burn rate” – the reports’ parlance for spending – was $154,000 a day in August, while the average rate over the entire 2021/22 financial year was expected to be $217,000 a day. In the next financial year, the year in which the census actually happens, that is expected to lift to more than $500,000 a day.
(Fun fact: Forecast spending of $140m for the 2022/23 financial year will eclipse the entire five-year budget of the 2018 census. So much for saving money.)
In general, there have been ongoing concerns across many months about staff workloads, analysis and data preparation systems being behind schedule, and systems not being ready for this year’s dress rehearsal, set down for March.
Last November, a new item was added to the list of problems about an “operational address file”. The note said if dwellings or people can’t be counted properly that will result in “failure to deliver all KPIs and strategic objectives”. (Management control: “Not yet effective.”)
It could be argued these are frank assessments designed to get urgent action, and risks are being adequately managed. Yet there seems a reluctance to escalate issues beyond the programme leadership team.
Something called the programme risk management summary report from November said at the top, in bold: “All of these risk are currently under management control. There are no risks that are escalated to the board and no risks that have mitigations which are failing.” (Some, though, are “not yet effective”.)
In the 11 months of presentations, most risk items were listed as “same” or “improving”. The only one listed as “deteriorating” was the dissemination plan for census information, “which the 2023 census programme is dependent on”. It was three to four months behind schedule, and, initially at least, there was no “clarity” about funding.
The item below that warned if there were delays in the processing or analysis of data, “we will not be able to meet the early release date in December 2023”.
This was just the tip of the iceberg. An asterisk at the bottom of the page noted 11 risks had been summarised, “however a total of 43 risks have current risk rating as high or above”. Details “available on request”, the note said, blandly.
It’s worth remembering a flaw with the 2018 census was an “optimism bias”. Some status ratings “did not reflect the seriousness of the issues the programme was confronting”, reviewers said. There were also “overly positive conclusions” on the value of the census test in 2017.
The independent review said few issues were escalated to governance boards.
“Something close to 50 percent of houses will get paper this time.” – Mark Sowden
Last month, as fate would have it, Stats NZ chief executive Mark Sowden appeared before the Governance and Administration Select Committee for the statistics department’s annual review. Alongside him was Mason, the census and collection operations boss.
Committee member Woodhouse asked for forgiveness for his “low-but-steep learning curve”, as he’d only been National’s statistics spokesman for 46 hours.
Sowden quipped: “I bet it’s been a fun 46 hours!”
Woodhouse quickly moved on, criticising the last census – which had an 83 percent response rate, meaning one in six New Zealanders never filled out a form. He asked if lessons had been learnt.
Sowden enthused that bolstering the census with administrative data meant the quality of the information produced was acceptable and reliable. All recommendations from the census reviews had been implemented, he said.
“So, for example, in 2018, we had 3 percent of houses got paper proactively. Something close to 50 percent of houses will get paper this time.”
More follow-up will be done by having more field collectors. “We’ll have at least double the number of field collectors we had last time,” Sowden told the select committee.
(That may sound impressive, but in 2018 there were only about 1800 census collectors. The Stats NZ annual review response said about 4500 collectors would be employed – which begs the question, what happened to the “approximately 4900 field staff” in the 2020 tender for a recruitment firm?)
Woodhouse moved on to Stats NZ staff. Why have numbers, and those earning more than $100,000, increased by so much? That’s the labour market, Sowden replied. And costs usually increase as a census approaches.
But Stats NZ’s overall budget hadn’t dropped much, compared to the lead-up to other census years, Woodhouse pointed out. And costs were rising more rapidly, in comparison.
That’s about boots on the ground, more community engagement, and paper forms, Sowden said. Also costs had increased since the business case was done in 2019; costs for things like labour and IT procurement.
Woodhouse tells Newsroom a bigger budget for the census is understandable but increased investment should lead to a better census.
He says the risks raised in the internal monthly reports stand in marked contrast to the more upbeat commentary in reports and Sowden’s statements to the select committee. He worries the chief executive made no mention last month of potential programme cuts – like, as mentioned in the status reports, “optimisation of paper form quantities and collection staff ie. field design optimisation”.
“That needs further examination,” he says. “I think there are much deeper problems and the government needs to get on top of this – the Minister needs to get on top of this.”
* This story has been updated with comment from David Clark.
Stats NZ’s anonymised comment, in full:
The 2023 Census programme has stringent programme controls, testing and feedback mechanisms in place to ensure final decisions about census design and initiatives are informed by evidence and risks are managed.
The programme has completed a number of independent quality assurance assessments that have reported improvements since each assessment. IQANZ has conducted these assessments and rates the census programme as ‘amber’. An amber status is not uncommon for high-risk programmes of this size. An amber status acknowledges there are known risks, which should not impact success with appropriate mitigations and focus.
The census programme is on track and concluded its detailed design planning phase as planned by the end of 2021. The programme is now in a 14-month design activation phase in which census initiatives will be developed for 2023. The census programme will confirm the final detailed design of the 2023 Census in July 2022.
The programme’s approach is focused on ensuring that the 2023 Census design is robust and fit for purpose, addresses issues from the 2018 Census, and retains what has worked in previous censuses.
The census programme conducted a test of aspects of census design in early 2021 and will conduct another test of some aspects of census design from mid-February to April 2022 (called the ‘dress rehearsal’ internally). The details for this test are currently being finalised in light of the evolving Covid-19 situation.
A drawdown of contingency funding was anticipated. It is standard practice for programmes of this size. It is released in consultation with Treasury
There will be an independent review conducted by the Risk Assurance Committee and Governance Advisory Board (all independent of the census programme) in August 2022 to provide a confidence measure about the programme’s readiness for the 2023 Census.
