The first release of census data has been delayed by two months. David Williams reports
Last month, Stats NZ staff gathered in offices in Wellington and Christchurch, and beamed in from other climes, to take stock six weeks after census day.
The official line from the census programme board meeting minutes was: “Overall field collection for the 2023 census is continuing to track to plan.”
Unofficially, the picture seemed chaotic and calamitous.
READ MORE: * Census turnout low despite $37m blowout * Everybody counts (but not twice, please)
Cyclone Gabrielle hit the day field staff were meant to begin knocking on doors, leading to a suspension of activities. The Government urgently approved a further $36.7 million, mainly to extend ground operations for affected areas.
By April 19, when the census programme board met, returns were going badly.
Responses from Māori and Pasifika were significantly lagging behind the national rate. These groups were meant to be a priority after the disastrous 2018 census.
Unfortunately, the areas worst-affected by the cyclone were also those with some of the lowest response rates in 2018.
Taxpayer money was being shovelled out the door to turn things around.
On top of an extra $1m advertising spend, there were a flurry of community engagement events, including with the Warriors rugby league team, to lift returns. Vouchers of up to $40 would be offered as an enticement – potentially costing another $1m.
Then there were the data collection and processing systems. Because of a lack of testing they were delivered late, which would cause yet further delays and increased costs.
The start date of the so-called post-enumeration survey, which provides the final, official census response rate, was pushed back three weeks to May 22.
It was possible, because of the cascade of delays, the first census data release would be pushed back from March next year to May.
Of course, the budget blew out.
An extra $8m to increase pay for field staff in a competitive labour market, the urgent $37m after the cyclone, and, unfortunately, because of an error, a further $7m-8m of work which previously hadn’t been accounted for.
Three years ago, then Statistics Minister James Shaw announced a census budget of $210m, with a $33m contingency quietly tucked away.
Now, the overall figure is estimated to be $317m. The contingency has reduced to $7.6m.
There’s already talk of needing $8.8m to prepare for the 2028 census.
Simon Mason is deputy government statistician and deputy chief executive of census and collection operations. He confirmed it was unlikely the census would achieve several key performance indicators, including those involving response rates from Māori and Pasifika.
“But we expect to get close to the KPI target of 90 percent for the national response rate.”
Raw figures show the struggle ahead, with estimates that more than 600,000 individual forms remain outstanding.
“It is too early to comment on data quality as the census forms have yet to be processed, data analysed, and the combined methodological approaches for data variables to be included,” Mason says.
The census board has agreed to moving the first data release to the end of May next year, he confirms, to reflect the extension of collection operations. “This release will include high-level counts of the population, dwellings, and Māori descent only.”
The situation disappoints Andrew Sporle (Ngāti Apa, Rangitane, Te Rarawa), a member of Te Mana Raraunga, the Māori Data Sovereignty Network, who works in the University of Auckland’s statistics department.
He thinks the census chickens are coming home to roost. “This was predicted by Stats NZ when they put up their budget bid to Cabinet [in 2020] and Cabinet chose to fund the option with the second-worst equitable response out of five options.”
As reported by RNZ in 2020, Cabinet’s favoured option was only $6.8m higher than one described by StatsNZ as the “minimum viable” which would likely produce poor quality data.
Census data is used to draw electoral boundaries, and is vital for decisions about health spending and where to build new schools. Three years ago, Sporle told RNZ that Cabinet, by picking the fourth of five options, decided to run an inequitable census for Māori, risking the third bad census for Māori and Pasifika people.
Sporle told Newsroom: “The supplementary funding was useful but too little and too late. Census planning is a five-year process, and Stats NZ had to get ahead of this as it’s a trend seen at least since 2013.”
“We are missing Māori responses EVERYWHERE.” – Comment at April 6 meeting of the census programme board
The meeting minutes we’ve referred to are part of a 370-page response by Stats NZ to our Official Information Act request; a request designed to capture what was being said by the census board as the nationwide survey got into gear.
It makes fascinating, and at times concerning, reading.
The papers show Stats NZ originally asked for $40.4m after the cyclone, but Cabinet lopped off the proposed contingency. After the sum was approved, Stats NZ was obligated to provide regular updates not just to Statistics Minister Deborah Russell, but Minister of Finance Grant Robertson.
A financial briefing noted the “burn rate” of spending would average more than $1.2 million a day in March. Between March 6 and March 12, the “advertising sales rate” was $3m.
Even before census day – officially March 7 – concerns were being raised about low responses in cyclone-hit areas, which could lead to lower quality data, especially for iwi affiliation.
Minutes from the March 2 census programme board meeting state: “The board has communicated to ministers and other stakeholders there is a likelihood we miss our KPIs.”
A “declining daily response trend” was noted between March 2 and March 4. While census advertising was paused and then reintroduced after the cyclone, a briefing in early March said there was “low interest in our messaging” and “census is not a priority for many”.
People were aware it was on, but that did little to shift the intention to participate.
“While overall census awareness is on track to be in high 90 percents on census day, due to the pause in advertising (and longer pause in cyclone affected areas ...) we’ve had low reach (less than 20 percent with Māori men and Pasifika men).”
A March 4 briefing said post-cyclone field operations in Hawkes Bay and Tairāwhiti were likely to be “assist lite” – paper packs for all, fewer follow-up visits, and an offer of “minimum data capture”, which is a truncated number of questions for people who otherwise refuse to participate.
No one in those areas would be prosecuted for refusing to fill in forms.
By mid-March, a briefing reinforced the likelihood that certain key performance indicators – the justification for taxpayer investment in the census – were unlikely to be met. They included a national response rate of at least 90 percent, and response rates of 90 percent for Māori and Pacific people. “But context can be provided when the results are reported,” the briefing said.
Briefing papers said the KPIs should stand, although other options put forward were updating the indicators or developing a new set – the latter of which was considered “likely not appropriate”.
Low responses rated ‘level 3’
The April 6 meeting of the census programme board noted the operational leadership team was treating the low response rate for Māori and Pacific peoples as “level 3” incidents, the most serious ranking.
The discussion is worth noting in full, with the original emphasis.
“It has been noted in the data received so far that we aren’t missing Māori responses in traditional regions or areas – we are missing Māori responses EVERYWHERE.
“We are also one of MANY agencies looking to engage with Māori households – including the health sector promoting vaccinations, electoral commissions promoting electoral roll and Māori roll – equally we can’t afford to step back.
“We need to trust and have faith in our collection teams and do something a bit different. Winter is approaching, promotional events will need to be indoors and offer a real incentive to come along and participate in a community that have many other priorities to focus on.
“What is that thing we can do – if we do nothing, we are flat-lining our responses and wasting our time. We’re just the next government rep knocking on the door.
“However, if we incentivise individuals, this will only encourage people to hold off to the last minute to get the prize – but if we can incentivise groups within a community we may get a better response (eg You give us x amount of responses, we can donate to a specific group/cause/sports team in your community).”
One of the many lessons from the 2018 census was people were more likely to participate if approached by local people. But this month, census has reverted to using “flying squads” to target areas “that need more visits and assistance”.
Pop-up collection events include visiting “large employers, marae, sports events, MSD and community probation”. (Locally led initiatives in Te Tairāwhiti included a Defence Force flyover of Tolaga Bay on Anzac Day “with a trail of pink cloud”.)
Last week, as individual forms returned reached 4.5 million, Stats NZ said it was sending final notice packs to 55,000 dwellings – those from which it has had no response.
Field collectors finish up on June 1; community engagement teams on June 4.
In these final few weeks, to push up returns, field staff have been told to use the “final visit contingency” of minimum data capture.
Stats NZ is working with Whānau Ora Commissioning Agency and South Seas Healthcare to lift responses in Auckland, with activities lasting until June 30 – the final day to submit forms online.
Confident of high-quality data
Simon Mason, the deputy government statistician, says: “The combined census model includes the use of alternative data sources, which means we are confident we will produce population data of a high quality. However, it is too early to understand what data quality will look like for all variables.
“Variables that do not have alternative data sources will be most impacted for population groups or geographies that have had low response.”
(In February interviews, Mason said a census could be involved in disaster recovery, a reference to extra questions added to the 2013 census after the Christchurch earthquakes. But at its meeting on March 2, the board meeting paused work on the post-cyclone collection of “additional data” because of significant pressure on staff.)
The Official Information Act documents released to Newsroom reveal Stats NZ has had problems with its data processing systems.
As noted earlier, raw data has included forms from overseas visitors, skewing figures in tourist areas. Also, statisticians haven’t been able to discern how many forms have been fully completed, and the number only partially filled in.
The OIA documents reveal Stats NZ is considering a switch in its public reporting, from individual form return rate to the “operational collection rate”. According to one briefing, this could be done on June 2 – a day after field collection ends in cyclone-affected areas.
(A final response rate might not be known until October next year, once numbers are crunched from the post-enumeration survey.)
This may lead to confusion, the briefing said, but “external users” prefer to hear about “rates” than “tallies”.
A further problem has been flagged. Though the overall rate will be higher than the existing tally of returns, collection rates for Māori descent, Pacific and Asian ethnicity will drop by several percent.
If the change is made next week, it seems likely confusion about the new census numbers will be paired with more disappointment.
