Statistics New Zealand has announced that it will not undertake a Census of Population. According its media release,
Wide-ranging improvements to the data system will modernise and future-proof how New Zealand’s economic and population statistics are produced… By tapping into information New Zealanders have already provided, we will deliver more relevant, useful, and timely data to help inform quality planning and decision making.
This decision was a long time coming. The Department of Statistics had been widely criticised for the low response rate to both the 2018 and 2023 censuses.
I have argued for many years that our society has changed, so it is almost impossible to do a census in the way it was done in the past. We need to understand that in our changing society, a traditional census of the population is becoming almost impossible to do. I wrote the first draft of the following article in 2019 to explain the problem.
Social Change
All the organisations that included cold-calling on households as part of their activities have given up and switched to other methods. The reason is that when you call on most dwellings, there is nobody home to respond to the call. In most households, all adults are working, so there is no one home during the day. Visiting at night or the weekend does not produce much better results, because many people work at night or the weekends, too. They are also busy in various social, sporting and cultural activities.
Only a few decades ago, many people made their living doing door-to-door sales, selling brushes, health foods and medicines, insurance and other products. Those activities are no longer profitable, so marketers have switched to party plans and on-line sales as a substitute.
Many charities undertook door-to-door collections on a Saturday morning. For some this was their main source of funding. These collections are no longer viable, because it is hard to get collectors willing to knock on doors and too many people are away from home on Saturday morning.
Church ministers no longer undertake casual pastoral visits in the way they did in the past, because it is too difficult to find people home.
Most churches no longer engage in door-to-door evangelism. Even the Jehovahs Witnesses have moved towards displays on city street corners. (Their stand beside the Bridge of Remembrance rightunder the noses of the census experts working a hundred meters in the office of Statistics NZ is a public demonstration of the social change).
A number of social changes have made it impossible activities that require cold calling impossible.
Many dwellings now have locked gates and intercoms, which make it impossible for cold-callers to enter. Many other dwellings have dogs loose in the yard, which prevent people from calling on their house. Organisations cannot demand their employs enter dwelling where the dogs could be dangerous.
More and more people are living in gated communities, which are impossible to enter, if you do not know a resident who will open the gate. The residents will often be upset, if one person lets a census taker onto their site.
People buying houses as an investment sometimes choose to leave them empty, rather than running the risk that tenants might damage the building. Collecting information from these dwellings is impossible, because no one is ever there, but collectors do not know that and waste time going back against and again.
Many people live in more than one dwelling. They go to one dwelling while they are working and go to a different one when they are off work. Some people will live a couple of nights a week with their partner, and live in their own home for the rest of the week. Children often spend every second week with their mother and the alternate weeks with their father. These fluid living arrangements make it increasingly difficult to contact people in the place where they live.
These social changes make it impossible for organisations that rely on cold-calling to operate in the way that they did in the past. It explains why most have given up and switched to other ways of operating.
The same applies to the operation of a census of the population. Attempting to contact every person living in a nation has become an impossible activity. A method that relies on calling at a dwelling and giving the forms a resident who will get the other members of the household to complete them and be there when the census taker returns for a second visit to collect the forms is an impossible task, given the way that society is structured.
A couple of decades ago, there was someone at home most of the time, usually the mother of the family. She felt a duty to do the right things and could organise her family to complete the forms. If her husband did not get around to doing it, she would complete the form for him, often without him knowing. More adult children were living at home, so their parents would persuade them to complete the forms, or do them for them. They could persuade any boarders that they should complete the forms. That has changed in modern societies.
Mothers are no longer at home, because most are working. Finding someone at home who will take forms and undertake to return them when they are complete has become much more difficult. I presume that women have always been much more conscientious about these things than men.
One person no longer has the moral authority to tell another member of their household what to do. It is no longer acceptable to complete forms for other people, unless they are children or medically incompetent.
Young adults no longer live at home. They often live together in flats. There is no one with moral authority to badger people into completing a task if they do not get around to it.
Once neighbours could tell a census worker quite a lot about the people living the house next door. Now they often don’t know the people who live there, and they would be unwilling to give a government official information about them if they did.
Many people have holiday homes that they live in during the weekends. This makes them harder to contact.
When undertaking a census, it is not sufficient to contact a household once. They have to be contacted at least twice, once to know how many forms to leave, and a second time to take the completed forms back. That is hard, but if a household is tardy in completing the forms, a census worker may have to contact a household them three or four times before they obtain the completed forms. In a society where people are frequently not at home, this becomes very difficult to do.
All over the world, statistical offices are finding it increasingly difficult to run population censuses using door-to-door visits. Many have switched to other methods for collecting information.
Allowing on-line collection is a partial solution to the people. Mobile people who are hard to contact at home, are usually comfortable sharing information on-line. Young people are comfortable sharing information about themselves on Facebook and Instagram. An on-line census is a much better way to collect information from this generation. However, it does not solve the problem, because the people who are not comfortable sharing information on-line will get more and more difficult to contact.
Social change takes place slowly, so the consequences appear slowly. However, they are most serious for activities that need participation by 100 percent of the population. A door-to-door salesperson only needs to find a few people at home for their activities to be economical, but most have still given up. The problems are much greater for census takers, because they need to find 100 percent of people at home, which is much harder to do.
Anti-Government Attitude
In the past, people trusted their governments and did what they told them to do. If they were asked to complete a census form, they felt obligated to do it. That has changed.
The proportion of people who do not trust their government and are hostile to its activities is growing. These people do not trust the government, so they do not want to give it their personal information.
The declining trust in government is reflected in voter participation in government elections. In recent elections, parliamentary elections more than twenty percent of the population do not vote. (Only 6 percent did not vote in 1984). More than half of the population do not vote in local government elections.
The decline in voter turnout reflects a declining obligation to support government activities. If twenty percent of the population did not bother to vote in elections for the government that can control their lives, then it should not be surprising if half of them did not complete a census form that has very little impact on their lives.
Our migrant population has increased rapidly. Many migrants have come from nations that use the information people provide to monitor their behaviour and harass them. When they come to New Zealand, they are often reluctant to give information to the government, because they fear it could place them in danger.
Busy Lives
Modern people live busy lives. Completing a census form might only take half an hour, but many modern people do not have half an hour to spare. So even people who are happy to complete the census, often don’t get around to doing it. We all have things that we intend to do, like phone an acquaintance, read an article, repair a blown light bulb, weed the garden, but we don’t get it done, because we get busy with other things. Things that we want to do get pushed down the list, and eventually forgotten.
A census form often just gets forgotten. People intend to do it, but then something happens to distract them: extra hours are offered at work, a relative dies, rugby training starts, Game of Thrones is on TV, a friend gets cancer, so the census form gets forgotten. No one asks for it, and before long the time for responding is over, without the form being completed.
Very few people get prosecuted for refusing to complete the census. The fine is only a few hundred dollars, so fear of prosecution is not a real incentive.
When people give examples of how a census should be done, they assume the easy examples: a conscientious person with lots of time. Unfortunately, a population census can be wrecked by the small proportion of people whose lives are not ordinary.
Response Rates
Censuses have always been hard to do. The politicians seem to be obsessed with response rates, but they are just one measure of quality, and they are a slippery concept to measure. The response rate is the number of the population responding divided by the total population. The problem is that the denominator in this formula is not known. To estimate the percentage that has responded, you need to know the size of the total population that you are trying measure. That number is not available, because it one of the figures that the census is trying to measure. You only know the number that responded, so a response rate can only be produced by guessing the number that should have responded.
When the census relies on contacts with household, you need to estimate the total number dwellings that are occupied by people, so that you can measure the number of households you should have had a response from. The total number of dwellings is known, but unfortunately, it is hard to count the number of dwellings that are unoccupied, because their status is not obvious, even if you knock on the door. The occupants might be away for the day.
So response rates have to be taken with a grain of salt, particularly those from the past, when guestimates of the total population being surveyed were much weaker. And I presume that diligent statisticians have always found clever ways to tweak their estimates to make their response rates look better. That probably explains why a 95 percent response rate was achieved in 2013.
Non-response is unavoidable in every statistical survey, even if it is mandated by the government. The key issue is what is done about the non-response. Statisticians have robust techniques for dealing with non-response, so it is not really a serious problem, provided it is well managed.
Complex Organisation
Most businesses start small and grow gradually. They learn and develop as they expand. A census has to be undertaken within a few weeks. Organising a census means taking on thousands of employees all at once and training them for a task they have not done before. They cannot be employed for too long, because the cost would get too high. No matter how good the screening process, some of the employees selected will be duds. The problem is that by the time that is discovered, the census is finished and the damage is done.
Most businesses develop and adapt as they expand. A census has to be completed in a couple of weeks. This means that there is very little ability to adapt and resolve problems as they emerge. By the time a pattern of problems emerges, it is to late to make a significant change, because by that time the census period is nearly complete.
Getting good people to work on a census is not that easy. Once there was a large pool of family carers at home, who were pleased to have a temporary job that would allow them to earn a few extra dollars while their children were at school. Many of these had done administrative jobs before they had children, so they were well-suited to census work. That situation no longer exists. The pool of carers and retired people who could do the job are already employed.
Census work cannot be done while children are at school, as that is the time when the fewest people are not home.
Most people do not feel comfortable cold-calling on people they do not know.
People do not want to go onto a situation where a dog is loose, but that is part of the job. Several census worker get attacked by dogs each census.
Census workers will get yelled at by people who are angry with the government. That makes the job quite unappealing.
Getting good people to be census workers is increasingly difficult.
Poor Quality Information
A population census relies on self-complete forms. These have to be designed so they can be understood by everyone. This is very difficult. The form has to be designed for someone with a reading age of about nine. The questions have to be kept simple, so they are easily understood, but this is not always possible because even simple questions can be complicated for some people.
Income is an example. At first thought, it would seem easy to collect. For some it is, but for others, it is not, and a census form has to cope with the tricky cases. Some people know the amount of their take-home pay, but they do not know their gross pay. People on casual jobs, with several employers find it very difficult to say what their income is for a particular period. Self-employed people often do not know their exact income until well after the end of the financial year. The income questions in a population census have to be sufficiently vague to make the information provided fairly useless.
The ethnicity of New Zealanders is increasingly diverse and complicated. This means that collecting information about ethnicity in a self-complete form is very difficult. If people do not understand the concept being asked, there is no one to ask. If too many guide notes are put in the form, it clutters the layout and makes it hard to follow.
There is always pressure from public agencies to add extra questions to a census form. A decision has to be made on how long the form can be without becoming an obstacle to collection. When completing voluntary form people get tired, so if they encounter questions that they find hard to answer, they often give up. The census is a long form, so not surprisingly, many people give up before they get to an end.
Some people are mischievous when they are completing a government form. They put in false information, just to be funny. I understand that in one census, a dozen people reported an occupation of Prime Minister, and several said they were the queen. People put mischievous responses when they are reporting their ethnicity or religion. This natural behaviour cannot be stopped, but it reduces the value of the information collected.
If governments need information about people to support social policy, they would be far better to use voluntary interviewer-administered surveys of representative samples. This is a far cheaper option than trying to take a full census. It is more efficient, because a social survey can provide detailed information that is far more useful. It will also produce more reliable information, because people are less likely to lie to an interviewer.
Governments have Information
The reality is that governments already have most of the information that they attempt to collect in the census. People supply this information in the forms they complete when engaging with the government for various services. Statisticians call this information “administrative data” as it is collected through administrative processes.
The government knows the number of people living in New Zealand. It can derive that from the numbers of births and deaths registered, and from immigration data reporting the numbers of departures and arrivals. Being a couple of islands, makes this much easier than it is for countries on large continents with fairly porous borders. Statistics NZ could continue producing reliable population projections even if there was never another census of the population.
Inland Revenue has accurate information about the incomes of all New Zealanders. They know exactly how much each person earns. Governments would be better to use this information than relying on vague information supplied in a self-completed form.
When people apply for National Superannuation and Social Welfare Benefits, they have to supply a huge amount of information about themselves. When families apply for “Working for Families” benefits, they have to supply information about their family and their partners. Governments are collecting data about us all the time, often without us being really aware of it. They should use this information better, rather than relying on expensive and unreliable information from a census.
The government says it needs census information to allocate health funding. Using census data is actually a lazy way to do it. The Ministry of Health collects information about people who visit doctors and more detailed information about people who get hospital treatment. The Ministry of Health already knows where the sick people live, and what is wrong with them. Statistical techniques are available for projecting this information forward, so governments should be using health information to allocate health funding. This would be better than relying on census data that does not record anything about health needs.
Governments also say that they need census data to know where to build schools. That is nonsense. The Ministry of Education has information about every child attending school and preschool or kindergarten in New Zealand. They know about every child born, through the birth registration process. The schools provide a statistical return to the Ministry of Education twice a year. This a better way to obtain information about social problems at the schools. The government should use this information that they already have to allocate education funding, and make sure that it is directed to where it is needed, rather than relying on census information that tells them very little about the education needs of students.
Building consents provide information about districts where new housing is being built. Using the information about the type of dwellings, it is quite easy to estimate the number of children that will be moving into these new residential areas. The government has this information before the houses are built. This should be used when deciding where to build new schools. If they wait for census information to be available, they will build the schools too late to meet the need.
Census information is used to shift the electorate boundaries for the general election. Running a census is a fairly inefficient way to do it, because, by the time the data is collected and the boundary adjustments made, the information used will be well out of date. Through the land information system and valuation process, the government knows where every dwelling in New Zealand. is. Through the building consents process, they know where new dwellings will be built, even before they are built. It is quite easy to estimate the number of people in each dwelling. The need for changes to the electorate boundaries mostly come from new housing development. Using the information about dwellings that they already have, governments could establish electorate boundaries that are fit for purpose.
Conclusion
Operating a population census in a modern society is becoming an increasingly impossible task. That does not matter, because governments have far better ways of obtaining information.
