What happened when Canada stopped counting its numbers

TORONTO — When a major Western country stops counting its numbers, bad things can happen. 

In June 2010, the Canadian government unveiled a grand experiment in data collection. In the name of privacy, Prime Minister Stephen Harper ended the mandatory long-form census for the country and swapped it out with a voluntary survey.

Five years later, there is a mass scramble to make sense of a rapidly changing country. Despite an explosion of corporate data-mining in most nations, researchers interested in tracking poverty, immigration and public health in Canada know less and less about the country as time progresses. They’re not, for example, entirely sure if income inequality is accelerating, stagnant or closing. Across the nation there is a loud, collective uneasiness among them.

Munir Sheikh was Canada’s chief statistician when the long form was killed. He publicly resigned in protest the next day.

He’s still not a fan. Running a country without hard data on its citizens, he said, is “like flying a plane without any guess as to where you’re going.”

“We’re not sure” has become the rallying cry of researchers, urban planners, health officials, policymakers and businesses. With its long-standing yardstick gone, the fogginess of new data means less and less is known about one of the world’s major economies as each year passes.

While Canada is one of the world’s wealthiest countries, it is faltering when measured against comparable nations. Modern economies rely on hard benchmarks against which data are measured. Even countries that do not hold a regular census, such as Germany, have measures in place to gain the crucial data needed to act as a standard. With crude oil prices in free fall, determining population and employment numbers is imperative for areas of Canada that depend on lofty prices per barrel. 

But even today, the government maintains that its voluntary survey has the integrity of a census, and there is no will in the party to reinstate its long-form predecessor.

Since 1971, every five years, each Canadian household has been mailed a census. Of these households, 80 percent received a short questionnaire with eight questions. The remaining 20 percent were sent the long-form version and were legally compelled to answer an additional 53 questions about income, education, housing and family structure.

In June 2010, the Conservative Party argued that the long-form census was too personally invasive. This argument was buttressed by the alleged thousands of complaints received by the former Industry Minister Maxime Bernier (a claim that was later withdrawn). A subsequent investigation by The Globe and Mail, one of Canada’s largest newspapers, revealed formal complaints against the census had been steadily decreasing over the years.

As a remedy to perceived government overreach into private life, the Conservative government kept the mandatory short questionnaire in place but ended the mandatory long-form part and replaced it with a voluntary questionnaire they called the National Household Survey (NHS).

Foreseeing trouble in achieving the same response rates as a mandatory survey, additional copies were distributed to fill the anticipated response gap. In 2006 the response rate for the long-form census was 93.5 percent. In 2011 the NHS response rate plunged to 68.6 percent. To Sheikh, the reason is simple.

“If you tell somebody, ‘Please fill out a survey,’ a lot of people will ignore that. If you tell them ‘Please fill out this survey, it’s the law of the land, you’re going to break the law,’ they complete it,” he said.

The government, he noted, is gifted with a singular power: the authority to compel citizens to answer a survey.

“Anybody can make a voluntary survey, but you can’t force anyone to respond to your questions. Whatever you get will be flawed because it doesn’t represent the country.”

While Sheikh is perhaps one of the most vocal in his frustration over the state of data in the country, he is by no means alone.

Economist Sheila Block suspected that, despite being a diverse country that prides itself on a policy of multiculturalism, Canada still has race issues that permeate the workplace. As she and her colleague Grace-Edward Galabuzi sorted through reams of employment data from the 2006 national census, they found results that dismayed them.

“We have a lot of evidence from previous censuses and other work that we have racism and discrimination in the labor market in Canada,” said Block, a senior economist at the Canadian Institute for Policy Alternatives.

Harvey Low, the manager of social research and analysis for Toronto, is a fast-talking, passionate numbers man. He worked with his first census as a university student in 1987 and later relied on the census data as an urban planner and social worker. For his entire career, he has been analyzing and reporting on survey numbers.

“We need to understand the profile of who we serve and the places where these people live, because we provide services by areas, by geographies and by neighborhoods,” he said. “How has that yardstick moved? If we’re investing in trying to get youth more jobs, we would like to know how that’s changed over time.”

This data deficit is cascading into other areas, with case after case of researchers and policymakers stopped in their tracks. Countless examples of hobbled researchers are emerging with increasing frequency as sources of reliable national data slowly dry up.

From 1981 to 2006, Lars Osberg, a professor of economics at Dalhousie University in Nova Scotia tracked the rise of income inequality in Canada. Because of the inconsistency in data in the 2011 NHS, the study is effectively terminated, as there is no way to consistently link the NHS to previous censuses.

On May 7, 2014, the premier of New Brunswick held a news conference to concede that, because of the inconsistent data provided by the NHS, there is uncertainty whether a province-run program tasked with reducing poverty is effective, because accurate measurement is so difficult. 

The University of Toronto’s David Hulchanski developed a methodology that many called groundbreaking for tracking urban poverty. He created maps of Toronto that visualized the clear dividing zones of income inequality. After the success of that project, he developed maps for Montreal, Vancouver, Halifax, Winnipeg and Calgary and Chicago in the U.S. Of all those cities, the only ongoing project with usable data is Chicago. 

The University of Toronto, along with other postsecondary institutions across the country, has called for the return of the census.

But it isn’t just researchers who are outraged.

Businesses are also expressing frustration with the diluted data previously used to understand employment trends and demographics. The Canadian Chamber of Commerce, representing more than 200,000 businesses across the country, is advocating bringing back the long-form census. “We will no longer be able to measure the effectiveness of our social programs and economic policies,” the Chamber wrote in a policy note. Business schools have written editorials arguing that the new census harms Canada’s ability to compete with other nations.

What irritates many who work with data is how a technical issue became fodder for politics. Throughout the controversy, economists and researchers have repeatedly pointed out that a voluntary survey can never replace one that mandates a response.

In his resignation letter posted to the Statistics Canada website (and later taken down), Sheikh was forceful in his disagreement that a voluntary survey would be of practical use.

“I want to take this opportunity to comment on a technical statistical issue which has become the subject of media discussion,” he wrote. “This relates to the question of whether a voluntary survey can become a substitute for a mandatory census. It can not.”

Mandatory surveys allow statisticians to identify and sample subpopulations, such as aboriginals, immigrants and low-income earners. After demographic concentrations are identified, they can be factored into additional voluntary surveys to ensure accurate sampling. Sheikh likened the census to an anchor. All surveys conducted in the country, because they’re voluntary, must be anchored against a reliable set of data.

“Anyone who runs a survey without the census will not be able to adjust their numbers to accurately reflect the nature of the country,” he said. “It doesn’t matter who does the survey — if it’s in the private sector or the provincial government or the municipal government or by an academic. The census is the foundation for all data that we collect.”

The United States Census Bureau examined the voluntary versus mandatory question and concluded that because of excess cost and significantly decreased response rates, it made little sense to hold a voluntary census. 

Ted Hsu, a former scientist and investment banker, likes to be able to measure what he’s doing and is open to changing his mind when his measurements show he’s wrong. Now a member of Parliament for the Liberal Party, he applies the same reasoning to government.

“If we’re spending money on programs trying to help people, we should be measuring if it’s having an effect,” he said. “And if it’s not having an effect, we should cancel the program and save money.” In September 2014 he introduced Bill C-626, which aimed to restore the long-form census.

After two readings in the House of Commons, a glimmer of hope emerged on Feb. 4 as C-626 was put to a vote. The bill had the support of not only every member of the opposition in the House but also more than 475 businesses and policy-oriented groups, ranging from left-leaning think tanks to the former chief economist of Toronto Dominion Bank.

The bill was defeated, with only one Conservative Party member breaking ranks.

Advocates for the voluntary survey received another victory last week as Wayne Smith, Sheikh’s replacement as Canada’s chief statistician, claimed the data from the NHS was “better than he had expected.” Jake Enwright, the press secretary for Industry Minister James Moore, echoed these comments and added that “the National Household Survey yielded high-quality data.”

Low, the numbers man, concedes that not all the data drawn from the NHS is useless. On large scales, for example, for entire cities, conclusions can be drawn. However, the wealth of detailed information that accompanied the long-form census is lost, as is the ability to understand small pockets and neighborhoods around the country.

“We’re moving away from evidence-based decisions to … I don’t know what. I don’t know what the label for it is,” said Day.

Five years later, Sheikh remains one of the harshest critics of the NHS. “There is a truth out there. The census gets the truth to you. If there’s no census, you have no idea what’s going on in the country,” he said. Of those who wish to keep the voluntary survey in place, he wryly added, “If you know what the ‘truth’ is, you don’t want to be burdened by other evidence.”