Why Canada's monthly jobs numbers don't tell the whole story

Once a month, Statistics Canada releases its latest data on the state of the Canadian job market.

News coverage of the Labour Force Survey often focuses on one number above all others: the net job change from the previous month.

Political players pick up on those headlines, citing every positive jobs number as proof that the government is handling the economy well, and every negative number to suggest it isn't.

Among the people who spend the most time interpreting and analyzing the reports, though, the reaction is very different. Some economists have compared their output to a "random number generator," arguing that it is virtually useless for single-handedly interpreting the state of the Canadian economy.

"We tend to view the LFS as not as reliable as some other surveys," Katherine Judge, an economist at CIBC World Markets, told CTVNews.ca.

University of Manitoba economics professor Gregory Mason agrees that the "breathlessness" with which the monthly survey is greeted in some quarters may be out of proportion to its actual usefulness.

"It's not a great leading indicator. We really wish we had something much more sensitive – but in the labour force, that's the best we can do," he said in an interview.

WHAT'S THE PROBLEM?

Hearing that the Canadian economy lost 2,200 jobs in June or gained 106,500 in May might make one think that StatCan has done a thorough accounting of every position added or removed in the country.

That's not the case – and considering the updates are often released just a few days into the new month, perhaps it shouldn't come as a surprise.

StatCan researchers gather the data for the survey by contacting 54,000 of the more than 14 million households in the country and asking questions about labour-related matters including employment status, hours worked and wages. They recruit participants on a six-month basis, meaning 9,000 households are replaced every month.

"The LFS … is the most timely employment survey, which is why it garners so much attention, but it also is extremely volatile," Judge said.

"You're not tracking the same people over time, so you can get lots of discrepancies that way."

The agency strives to survey a representative sample of Canadians, but the regular turnover creates the opportunity for sudden changes in the data – particularly when it is broken down to a provincial, municipal or sector-by-sector level.

The unemployment rates most commonly cited in the media are based on a three-month moving average, rather than just one month's data, and adjusted for time of year – but even with those adjustments, StatCan cautions that the small sample size makes the data prone to significant swings.

That has certainly been the case this year in Guelph, Ont., which recorded a 1.9 per cent unemployment rate in January – by far the lowest in the country – and followed it up with a 1.7 per cent rate the following month.

Unemployment in Guelph has steadily increased since then, with the June figure of 5.4 per cent putting it solidly in the middle of the pack nationally. But does that mean far more people in Guelph are out of work now than were a few months ago, or does it mean the sampling has changed to include more people without jobs? It's not clear from the data.

Changes in the sample aren't the only reason some of the survey's results can vary significantly from month to month. Natural disasters can cause significant temporary disruptions, and even the rise of job-hopping can skew the results for any particular month.

A StatCan spokesperson said in a statement to CTVNews.ca that variability in Labour Force Survey data "varies from month to month and is generally small."

Monthly survey results are revised after the fact as the agency compiles more data – but those adjustments and corrections come long after the initial numbers have been trumpeted across the country by the media and politicians.

IS THERE A BETTER WAY?

The Labour Force Survey gets all the attention, but it isn't the only way StatCan measures the state of the job market.

The Survey of Employment, Payrolls and Hours, or SEPH, also comes out every month. Unlike the more popular survey, it isn't based on a sampling of phone calls – it comes from payroll information submitted to the Canada Revenue Agency. This means the results in SEPH can be much more accurate, as it comes from a larger pool of data and reduces the chances of human error.

There is one problem, though. Because the payroll data takes longer to collect, SEPH results lag two months behind the Labour Force Survey. When the Labour Force Survey is purporting to explain what happened to the job market in June, SEPH data for April may seem like old news by comparison.

SEPH also falls short of the Labour Force Survey in one important way: Because it's based on tracking payroll data, it has no way of accounting for people who are not in the workforce, meaning it cannot be used to track Canada's unemployment rate. Even in the U.S., where a SEPH-like product is the norm for monthly jobs numbers, something similar to the Labour Force Survey is used to track unemployment.

StatCan says that, while the Labour Force Survey "provides a timely, reliable, and accurate picture of the Canadian labour force," anyone looking for "a more comprehensive portrait of the Canadian Labour Market" should combine insights from the survey with those found in SEPH and other StatCan products.

Additionally, the agency says it is important to consider each data release in the context of longer-term trends instead of placing undue influence on the numbers for any one month. Mason agrees, likening it to a football team judging its performance based on the success of an entire drive, and not any one play in isolation.

"They get a wonderful gain, and get first and goal to go, and then they get thrown back five yards – well, you're still in contention. You're still very close to the goal line," he said.