OTTAWA -- Prime Minister Justin Trudeau is standing by his comments to a heckler last week after telling her she was intolerant.Trudeau was speaking at a rally in Sabrevois, Que., when a woman started yelling that she wanted back the $146 million from Quebecers that he's "given to illegal immigrants," referring to the amount the Quebec government says asylum seekers have cost since a recent surge in irregular arrivals.

"That intolerance with regard to immigrants doesn't belong in Canada. That intolerance with regard to diversity, you don't belong in Canada," Trudeau said in French Thursday night.

"Madame, your intolerance doesn't belong here."

The woman then asked Trudeau if he was tolerant toward "Quebecois de souche," a reference to Quebecers who can trace their ancestry to the earliest French settlers.

In a video posted online, Trudeau can be heard telling the woman her racism isn't welcome.

The woman who confronted Trudeau was later the subject of debate on social media over her ties to white supremacist groups. She is a member of the Facebook group for Storm Alliance’s Quebec wing. The group says it is "made up of Canadian citizens whose goal is to make our Country stronger, to make it a Country that takes care of its own communities." It was part of a protest last year at the Lacolle, Que. border crossing, against a rise in asylum seekers.

Speaking in Ottawa Monday, Trudeau said he won't "flinch from highlighting the politics of division."

"It's important that we have people who come at problems from very different perspectives because that actually ensures that we solve them right," Trudeau said.

But "I will point out, and I will not flinch from highlighting, when the politics of division, of fear, of spreading misinformation, is actually harming the fabric of this country... So while I will, as I did in the previous election, stay positive and not engage in personal attacks, I will also be very sharp wherever we see significant policy differences and whenever someone is pulling up intolerance and playing fear as a way of getting elected."

Trudeau referred to "people who are trying to feed fears and intolerance."

"One of the things we are facing in the world right now is a rise of extreme populism, of the politics of fear, division, of a kind of polarization that has short-term political advantages in some cases, but ends up creating fault lines within communities," he said.

"Hiding behind half-truths and torquing up fears is something that I and our government will always call out. Not just because it's a problem for us, but because it's a dangerous path for any democracy to be on."

Asked to whom Trudeau was referring, his spokeswoman pointed CTV News to his speech Sunday night after he was officially nominated to run again for the Liberals in Montreal's Papineau riding. Trudeau told the crowd that the Conservative Party under leader Andrew Scheer is the same as it was under Stephen Harper, and that it exploits fear for political gain.

"Unfortunately even as my conviction in positive politics hasn't changed, our opponents' belief in the politics of fear and division hasn't changed either," he said.

Though, in a series of tweets Monday evening, Scheer took aim at Trudeau, saying "no one has done more to divide Canadians" than he has.

Scheer called Trudeau's comments to the heckler "vile personal insults," and said the prime minister is at fault for polarizing the debate and shutting down criticism of the government's record.

Conservative Quebec MP Maxime Bernier has recently spoken out against "more diversity," sparking condemnation even from within his party.

"Why should we promote ever more diversity?" Bernier tweeted on Aug. 12.

"Trudeau's extreme multiculturalism and cult of diversity will divide us into little tribes that have less and less in common, apart from their dependence on government in Ottawa... Cultural balkanisation brings distrust, social conflict, and potentially violence, as we are seeing everywhere. It's time we reverse this trend before the situation gets worse. More diversity will not be our strength, it will destroy what has made us such a great country."

Pollster Nik Nanos told CTV News that Quebec will be a key province for the Liberals if they want to hold on to power, making this issue a political hot potato for them.

"The polling suggests that many Quebecers are attached to what I’ll say is a more traditional view of Canada, Canada as a bilingual nation," Nanos said.

"The other thing is that Quebecers, you know, they are welcoming of immigrants and refugees but they’re very rules-based. They don’t like queue jumpers and they don’t like dramatic change, and that’s why this refugee, immigration issue is so potentially explosive in the province of Quebec."

Both Quebec and Ontario have asked for federal funding to cover their costs given an influx of asylum claimants since 2017 crossing the border between official entry points.

The government of Ontario puts the cost at $200 million, including social assistance, temporary housing, primary and secondary education spaces, and additional supports.

Ontario minister responsible for immigration Lisa MacLeod said Monday that Trudeau's comments to the woman were unfair.

"The prime minister, when confronted with some of the problems his government has created, turns around and fear mongers and calls people 'un-Canadian' or 'racist' that really debases the debate that we’re having and it’s a very serious one," she said.

With files from CTV News' Rachel Aiello