Outliers, precursors, and house effects in 2020 federal polling

Warning: This is a nerdier post than usual. We will take quick a look at all the (publicly available) federal polls of 2020. As usual, all these polls are listed on this page.

We will focus on the Liberal and Conservative numbers compared to the rolling 338Canada average. On the graph below, the x-axis represents the LPC poll numbers, and the y-axis the CPC numbers. The graph's origin is the 338Canada average on the middle field date of each poll:

We will look at all major polling firms in Canada that publicly released at least six federal polls in 2020. 


As you can see below, Léger has released a lot of polls in 2020. Starting with the COVID-19 pandemic in March, Léger released weekly federal numbers until the November U.S. election. Since then, Léger has moved to biweekly numbers.

Overall, the Léger polls are mostly balanced around the average. Léger had the Liberals higher than other firms in the summer months (dot further away in the 4th quadrant below):

It's worth stating that Léger had a very good year with polls in Canada. It was near-perfect in British Columbia (earning an A+ grade) in October and was the most accurate pollster in New Brunswick in September

Nanos Research

The polls from Nanos Research (shown below) show a similar pattern as Léger with dots distributed in all quadrants of the graph, but with higher variance - which is entirely expected from a rolling poll of 250 respondents per week. Smaller sample sizes and probabilistic sampling naturally leads to higher variation from poll to poll. Nevertheless, like Léger, we see no apparent House effect in these polls: 


Ipsos polls (usually published by Global News) also detected the same patterns as other firms in Canada. The six federal Ipsos polls are shown below and all of them are located near the bullseye:

Mainstreet Research

When a poll shows wildly different numbers than current trends, there are usually two explanations: 1) the poll is an outlier, either by pure statistical nature (the infamous 20th out of 20) or by poor sampling (which happens on occasion even to the better firms); or 2) the poll is a precursor, meaning it is the first to detect a new trend or a significant shift in voting intentions.

See all seven Mainstreet Research polls below:

There is one point on the far right of the graph, meaning this one poll had the Liberals a little over nine points above that day's average, the furthest point from the graph's origin.

And yet, it was not an outlier.

The same poll is indicated with a black arrow in the graph below:

In mid-March, as parts of Canada were entering their first-wave lockdown, the Liberals took a significant lead over the Conservatives. It just so happened that Mainstreet was the first poll to detect this shift. While Mainstreet was on the field for this poll, the Liberals' average was just below 30 per cent, yet Mainstreet measured the LPC support jump to 39 per cent. In the days that followed, EKOS, Léger and Innovative Research all had the Liberals at 40 per cent, and Abacus Data had the LPC at 38 per cent. 

This poll alone, clearly a precursor in hindsight, skews Mainstreet's 2020 average by +1.4 for the Liberals. 

Abacus Data

Here are Abacus Data's 2020 polls, all but one inside the 4-point radius:

The Angus Reid Institute

The Angus Reid Institute's House Effect has been documented many times in recent years, and the 2020 ARI polls all fall within the same upper-left quadrant (CPC higher, LPC lower than polling average), including four of six polls outside the 5-point radius.  

EKOS Research Associates

Here are all six publicly released federal polls from EKOS:

Innovative Research Group

Seven of Innovative Research Group eight federal polls lay in the fourth quadrant (LPC higher, CPC lower), but five of eight are located within the 4-point radius:

In Conclusion

Here is each firm's average for 2020. In short, all firms except Angus Reid are located within the 3-point radius:

While Angus Reid's average lies on the edge of the 4-point radius, it is not even that far off from its competitors with all things considered. Angus Reid may have the clearest House effect among Canadian firms (spanning several years now), but nevertheless their polls remain mostly consistent and do not swing wildly from month to month, which is a significant plus.

In fact, no polling firm appears to outright contradict other as much as Forum Research used to only a few years ago. I had drawn a similar graph for Maclean's in 2019 with polls ranging from early 2017 to May 2019. Here it is:

The Forum polls are indicated with black arrows:

No additional comments needed here. 

Ok, now back to some vacation. Take care and stay safe, dear readers. 

Philippe J. Fournier is the creator of Qc125 and 338Canada. He teaches physics and astronomy at Cégep de Saint-Laurent in Montreal. For information or media request, please write to info@Qc125.com.

Philippe J. Fournier est le créateur de Qc125 et 338Canada. Il est professeur de physique et d'astronomie au Cégep de Saint-Laurent à Montréal. Pour toute information ou pour une demande d'entrevue médiatique, écrivez à info@Qc125.com.