The Incompetent Pollsters Of New Caledonia

Although the attention of most of the people I happen to know this week was on the election here in the United States, about which I have little that I want to say, there was another momentous election that happened this week which received much less attention here.  I speak, of course, of the first of the independence referenda to be held as a result of the Noumea accords that settled the conflict between New Caledonia’s indigenous Kanaks and the French government.  The referendum by a thirteen-percent margin showed that a majority of the people eligible to vote in New Caledonia opposed independence.  However, this vote carried with it a somewhat ominous reminder of the area’s sectional divisions because the people near the capital voted heavily in favor of remaining a part of France while those areas distant from the capital voted heavily in favor of independence, suggesting a deeply divided and regionally divided area that may have future problems and disagreements concerning the status of New Caledonia.

There is more to the referendum, though, than this.  Recent events in Scotland, where a similar referendum was narrowly defeated in favor of union, and in Catalonia, where a referendum in favor of independence resulted in the attempt of the Spanish government to overthrow the autonomous Catalan government, have demonstrated that putting the question of independence to a vote does not necessarily settle an argument.  The whole Brexit controversy only adds to that concern, in that it is one thing to make a vote, and another to deal with the repercussions of that vote.  Whether or not New Caledonia ends up being free and independent, it is clear that there are some major divergent interests between the ordinary and indigenous people on the island and those who live in and around the capital, where they are presumably subject to greater pressure to support union and receive greater benefits from union.  It should also be noted that about 40% of the electorate is made up of non-indigenous people, which does give France a significant base of support and requiring near unanimity among the local population for independence to pass.

What struck me as the most salient point about this referendum, though, was how wrong the pollsters were.  None of the pollsters were even remotely close to guessing how close the referendum would be.  Remember, the vote ended up being 56.4% in favor of maintaining ties with France and 43.6% in favor of independence, by no means a landslide vote to stay with France, and demonstrating that with a shift of some 9,000 votes from the anti-independence to the pro-independence side, the election would have gone the other way.  The pollsters, though, drastically underestimated the amount of pro-independence sentiment in New Caledonia.  In a series of polls taken between March 2017 and September 2018, pollsters like Harris Interactive and I-Scope and Quidnovi estimated margins of anti-independence sentiment between 30 and 50%, where the final margin ended up only being a bit less than thirteen percent.  While turnout was high at 80%, clearly the pollsters screwed up, and it is worthwhile to examine how it is that this could have happened.

The most recent Harris Interactive poll, for example, showed anti-independence sentiment with a roughly 2-1 lead, 66-34%, far outpacing the slight majority it actually ended up winning, with 1,000 people sampled out of a voting population of around 140,000 voters.  Having a bit under 1% of the entire electorate would appear to be a very strong sample, so how did Harris Interactive get it so wrong?  Unfortunately, the pollster did not include cross-tabs, but there are at least a few potential areas where the pollster may have screwed up.  It is possible that the pollster may have underestimated turnout, assuming that the more densely populated areas around the capital would vote in proportions far outweighing the remote districts, which was not the case.  Additionally, it is possible that ineligible metropolitan French who could not vote in the referendum were sampled, skewing the sample towards the anti-independence side by underrepresenting the voting electorate.  At any rate, with polling this abysmal, it is a wonder any of the pollsters would be able to keep their jobs or have anyone hire their firms ever again.

It should be noted that I am not just picking on Harris Interactive, because it is not as if they were even the worst example of polling on this referendum to be found.  No, that honor belongs to Quidnovi, which made three attempts to poll the referendum between April and August 2018 and found the following margins between pro and anti-independence sides for the referendum:  15-58, 15-65, and 20-69, with the rest undecided.  All of the margins were over 40% in favor of the anti-independence side, which ranged between 3 and 4 times the margin gained by that side in the actual vote.  With polling that atrocious, one would almost do better prognosticating an election by reading goat entrails or hiring a Etrustcan to engage in augury by seeing if an auspicious bird was in the air around the polling station.  Again, sample size was not a problem here; the poll used a heavily mobile phone poll, considered to be more reliable than land-lines, with a 2-1 mobile to landline ratio, and had samples in the 700’s, which amounted to .5% of the the electorate, which is a far bigger percentage than most samples of electoral races in the United States.  And yet the polls overestimated the margin of victory for the anti-independence side of 25-35%.  That level of error is simply unacceptable.

What were these people doing?  It should be noted that the margins predicted by the pollsters were not entirely out of line if one looks at the South Province alone, where anti-independence forces won a 40,000 vote majority with nearly 3 to 1 margins against the pro-independence side.  It was just that the majorities in favor of independence were as lopsided the other way in the two smaller provinces of New Caledonia, and with a higher turnout in the Loyalty Islands province, the pro-independence side may have won this election and with a higher turnout may win next time.  While I am no expert in the electoral politics of New Caledonia, I am certainly at least as much an expert as the incompetent pollsters who attempted to take the temperature of the island and failed miserably.  Their failure seems mostly due to oversampling voters close to the city and not capturing a good cross-section of the voting population that was in line with turnout and the opinions outside of the capital.  If you sample half a percent to a percent of an electorate and are this wrong, what business do you have trying to prognosticate elections at all, and what credence can we give to pollsters who do their jobs this badly?

About nathanalbright

I'm a person with diverse interests who loves to read. If you want to know something about me, just ask.
This entry was posted in Musings and tagged , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s