Pre-election polls are a national sport in Israel, but they’re not very accurate. Now there’s a new player on the field, running a new polling project that claims to to have found the system’s flaws — and fixed them. A look at how polls affect voting patterns, which populations are traditionally under-represented and the mistakes that must continue to be made if ‘Project 61’ is to succeed.
By Angela Gruber
Predicting the future can be tricky business, especially for those trying to figure out how Israelis will vote in next month’s elections. Previous elections have proven that the big polling companies are not always spot-on with their surveys. The final results of the 2013 elections, for example, were largely surprising for everyone: Yair Lapid’s Yesh Atid party ended up with 19 Knesset seats compared to the eight to 13 seats polling companies gave him leading up to the vote. That’s a pretty large margin of error.
Fast-forward two years, however, and there’s a new player in the field of polling, 28-year-old political analyst and freelance strategist Nehemia Gershuni-Aylho, who runs Project 61. (“61″ refers to the minimum number of seats needed to form a coalition in the Israeli Knesset.) Using some rather simple logic and some complicated mathematics, he thinks he found a way to make better predictions of actual election day results.
Gershuni-Aylho does not do any polling himself. He looks at the polls that are already out there, corrects the errors he is able to identify and then repackages the improved results in a way anybody can understand. To do so, Gershuni-Aylho had to learn about each polling company’s weak spots. He says he built a database of all polls from the 2006, 2009 and 2013 elections up to 30 days before election day and then compared the predictions with results at the ballot box.
“I had to find out if one of the pollsters was biased in any way,” he explains. “I was looking for a systematic error for each pollster toward a certain party and then I tried to erase this error with my formula.“
With his new formula, Gershuni-Aylho has reached a few interesting conclusions about the quality of different polls. The best polling in the last elections apparently came from a 79-year-old lady, Professor Mina Tzemach, who at the time worked for major polling company Dahaf. Internet pollster Panels Politics comes in second. The least reliable data, according to Gershuni-Aylho, came from Shvakim Panorama and Geocartography.
Perhaps most interestingly, Project 61 found that pollsters indeed produce party-specific errors.
“Every pollster has its own signature,” Gershuni-Aylho says. In other words, certain pollsters consistently predict too many (or too few) seats for a certain party. Parties like Likud or Labor tend to be more successful in polls than at the ballot box. The number of voters who cast ballots for Arab or ultra-Orthodox parties on the other hand, tends to be underestimated.
Gershuni-Aylho doesn’t attribute those errors to deliberate manipulations, it’s just that the pollsters’ complex models are not tuned to perfection.
“It’s commercial polling that is their bread and butter,” he says. “It’s not like they work on improving their election polls 12 months a year.“
The basis for each poll is a representative sample of the adult population in Israel, explains public opinion researcher and political analyst Dahlia Scheindlin. Ideally, this sample is a tiny make-up of the Israeli population and includes Arabs and Jews, men and women, young and old, according to their share in Israeli society.
“The polls regularly published in the Israeli media, based on samples of 500-600, have approximately a 4.5 percent margin of error, give or take,” explains Scheindlin.
Getting good samples isn’t as easy as it used to be, says Scheindlin. “More and more people in Israel only have mobile phone numbers, which are almost impossible to get as a pollster. This means that if you call people only on their landlines, you will miss out on certain parts of the population.”
Internet polling, according to Scheindlin, is an increasingly popular method nowadays, in part because it is cheaper. The downside is that Internet polls only reach a self-selected group of people who have signed up to be part of the pollster’s database.
But why is it so hard for statisticians to put out precise predictions, even when you asking a random sample group?
Understandably, there is a lot of uncertainty involved when questioning people about their future decisions, says Tamir Sheafer, a professor of political science and communications at Hebrew University in Jerusalem.
“The problem in Israel is that we have a high amount of undecideds, people who make a last minute decision,” he explained. Polling in a multi-party system like Israel’s makes things even more complicated. “The parties in each block (Left, Right and center) are often very similar ideologically so voters tend to shift their opinion within one block.”
To make things even more complicated, the polling results that the public sees have been processed and packaged to show how many seats a given party is expected to hold in the next Knesset.
“For this kind of mandate modeling, you have to make assumptions about many factors, for example, the actual voter turnout,” says Scheindlin. This translates to an additional source of uncertainty.
Ironically, the very existence of the polls themselves has the potential to influence their accuracy. If small parties are predicted not to pass the threshold (3.25 percent of all votes), they become unattractive choices for people who fear wasting their vote. “People will just vote for a different party that is predicted to pass the threshold. We call this strategic voting,” says Sheafer.
So would it be better to skip the whole polling altogether? No, according to Jeremy Saltan, a campaigner for Naftali Bennett’s Jewish Home party and a political analyst who tries to make sense of the pre-election polls on his blog.
“To ignore the most useful political analysis tool that is based on science in order to go with your gut does not make that much sense either,” Saltan says. “Overall, polls are the best measure we have to predict the public’s reaction to a given situation.“
Asked about Project 61, Saltan was reluctant to evaluate his colleague’s work, but praised the additional perspective the project is adding to the polling results.
Whether Gershuni-Aylho’s formula enables him to make better predictions than the veteran pollsters remains to be seen.
His model’s biggiest weakness is that he relies on the polling companies to continue making the same mistakes they have made in the past. “If they look at my data and change their system, my formulas will be lost. So I can only hope they don’t change anything up to the elections.“
Since starting Project 61 a few months ago, Gershuni-Aylho has published all of his research on a enormous Google-Spreadsheet and has been busy creating all sorts of diagrams with his findings, which he puts up on Twitter and Facebook. He has received lots of reactions to his fieldwork, he says. “The public wants to know public opinion. I want to know what others think about the parties — who will be the next prime minister, which party will vanish.“
Gershuni-Aylho humbly rejects comparisons to American statistician-wunderkind Nate Silver, who in 2012 managed to establish a prediction model that was much more accurate than the models the big pollsters had come up with. Nonetheless, he hopes his system will prove its value on election day.
“I just hope I will be as close as anyone can be. One seat up or down for each party. If that is my margin of error, I will be very happy.“
Angela Gruber, 26, is a German journalist, blogger and intern at +972 Magazine. She’s a scholar for the “trialog of cultures” scholarship of the German Quandt foundation and reports from Israel. Follow her on Twitter: @netzkolumnistin.
Read this article in Hebrew here on +972’s sister site, Local Call.
