Polling is tough stuff; look at the primary results for proof
According to polling, Abdul El-Sayed was supposed to win the Democratic U.S. Senate primary by 14 points. Or 15. Or 16.
He won by one.
Which brings me to a poll MIRS News decided not to publish two months ago — and the national story that decision generated.
To take a step back, Mitchell Research and Communications and MIRS News have had a long relationship dating back several years. Sometimes we pay them to poll certain races. Sometimes, the owner, Steve Mitchell, just gives us polls we can run.
The poll conducted June 11-13 fell into the latter category. Steve polled the Republican primary, the Democratic primary and then made a general election poll using data from both.
The Democratic primary showed El-Sayed 42%, Haley Stevens 33% and Mallory McMorrow 6%, marking the first poll showing McMorrow in single digits. She had not dropped out that this point.
I shared this information with all three campaigns for response. The McMorrow campaign came back with a question about how the poll was conducted: Was this an open-link poll?”
Maybe you’ve seen these text messages asking you to fill out a poll online. Respondents click a link and fill out the survey online using “survey monkey” or a similar service.
Not being completely informed on the industry, I thought all of these links were private links. You respond to the link. You fill out the survey. The link goes away. I was wrong.
Mitchell, Emerson College and others use “open links,” where the web address is live for whoever wants to take a survey during a certain window of time.
It’s apparently cheaper to do it this way. I’m told it costs a lot of money to do to run a “closed-link” poll. For the open link polls, however, safeguards are put in place to protect people from stuffing a survey. Polls that are completed very quickly or surveys using the same IP addresses are tossed out.
But, theoretically, if someone was savvy enough to share a link with a group of friends to influence the results of a poll, it could be done. I tested it myself.
I didn’t feel comfortable with the results of this particular June 11-13 Mitchell poll.
How could McMorrow – who had polled at 17% only a month prior – suddenly be in single digits?
And then the cross tabs on the 400-voter sample were strange. El-Sayed polling at 86% among voters aged 18-44?
I told Mitchell and the other campaigns I didn’t feel comfortable with the June 11-13 results. It’s possible someone played games with the results, and I didn’t want our name attached to it.
The next day, a POLITICO reporter called. He obtained the poll results. His story was that MIRS wasn’t running the results because we buckled under pressure from the McMorrow campaign. Then they published the results.
The insinuation was, “McMorrow doesn’t like this poll, therefore we’re killing it.” That wasn’t what happened. The campaign raised a methodological question. I investigated it, looked harder at the numbers and found results I couldn’t comfortably defend to readers.
Fast-forward to Aug. 4. Primary election day. Going in, Emerson had El-Sayed winning by 15 points. A July 28-29 Mitchell poll (which we also didn’t run) had El-Sayed winning by 14 points. Tulchin Research, which does polling for El-Sayed interests, had him winning by 16.
El-Sayed defeated Stevens, 48.5% to 47.5%.
Polling has always been a tough business. Predicting the exact universe of people who will vote in each election is tough stuff.
Now, pollsters are trying to figure out a way to connect with these voters – without manipulation — at a price an increasingly cash-strapped media can afford.
In this case with Abdul El-Sayed, maybe the sampling was way off. Maybe not enough little ol’ ladies who watch Wheel of Fortune and Jeopardy! were called. Maybe some other methodological quirk skewed the result. Who knows?
What is clear is that polling is harder. Cheaper options carry risks. Turnout models can fail if a certain demographic isn’t responding.
Polls are useful. They aren’t gospel.
Journalists shouldn’t treat them that way.
