Nate Silver:
It’s time to stop worrying about outliers and start worrying about inliers. Earlier this year, my colleague Harry Enten documented evidence of pollster “herding” — the tendency of polling firms to produce results that closely match one another, especially toward the end of a campaign.1 What’s wrong with the polls agreeing with one another? The problem is that it’s sometimes a case of the blind leading the blind...
A few pollsters are shameless about their herding. One of them is Public Policy Polling (PPP), an polling firm that conducts automated polls for both public consumption and for liberal and Democratic clients.
Take a look at this exchange, for example, between The New York Times’ Nate Cohn13 and PPP’s Tom Jensen. Cohn discovered that in 2012, the racial composition of PPP’s polls was correlated in an unusual way with President Obama’s performance among white voters in their surveys. If Obama was performing especially poorly among whites in one PPP poll, it tended to have a higher share of nonwhite voters, which boosted Obama’s result. And if Obama was doing relatively well among whites, PPP projected less nonwhite turnout, keeping his lead in check. As a result, PPP’s polls tended to show an unusually steady race between Obama and Mitt Romney.
I’m picking on PPP for a reason: They’re the biggest herders in the business