The Flaws with Virginia Public Polls

The hardest thing to do in any election that isn’t presidential is determining who is going to vote and who isn’t. As a pollster, what drives me crazy when a poll I do is off from the final election results comes a few months later when I go back and see who in the poll told me they were “definitely” going to vote. I usually find that the definites were no more likely to have actually voted that those who said they were “probably” going to vote. I have tightened my calling and sampling so that I am calling mostly people who have voted in previous election years since that is the best way to determine future voting. This is especially important in odd year elections and primaries.

Virginia is a strange state. We have elections every year and don’t have gubernatorial elections when we elect the President or Senator. Some years my activist wife and I are happy and other years we are not. 2008 was the cap on a strong 4 year run and then 2009 happened. 2012 was great but who knows from 2013 when half the number of people are going to vote than did in 2012. Digging into the Washington Post and Marist/NBC polls for the Governor’s race, we see that Republican Ken Cuccinelli does better with “almost certain to vote” people and “likely voters”. That is usually the case as Republicans do better the smaller the electorate. The problem with these polls is that their pool of likely or certain voters is too large and we don’t know from those interviews, who really will vote and who won’t.

Both companies do random digit dialing samples. That is considered the best way to poll but has drawbacks. 1) People don’t know whether they will vote or not, and 2) there is no way to verify against a voter file. The advantage is that there is a larger pool of potential respondents if you call unlisted numbers and cell phones that are not associated with voters in various databases. The problem with both these polls is that they overestimate the likely voter universe:

Marist/NBC
Total % of Adults % of Reg Voters
1218 100% Adults
1086 89% 100% Registered Voters
692 57% 64% Likely Voters
Washington Post
Total % of Adults % of Reg Voters
1000 100% Adults
887 89% 100% Registered Voters
663 66% 75% Almost Certain

These polls claim to represent of the population of Virginia. From these polls, 89% of adults say they are registered voters, and among registered voters 64% in the Marist poll and 75% in the Washington Post poll were determined to be likely or certain to vote . This both overshoots the percentage of the population of Virginia who are registered and certainly overshoots the percentage who will vote based on recent history:

2009 Virginia Voters
Total % of Adults % of Reg Voters
6,080,805 100% Adults
4,955,755 81% 100% Registered Voters
2,000,819 33% 40% Voted
2005 Virginia Voters
Total % of Adults % of Reg Voters
5,760,835 100% Adults
4,448,852 77% 100% Registered Voters
2,000,052 35% 45% Voted

Whereas roughly 90% of adults in the Post and Marist poll say they are registered, the actual share, based on Census numbers is around 80%. Among registered voters, 40% voted in 2009 (when Republican Bob McDonnell won) and 45% voted in 2005 (when Democrat Tim Kaine won). So, reporting registered voter numbers in this race provides useless information. The vote among likely or certain voters is not much better. If the public polls showed the vote by 2008/2012 only voters and by 2005/2009 voters, we would have a much better idea of who is actually ahead among people who will vote and who needs to do the most work turning out unlikely voters. Unfortunately, the only people who will see this before November work in the campaigns since public polls won’t sample from a voter file.

Leave a comment