For a poll to be predictive, the pollster needs to understand who will show up on election day. Polls that try to predict turnout on Election Day, describe their sample as likely voters.
Polls that sample registered or eligible voters are not intentionally predictive. Each Election Year, a significant percentage of registered voters do not show up. Likewise, many eligible voters do not register to vote.
Most polls reported in the news are of registered or eligible voters. Republicans almost always overperform against these two poll types. Historically, Republicans record a higher turnout rate going back decades. Polls of eligible and registered voters are not intentionally predictive.
Polls of likely voters are intended to be predictive. These polls attempt to identify the voters most likely to vote though statistical measures. These types of polls are more favorable to Republicans because they factor in voter turnout expectations and this estimate is reflected in the poll results.
This Article Explains
- How undersampling significantly affects poll results with the latest FOX NEWS poll.
- How to unskew polls and get an estimate of actual poll results.
This Article Finds
- After unskewing the sample size of the FOX NEWS poll of registered voters, Trump and Biden are statistically tied.
- Because this is a poll of registered voters, it is not predictive of what will happen on Election Day. We should expect Trump to overperform this poll because Republicans generally have higher turnout. We have reason to predict a Trump win based on this poll.
PART 1 FOX NEWS METHODOLOGY
Their methodology appears promising. However, the fact remains FOX NEWS consistently undersamples independent voters, driving materially false conclusions. In order to assess the gravity of the situation, we compared FOX NEWS’ data to larger and reliable data sets for comparative purposes.
PART 2 THE DATA
Voter Registration Data
First, I attempted to gather voter registration data from all states. But, some states restrict registration data to political candidates only. In the states where I obtained registration data, the ratio of Republican, Democrat and independent voters look identical to Gallup’s numbers.
I was only able to get registration data from 28 states. From those states, here are the numbers:
|28 State Sample|
Gallup has been tracking political affiliation since 2004. For July 2020, here are their results: 28% Republicans, 38% independents, 29% Democrats.
I also reviewed Pew.
Pew has been tracking voter affiliation since 1994. Their latest report shows: 29% Republicans, 34% independents, 33% Democrats.
PART 4 THE ANALYSIS
FOX NEWS’ Sample Compared
Since FOX NEWS identifies their sample as registered voters, and their methodology appears compatible with sound data management, we should expect a poll sample that corresponds to the nationwide ratio of Democrats, Republicans and independents.
It does not. FOX NEWS’ undersampled independents by 70% in their latest poll. It’s possible this particular poll was an outlier. Sadly, this is not the case. In fact, they have been undersampling independent voters for months. Such consistent and persistent anomalies suggests intentional manipulation.
Data from FOX NEWS’ poll of registered voters, and the estimated over/undersample.
|Fox Poll August/2020||43%||46%||11%|
|Avg. of Pew, Gallup, registration||28.3%||30.3%||37.6%|
The data suggests FOX NEWS oversamples registered Republicans by 51.9% and Democrats by 51.8%. FOX NEWS undersampled independent voters by 70.74%. Remember, they are claiming their poll is based on registered voters. The disproportionate set of registered voters’ political ID to our larger data sets suggests there was self-selection of the sample. In other words, they crafted their data sample to produce a desired outcome.
PART 5 UNSKEWING THE DATA
Undersampling independent voters hurts Trump’s numbers significantly. Why? Because FOX NEWS’ own poll numbers show that Trump performs better with independent voters than Biden. When FOX NEWS cuts people from this segment, they also remove more Trump supporters than Biden supporters.
PLUG THE DATA HOLES
Let’s adjust their result by extrapolating the results based on a ratio of Republican, Democrat and independent voter ratios that match the latest Pew/Gallup numbers.
We will use FOX NEWS’ poll data to deduct how each of these groups would vote. This will unskew the data, and give independent voters their fair share in contributing to the final poll results.
PLUG THE DATA HOLES
Extrapolating the data based on the current ratio of Republicans, Democrats and independents renders the polls a statistical dead heat, assuming a similar margin of error of ±3%.
Scope: It’s important to obtain comparative that corresponds to the scope of your analysis. For example: If you want to deskew the popular vote, use national data. To unscrew data for the Electoral College, you want to focus on state data like state exit polls.
- Compare multiple points of reputable data. State registrations, exit poll data, Gallup and Pew all show similar ratios of nationwide Republicans, Democrats and independents. We took the average of these and calculated a national ratio of Republicans, Democrats, independents and other of 28.4% to 30.3% to 37.6% to 3.8%, respectively.
- Extrapolate the number people in each Party ID that is consistent with the established data in step #1. We reviewed FOX NEWS’ poll data and for each one of these Party ID segments, we calculated how many people would belong in each one of these sub groups if they followed the national ratio.
- Extrapolate the number of people in each Party ID that intends to vote for a certain candidate. We looking at the raw FOX NEWS poll data and the voter candidate preference ratios in each Party ID subgroup. We apply that ratio to the extrapolated number of people in each Party ID segment to estimate how many people are voting for each candidate.
For example, according to FOX NEWS, the Republican pool prefered Trump 86% of the time and Biden 8%. (Note: for simplicity, we will not attempt to deskew this ratio. However, this ratio is suspect for a couple of reasons). This gives Trump 249 people from the registered republican subset.
- The full extrapolated data is in the table below. This adjustment is net positive for Trump. Whereas before he was down by 7.5 points vs. Biden. The extrapolated data puts Trump in a statistical dead heat.
|Pool 28.3%||Pool 30.3%||Pool 37.6%||Pool 3.8%|
Make no mistake: This is purposeful undersampling.
PART 5 BOTTOM LINE
This is an important lesson: It’s important to be precise when analyzing poll numbers if you want to assess the predictive qualities of a poll. Pollsters can materially change the outcome of the poll by just skewing independent voter ratios. The data suggests this is how FOX NEWS is manipulating polls.
Pollsters are being driven by outside agendas. Nearly all polls are suspect.
- Know the difference between polls of eligible voters, registered voters and likely voters. Each poll type has results changing assumptions.
- Know how to assess if a poll over or undersamples. As a baseline you can consider the previous Presidential Election to quickly assess if the poll oversamples or undersamples by comparing the ratio of participants by party ID.
- Know that over or undersampling of of voters based on Party ID can dramatically affect the results of the poll.