University of Mississippi political science professor offers history, guidelines for voters’ consideration
by Edwin Smith
During election years, it seems that new political polls come out every week, often with conflicting results. But voters can make better sense of exactly what different polls mean by paying attention to how they were conducted and what they measure, a University of Mississippi professor advises.
“Remember, polls serve many purposes that are not related to predicting elections,” said Miles T. Armaly, associate professor of political science. “While polls can offer a snapshot of current sentiment, they’re also useful for campaign strategy, for spurring public conversation, and can help those in politics make decisions.”
Armaly said that it is critical to note that not all polls are created equal.
“Some pollsters use techniques that make it more likely to achieve population-sample congruence, or random sampling,” he said.
These techniques include stratified sampling—where researchers divide a population into smaller groups based on specific characteristics, and every member of the population studied is in exactly one category or quota sampling, where the samples are not random. Users of each method try to make sure certain characteristics are represented in the sample according to their proportion in the population, Armaly said.
Another common technique is to use weighting, where pollsters adjust results to correct imbalances. This is what many pollsters do these days, including the poll aggregators.
“Often, weights are assigned based on demographic information in a way that allows the sample to represent the population more accurately,” Armaly said. “While it is difficult to verify that a single poll is reliable, it is possible to verify that the source of polls is accurate.
“That is, once-off polls might have strange quirks due to the timing of the poll, an uncommon or difficult to understand question wording, or something similar. But this would not indicate a systematic lack of reliability.”
Armaly encourages voters to assess reliability, including the pollster’s reputation, their transparency, and sampling methodology.
“Transparency in their methodology indicates reliability,” he said. “We should be skeptical of polls unwilling to tell us how they came to their conclusions.”
Armaly said name recognition and past-performance matters.
“Polls like Pew and Gallup have been around for generations,” he said. “Often, polls associated with universities are of quality, but this is not a universal rule. News organizations typically do well.”
The no-name pollsters might deserve some additional scrutiny, Armaly said.
“If a pollster is consistently off, that should be taken into account,” he said. “This is compared to polls conducted by campaigns themselves, PACs or advocacy groups who have a stake in the election.”
Armaly said that the average person can find this information at the pollster’s website.
“If the information cannot be found or if you have to pay for it, those are perfectly valid reasons to scrutinize the results,” he said.
Many of these indicators of reliability to require some familiarity with statistics and probability. When interpreting poll results, voters should consider all the factors affecting a particular race, he said. In presidential elections, for example, the Electoral College is more important than the raw popular vote.
“Polls could nail the vote margins down to the percent and still be wrong about who wins the presidency,” Armaly said. “This does not impact House or Senate races. Thus, for a full picture of interpreting the results of any poll, one must consider things beyond the polling itself, like politics.”
Other important considerations include the demographics of those being polled. Some polls sample likely voters, but others sample all who are eligible to vote.
“The results from such polls should not be treated as apples-to-apples comparisons,” he said.
When looking at pre-election polling, there is cause for both optimism and caution, Armaly said.
“The 2020 polls were the least accurate in decades, but the 2022 midterm polls did very well, likely because of updated methodologies and weighting,” he said. “For instance, some polls are now using past voting behavior in addition to demographics.
“Polling firms are always updating their sampling and weighting strategies to accurately reflect the population, and this is no different after major polling errors.”
However, if groups that strongly support one presidential candidate or party are systematically opting out of polls, then those surveys will continue to underestimate support for that candidate and party.
“It is also important to remember that polls tend to become more accurate closer to the election, so the reliability should increase through November,” Armaly said.
