How to Find Reliable Opinion Polls

PDF download Download Article
PDF download Download Article

Chances are, you've probably heard of polling before an election. Polls are a type of questionnaire, typically conducted with registered or likely voters before an election, to help gauge public opinion. [1] However, since politics is a divisive topic, it can be hard to find high-quality, reliable polls that are representative of public opinion. Luckily, there are ways to weed out polls that have been influenced by outside sources.

Part 1
Part 1 of 2:

Avoiding Unreliable Polls

PDF download Download Article
  1. 1
    Stay away from polls sponsored by partisan organizations. Whether it be a party you identify with or a party that you oppose, partisan polls should be met with more skepticism than their nonpartisan counterparts. Partisan polls include those done by PACs, Super PACs, and Hybrid PACs, although some polling organizations have different guidelines regarding what they consider partisan. The ulterior motive of partisan groups and media outlets is to make their side look good . This may be via avoiding covering poll results that are bad for their candidate or withholding details (such as the dates of the poll).
    • On average, "internal polls" often show results that are 4 to 5 percentage points more favorable to their sponsor than is actually the case. [2]
    • Major news outlets may also exaggerate results of their own polls for views. Consider if the "shock poll" in the headline is actually shocking, or if it's simply trying to elicit an emotional response. [3]
    • Resources like ABC's FiveThirtyEight, which covers polling for many major elections within the US, specify on their lists of polls which are partisan-affiliated. [4]
  2. 2
    Be critical of the poll's content. Analyze the specifics. Was the poll opt-in or did it randomly select its participants? Did the poll sample participants from all major parties? Is it transparent regarding how it was conducted? If any information concerning methodology, sponsorship, sample size, or poll questions seems dubious, it's best to take it with a grain of salt. [5]
    • It's important to look at the margin of error for the given poll. Keeping the margin of error in mind can help weed out sampling errors or leads that may be too close to call. [6]
    • As a general rule, a national survey should include around 1000-1500 participants. Any number significantly fewer than that will not provide a sufficiently representative sample of the population. [7]
    • Alongside random sampling, quota sampling is another representative method of sampling. Here, researchers set specific quotas for certain characteristics and then seek respondents who fit these quotas to participate in the survey. [8]
    Advertisement
  3. 3
    Analyze the poll's questionnaire. Oftentimes, low-quality pollsters will skew their questionnaires to be more favorable to a certain response. Avoid polls that seem like their questions are biased or otherwise may be influencing the respondent to give a specific answer, as they may not be reflective of the actual public opinion. Be aware of when the poll was conducted, as the results may be taken after a major event has swayed public opinion on a candidate. [9]
    • If you'd like, you can compare the questionnaire in a poll to a poll that has been relatively accurate in the past. This way, you have a baseline for what kinds of questions are better suited for opinion polls.
    • Stay away from polls that don't release their questionnaires. Not only is the pollster not being clear, but they may also be insincere about other aspects of the poll. Transparency is key.
  4. 4
    Avoid polls with extreme sample sizes. Polls with fewer than 400 participants may not be representative, [10] but bigger sample sizes aren't necessarily better. Oftentimes, huge polls are actually using unreliable polling methods, and wind up oversampling certain demographics, which introduces error. [11]
    • The "sweet spot" for polls is often about 1,000 to 1,200. It's just enough people to be representative of a population, but not enough people to be overzealous and risk sampling error. [12]
  5. 5
    Be conscious of different types of bias. Aside from sampling and questionnaire bias, there are other types, such as nonresponse bias, survivorship bias, acquiescence bias, primacy bias, and conformity bias. These are all different types of bias common in surveys, and opinion polls are not exempt from these. It is important to pay attention to the poll's methodology statement and consider these types of bias. [13]
    • Nonresponse bias is a type of bias where the response rate is abnormally low, thus skewing the results. [14]
    • Survivorship bias involves only receiving responses from people loyal to the company conducting the survey, which can cause the poll to shift in the direction of the types of people who are more likely to participate in polls. Think: if a poll is conducted by a news organization, the poll may be skewed in the direction that the news tends to be biased toward. [15]
    • Acquiescence bias is a type of bias that results from the desire to say "yes" to seem polite. People are more likely to vote in a manner that reflects their beliefs when they are voting than when responding to a poll. [16]
    • Primacy bias is a type of bias that, much like questionnaire bias, stems from the content being asked in the poll's questionnaire. However, it results in a tendency towards choosing the last answer choice due to it being fresher in the respondent's mind. [17]
    • Conformity bias is a type of bias where the respondent chooses their answers in a manner that they deem to be more "socially acceptable," affecting the outcome of the survey as a result. Regardless of confidentiality (meaning this type of bias can also affect elections), people may feel pressured to misrepresent their opinions out of fear of social repercussions.
  6. 6
    Be wary of opt-in polls. Polls can only represent the opinions of a population when everyone in that population has an equal chance to take it. Opt-in polls usually attract people who feel passionately about the subject of the poll, rather than a representative sample. Extra information (such as gender and age) that would allow a judgment to be made about the nature of the sample is rarely collected by such polls, and so they should never be interpreted as a representative survey . [18]
  7. 7
    Vet exit polls carefully. While they are an excellent source when properly conducted, being the only chance to survey only voters, they can be skewed by three main sources of error: [19]
    • Differential non-response, where voters for a particular party or candidate are more likely to disclose their support than others.
    • If the sample of polling stations is not representative. Even with a large sample, the overall results could be skewed if the political balance of those stations is not representative of the entire electorate.
    • People accidentally placing the wrong vote, spoiling their ballot paper, or not filling out the paper correctly. Although it sounds unlikely, it is thought to be responsible for historically skewed exit polls.
    8. Advertisement
Part 2
Part 2 of 2:

Finding High-quality Polls

PDF download Download Article
  1. 1
    Be mindful of what kind of polls you're reading. Polls are sorted into 3 general categories— General Population (GP), Registered Voters (RV), and Likely Voters (LV); from most to least broad. Excluding non-voters from polls prior to an election tends to draw more accurate conclusions, as those excluded will have no impact on the final election result. However, GP polls are more common during non-election years. [20]
    • LV polls are generally considered the most reliable. This is because they exclude voters who are registered, but may be conflicted or otherwise against voting in the upcoming election cycle.
  2. 2
    Check pollster ratings. Sites like ABC's FiveThirtyEight [21] and Nate Silver's Silver Bulletin [22] have ratings of different pollsters based on their performance in past election cycles. If a certain pollster has a higher rating, it is more likely to be of a higher quality.
    • Pay attention to the last time pollster ratings were updated, as their reliability can shift over time depending on what is happening in the race.
  3. 3
    Take note of trends in data over time. If a poll has done multiple surveys for a specific election, taking note of the given trend line can help see who is ahead in the race. [23]
    • For example, if a candidate is polling ahead but by a less significant margin than seen in previous polls, that may have more to say about the outcome of the election.
    • If the trend line doesn't change at all over time for one poll compared to others, it may have more to say about the pollster's methodology than the actual outcomes of the election.
  4. 4
    Look for polls sponsored by major news organizations. Some of the most reliable pollsters in the US are those sponsored by major news networks, like CNN, Fox, NBC, and CBS. They run all kinds of polls, including favorability, election results, and key issues surrounding elections. News organizations also frequently partner with research organizations to help produce better-quality polls; [24] for example, Reuters/Ipsos and YouGov/The Economist.
    • It is also important to check the reliability of both the news organization's polls and the research organization's polls to ensure that one of the pollsters isn't forking the overall results in a certain direction.
  5. 5
    Pay attention to how weighting works in polls. Weighting is the act of adjusting the results to give less weight to certain respondents and more to others. While this may sound like an act of censorship, it's quite the opposite! This process corrects for demographic imbalances and makes sure the sample more accurately reflects the broader population. For example, non-college-educated voters are known to be harder to reach for polling, so their voices are given a boost in polls to put them on-par with the rest of the respondents. [25] For example, if 48% of Americans have some sort of college degree, [26] the remaining 52% should be non-college-educated. If only 26% of the respondents are non-college-educated, their responses need to hold twice the weight of a college-educated person's.
    • Making sure responses are actually weighed correctly may take a bit of math. It helps to have a calculator on hand if you really want to be precise.
    6. Advertisement

Expert Q&A

Ask a Question
200 characters left
Include your email address to get a message when this question is answered.
Submit
      Advertisement

      Tips

      *Don't just look at one poll: this provides an incomplete picture. Polls will yield different results depending on methodology, sample size, and timing, so consult multiple sources before forming concrete opinions on political matters.

      • There is some evidence that anonymous online surveys secure greater honesty regarding sensitive topics than face-to-face or telephone surveys. [27] [28]
        Thanks
        Helpful 0 Not Helpful 0
      • If any wording in a poll questionnaire seems ambiguous, this is likely an intentional move to bias responses.
        Thanks
        Helpful 0 Not Helpful 0
      Submit a Tip
      All tip submissions are carefully reviewed before being published
      Name
      Please provide your name and last initial
      Submit
      Thanks for submitting a tip for review!
      Advertisement

      Warnings

      • Be aware of push polls. This is where, under the guise of conducting an opinion poll, individuals or organizations phone around looking for responses to a purported survey. The "questions" are actually attempts to influence prospective voters' views via accusations or rumors. The conductors of these surveys have no intent of collecting responses or any connection with genuine opinion polls. [29] [30]
        Thanks
        Helpful 0 Not Helpful 0
      • Research carried out by the Pew Research Center suggested that opt-in surveys were more susceptible to "bogus respondents" — responses from people ineligible to take part, or who aren't providing sincere answers (e.g., answering "no" to every question). [31]
        Thanks
        Helpful 0 Not Helpful 0
      • The order of questions can also bias responses. For example, if a poll focuses on healthcare and then asks about a candidate's favorability, respondents might be more likely to evaluate the candidate based on their stance on healthcare, rather than who they support overall. [32]
        Thanks
        Helpful 0 Not Helpful 0
      Advertisement

      You Might Also Like

      How to
      Be a Responsible Digital Citizen
      How to
      Recognize Bias in a Newspaper Article
      How to
      Evaluate the Credibility of a Source
      How to
      Analyze Newspaper Language
      Advertisement

      References

      1. ↑ https://www.pewresearch.org/course/public-opinion-polling-basics/#how-does-polling-work
      2. ↑ https://fivethirtyeight.com/features/how-to-read-2020-polls-like-a-pro/
      3. ↑ https://fivethirtyeight.com/features/how-to-read-2020-polls-like-a-pro/
      4. ↑ https://abcnews.go.com/538/538s-polls-policy-faqs/story?id=104489193
      5. ↑ https://www.jsonline.com/story/news/solutions/2021/11/17/wisconsin-politics-five-red-flags-watch-opinion-polls/8632846002/
      6. ↑ https://www.pewresearch.org/short-reads/2016/09/08/understanding-the-margin-of-error-in-election-polls/
      7. ↑ https://www.jsonline.com/story/news/solutions/2021/11/17/wisconsin-politics-five-red-flags-watch-opinion-polls/8632846002/
      8. ↑ https://publications.parliament.uk/pa/ld201719/ldselect/ldppdm/106/10616.htm
      9. ↑ https://www.pewresearch.org/course/public-opinion-polling-basics/#how-does-polling-work
      1. ↑ https://www.nytimes.com/2016/10/13/upshot/the-savvy-persons-guide-to-reading-the-latest-polls.html
      2. ↑ https://www.pewresearch.org/short-reads/2020/08/05/key-things-to-know-about-election-polling-in-the-united-states/
      3. ↑ https://www.markpack.org.uk/168548/why-is-a-1000-sample-enough-for-an-opinion-poll/
      4. ↑ https://delighted.com/blog/avoid-7-types-sampling-response-survey-bias
      5. ↑ https://delighted.com/blog/avoid-7-types-sampling-response-survey-bias
      6. ↑ https://delighted.com/blog/avoid-7-types-sampling-response-survey-bias
      7. ↑ https://delighted.com/blog/avoid-7-types-sampling-response-survey-bias
      8. ↑ https://delighted.com/blog/avoid-7-types-sampling-response-survey-bias
      9. ↑ https://publications.parliament.uk/pa/ld201719/ldselect/ldppdm/106/10616.htm
      10. ↑ https://publications.parliament.uk/pa/ld201719/ldselect/ldppdm/106/10616.htm
      11. ↑ https://www.pewresearch.org/short-reads/2017/02/16/does-poll-include-or-exclude-nonvoters/
      12. ↑ https://projects.fivethirtyeight.com/pollster-ratings/
      13. ↑ https://www.natesilver.net/p/pollster-ratings-silver-bulletin
      14. ↑ https://www.cnn.com/2023/07/15/politics/polls-elections-what-matters/index.html
      15. ↑ https://www.pewresearch.org/course/public-opinion-polling-basics/#what-should-you-look-for-in-a-poll
      16. ↑ https://www.pewresearch.org/methods/2018/01/26/how-different-weighting-methods-work/
      17. ↑ https://www.collegetransitions.com/blog/percentage-of-americans-with-college-degrees/
      18. ↑ https://delighted.com/blog/avoid-7-types-sampling-response-survey-bias
      19. ↑ https://publications.parliament.uk/pa/ld201719/ldselect/ldppdm/106/10616.htm
      20. ↑ https://www.maine.gov/ethics/political-activity/node/309
      21. ↑ https://publications.parliament.uk/pa/ld201719/ldselect/ldppdm/106/10616.htm
      22. ↑ https://www.pewresearch.org/course/public-opinion-polling-basics/#what-should-you-look-for-in-a-poll
      23. ↑ https://fivethirtyeight.com/features/how-to-read-2020-polls-like-a-pro/

      About This Article

      Thanks to all authors for creating a page that has been read 446 times.

      Did this article help you?

      Advertisement