How to Ensure Your Political Research is both Reliable & Valid

View all blog posts under Articles | View all blog posts under Master's in Political Management Online

Campaign season isn’t the only period that sees more than its fair share of polls, whether conducted online, over the phone or in person. Survey research goes on throughout the year and often helps political campaign managers and candidates get a snapshot of how the public feels about a particular subject matter, issue or news item. Elected officials will often use these insights to gain a better understanding of their constituents’ concerns.

Interactive poll conducted on a computer tablet

The takeaway value of these polls, however, hinges on their accuracy — meaning truly reflective of the groups of people the surveys are designed to represent. For this to happen, they must be both reliable and valid.

This is something that you can learn more about through the Master’s in Political Management online program at the Graduate School of Political Management at the George Washington University. The GSPM curriculum is composed of four clusters: Core Classes, Electoral Politics, Advocacy Politics and Applied Proficiencies. Audience Research, part of Applied Proficiencies, is the main one that addresses survey research. The insights you glean can help you design and draw samples, objectively construct questionnaires and accurately interpret data so they can be used to better inform your overall campaign strategy.

Reliability and validity are two polling touchstones for surveys to be considered truly informative and trustworthy. Audience Research delves into the distinctions between the two more comprehensively as part of the overall GSPM curriculum on the road to obtaining a political management degree.

How do reliability and validity differ?

On the surface, the terms “reliable” and “valid” may seem like distinctions without much of a difference. While similar, they’re not the same. Reliability refers to the degree to which something that is measured ― such as a group of likely voters, for instance ― can be replicated. So in the context of polling, a reliable survey would be one that if it were to be done on multiple occasions would produce virtually the same results as the original.

Validity, on the other hand, is a bit more nuanced and often requires additional legwork to attain. If a poll is deemed valid, it’s an indication that the conclusions drawn from the survey are accurate, or at the very least, as close to accurate as possible.

Can a survey be valid but unreliable?

For surveys to be considered valuable, data gathered must be both valid and reliable. If a poll is deemed valid, then by definition it is also reliable; what’s being measured is accurate and reflective of a certain group or population. But the same cannot be said of reliability. A statistician or pollster may arrive at the same outcome in the second go-round, an indication of reliability, but if the sample is inaccurate, it doesn’t much matter if a follow-up poll produces matching results if what’s being evaluated is deemed invalid. This further explains why reliability and validity in research may not be mutually exclusive terms.

No survey is perfect, even those that are scientific. Organizations like the Pew Research Center, Gallup and Public Policy Polling specialize in advanced survey research so they can be used to glean insight about consumers in a given region, group, country or party affiliation.

But there are a host of factors that can influence a survey’s validity, noted polling expert Matthew Dickinson.

“One of the things that I think a lot of listeners underestimate is that, even in a perfectly well-done poll, you’re going to have tremendous statistical variations from poll to poll,” Dickinson told WBUR, a Boston-based public radio station. “And that’s just simply a fact associated with [the fact that] we’re taking a sample from a population. So if you dip back into that population many different times, you’re going to come up with slightly different measures.”

A classic example in the electoral politics realm is polling “likely voters” versus “registered voters,” Dickinson added. Although likely voters can provide insight about how citizens intend to cast their ballots, there isn’t a universal standard for “likely voter,” as one pollster’s definition often differs from the next. Some may ascribe “likely” to those who voted in the previous election or who simply state that they intend to perform their civic duty come November. These factors can influence both the reliability and validity of a survey’s conclusions.

Some of the more memorable presidential elections over the years have been those where the surveys said one thing, but the outcomes proved otherwise. Campaigns often make decisions about where they should devote their resources to seek more votes based on what the polls suggest.

Surveys that are valid and reliable are traditionally more accurate ways to get a read on the public, but each has its inherent flaws, influenced by the people being questioned, the person conducting the poll or the wording of the questions posed.  This is why pollsters and political campaign experts say it’s best to look at several different surveys before making any assumptions about how individuals may feel about a particular subject matter or who they plan on supporting during the next local, state or federal election cycle.

What are the expected student outcomes of Audience Research?

How can you assure data reliability and data validity? The Audience Research course through the GSPM curriculum will show you how it’s done. Electoral politics is an inexact science, but this three-credit course can provide you with the insight and intelligence you need to formulate polls so they can be used to gauge public opinion or help you develop a campaign strategy. In addition to the various aspects surrounding validity and reliability, this course will also touch on qualitative and quantitative data and how they intersect with the hallmarks of polling accuracy and ability to think critically. The learning outcomes of Audience Research complement other courses in the Applied Proficiencies cluster, such as Maximizing Social Media.

For more information on the online political master’s program at GSPM, follow our website for a brief description regarding what you can expect in each course.

Recommended reading:
Benefits of attending an accredited graduate program
How local campaigns differ from national campaigns

Sources:
WBUR.org
KeyDifferences.com
Primer on the Validity of Assessment Instruments