Effective campaign research rarely comes from a single source of data. At Corona Insights, we frequently design studies that pair quantitative surveys with qualitative methods, including focus groups and in-depth interviews (IDIs), to give clients the fullest possible picture of how their audiences think, feel, and respond to messages and materials. The sequencing of those methods matters and varies depending on the research questions at hand. Sometimes a survey goes first to map the landscape of attitudes and segment audiences, followed by qualitative work that digs into the “why” behind the numbers or go deeper with certain populations of interest. Other times, exploratory focus groups or IDIs come first to surface the language, frames, and concerns that should inform a subsequent survey or give the client a chance to incorporate some of the qualitative feedback into the materials to be tested in the survey. Any way you sequence data collection, the research design should follow the questions, not the other way around.

A mixed methods approach works beautifully when the two methods point in the same direction, which they often do. But one of the most instructive, and sometimes disorienting, situations in mixed methods research is when they don’t. When a survey tells you that most respondents responded favorably to a particular message, and then focus groups suggest people found that same message off-putting or unconvincing, what do you do with that? Rather than treating the discrepancy as a flaw in the research, we treat it as one of the most valuable findings the study can produce, because understanding why the methods diverge often tells you something essential about how the campaign might need to be designed and deployed.

One of the most important explanations for discrepancy is social desirability bias: the well-documented tendency for people to give answers they think are expected, appropriate, or acceptable, rather than what they actually think or feel. In a self-administered, anonymous online survey, respondents often feel more comfortable expressing skepticism, indifference, or even opposition to a message. They might also feel more supportive of messages and ideas that in social settings could be deemed controversial. Focus groups and IDIs, by contrast, involve a human interaction, whether it is in a group setting with other participants and a moderator or a one-on-one conversation with a researcher. That interaction can shift what people say, even when it is not their intention to censor themselves or be less than candid. However, as I will discuss below, these social dynamics might be an inevitable part of what we want to study. They may have effects not only in qualitive research, but also in real world interactions. In a group setting especially, people may echo the enthusiasm or criticism of early speakers, soften a critical reaction in the presence of strangers, or present themselves as more open-minded than their private views would suggest.

That said, it is too simple to conclude that survey responses are always more candid and qualitative participation is always more filtered. The more useful question is: what matters most for this campaign? How does the client want people to interact with the campaign? And what is the ultimate behavior, if any, that they are targeting? If a client is running digital ads that will reach people individually, sitting alone at their computers or scrolling on their phones, then anonymous survey reactions may actually be more effective at predicting reactions to the actual campaign. And what about when it comes to the targeted behavior? Take a ballot measure campaign, for example. While one’s political views are no doubt socially constructed, the behavior targeted is strictly individual: voters ultimately make their decision in private, in a voting booth. If a message tests well in a survey but falls flat in group discussion, that may not be a problem at all. The campaign may not need the message to hold up in social settings; it just needs to move someone who encounters it alone.

This strategy may shift considerably when the campaign is trying to change something that is fundamentally social in nature. Mental health stigma is a strong example. Stigma is not just a private attitude; it is enacted through conversation, social comparison, and the norms people perceive within their communities. A campaign trying to reduce stigma may need to understand not just what people believe when no one is watching, but how they grapple with those beliefs when other people are present. Do they soften their language? Do they change their position when someone else in the group shares a personal story? Do they resist the message more when they feel socially observed? In those cases, the messiness of the qualitative setting is not a confound, it’s just a reality of human research. Beliefs and attitudes and how they are expressed in conversational settings are messy.

The social interaction consideration is also relevant when clients want to understand whether and how people might share or talk about a campaign with others. Would someone send this ad to a friend? Bring it up with a family member? Repeat the key message in their own words to someone they care about? These are often difficult questions for participants to answer; however, they are usually better explored in a setting where people are already in conversation, responding to each other in real time, than in an anonymous survey where each respondent is an island.

When survey and focus group findings diverge, we typically present both sets of findings transparently to clients, with an explanation of what might be driving the difference and what each data source is most useful for predicting. This means resisting the urge to smooth over the tension or to simply talk about findings in a high level and vague enough way to reach a tidy conclusion. A client who knows that their message scores well in anonymous testing but generates more resistance in conversation is a client who can make a more informed strategic decision about how and where to deploy that message. They may choose to test it more heavily in digital placements while continuing to refine the language for community outreach or group-facing contexts. Critically, good research not only assesses what performs well, but also understands why. It might be that one creative concept scores well on capturing attention on a survey, but also yields significant backlash when people deliberate on what they like and dislike about the same message in a focus group. Effective research designs capture the multidimensional performance of these efforts to better understand these tradeoffs and make sense of seemingly contradictory feedback.

Mixed methods research is most powerful not when everything aligns neatly, but when the combination of methods gives clients something no single methodology could provide: a view of their audience as full, complicated human beings who respond differently depending on context. The discrepancies are not defects in the research design. Handled well, they are some of the most valuable insights the study produces.