Category: Qualitative Research

Who’s Excited for 2016?

Oh man, where did 2015 even go? Sometimes the end of the year makes me anxious because I start thinking about all the things that need to be done between now and December 31st. And then I start thinking about things that I need to do in the upcoming year, like figuring out how to be smarter than robots so that they don’t steal my job and learning a programming language since I’ll probably need it to talk to the robots that I work with in the future. Ugh.2015 Calendar

Feeling anxious and feeling excited share many of the same physical features (e.g., sweaty palms, racing heart, etc.),  and research has shown that it is possible to shift feelings of anxiety to feelings of excitement even by doing something as simple as telling yourself you are excited. So, let me put these clammy hands to use and share some of the things that I am excited about for 2016:

  • Technological advancements for data collection. Changes in phone survey sampling are improving the cell phone component of a survey. Also, we have been looking at so many new, cool ways of collecting data, especially qualitative data. Cell phones, which are super annoying for phone surveys, are simultaneously super exciting for qualitative research. I’m excited to try some of these new techniques in 2016.
  • Improvements in the technology that allows us to more easily connect with both clients and people who work remotely. We use this more and more in our office. I’m not sure if in 2016 we will finally have robots with iPads for heads that allow people to Skype their faces into the office, but I can dream.
  • Work trips! I realize that work trips might be the stuff of nightmares at other jobs. But Coronerds understand the importance of finding humor, delicious food, and sometimes a cocktail during a work trip.
  • New research for clients old and new. This year I’ve learned all sorts of interesting facts about deck contractors, the future of museums, teenage relationships, people’s health behaviors, motorcyclists, business patterns in certain states, how arts can transform a city, and many more! I can’t wait to see what projects we work on next year.
  • Retreat. For people who really love data and planning, there is nothing as soothing as getting together as a firm to pore over a year’s worth of data about our own company and draw insights and plans from it.

Alright, I feel a lot better about 2016. Now I’m off to remind myself that these clammy hands also mean that I’m very excited about holiday travel, last minute shopping, and holiday political discussions with the extended family…

Magic 8 Ball Says…Outlook Good!

It’s hard to believe that it’s already that time of year where you’re thinking not only about turkeys, in-laws and shopping, but also reflecting on what you’ve accomplished in the last year.  For me, this November marks my one-year anniversary with Corona Insights, and since Gregory joined our team this fall, I’m officially no longer the newbie! Future

It’s also the time of year when many start thinking about what the future may hold for the year ahead.  While I wouldn’t recommend calling me for your late night psychic readings (as I feel my fortune-telling skills are still a little below average), I can predict what you can expect if you hire Corona Insights for your research needs.

  • You’ll learn something new…One of the most exciting things about research is finding that nugget of information that you never knew existed. After conducting research, you’ll have a far better understanding of what your customers are thinking, some of which you’ll be hearing for the first time!
  • And you’ll confirm what you already know. Our clients are experts in their fields, so it’s no surprise that some information uncovered by our research confirms what they already know, and that’s a good thing! Having research and data to back-up what you already believed about your organization can be instrumental in making sound decisions to guide your organization forward.
  • You’ll be surprised by what people know…When I first started at Corona, I sometimes worried that people would not know enough about a topic or have enough opinions to fill the time of a full focus group, but my worries quickly melted away. People love to be asked their opinions, and are usually more than willing to share, even if they don’t know a lot about the topic.  Learning about people’s perceptions and opinions (even from those who don’t know a lot about your organization) and where those thoughts come from is extremely valuable.
  • And probably frustrated by what they don’t. I’m sure there is nothing more frustrating for our clients observing focus groups than hearing participants say they haven’t heard of their organization, or have misconceptions, despite all their best marketing efforts.  Understandably, clients often feel the urge to run in to the group and tell participants the real story.  But, from Corona’s perspective, it’s a GOOD thing to hear about what people don’t know, as well as their misconceptions.  After all, it’s hard to enhance your work when you don’t know what you most need to improve upon.  While it may be frustrating to hear, it’s probably part of why you decided to conduct research in the first place!

Above all I can predict (even without consulting my Magic 8 ball) that at the end of the day, you’ll come away with actionable insights.

How to Choose your own Adventure when it comes to Research

One of the things we’ve been doing at Corona this year that I’ve really enjoyed is resurrecting our book club. I enjoy it because it’s one way to think about the things we are doing from a bigger picture point of view, which is a welcome contrast to the project-specific thinking we are normally doing. One topic that’s come up repeatedly during our book club meetings is the pros and cons of different types of research methodology.

Knowing what kind of research you need to answer a question can be difficult if you have little experience with research. Understanding the different strengths and weaknesses of different methodologies can make the process a little easier and help ensure that you’re getting the most out of your research. Below I discuss some of the key differences between qualitative and quantitative research.

Qualitative Research

Qualitative research usually consists of interviews or focus groups, although other methodologies exist. The main benefit of qualitative research is that it is so open. Instead of constraining people in their responses, qualitative research generally allows for free-flowing, more natural responses. Focus group moderators and interviewers can respond in the moment to what participants are saying to draw out even deeper thinking about a topic. Qualitative research is great for brainstorming or finding key themes and language.

Qualitative data tend to be very rich, and you can explore many different themes within the data. One nice feature of qualitative research is that you can ask about topics that you have very little information about. For example, you might have a question in a survey that asks, “Which of the following best describes this organization? X, Y, Z, or none of the above.” This quantitative question assumes that X, Y, and Z are the three ways that people describe this organization, which requires at least some knowledge. A qualitative research question for this topic would ask, “How would you describe this organization?”. This is one of the reasons why qualitative research is great for exploratory research.

The primary weakness of qualitative research is that you can’t generate a valid population statistic from it. For example, although you could calculate what percent of people in focus groups said that Y was a barrier to working with your organization, you couldn’t generalize that estimate to the larger population. However, if you just wanted to identify the main barriers, then that would be possible with qualitative research. So even if 30% of focus group participants reported this barrier, we don’t know what percent of people overall would report that same barrier. We would only be able to say that this is a potential barrier. It’s important to think carefully about whether or not this would be a weakness for your research project.

Quantitative Research

The main goals of quantitative research are to estimate population quantities (e.g., 61% of your donors are in Colorado) and test for statistical difference between groups (e.g., donors in Colorado gave more money than those in other states). With quantitative research, you’re often sacrificing depth of understanding for precision.

One of the benefits to quantitative research, aside from being able to estimate population values, is that you can do a lot of interesting statistical analyses. Unlike a small sample of 30 people from focus groups, a large sample of 500 survey respondents allows for all sorts of analyses. You can look for statistical differences between groups, identify key clusters of respondents based on their responses, see if you can predict people’s responses from certain variables, etc.

There usually is not one single best way to answer a question with data, so thinking through your options and the benefits afforded by those options is important. And as always, we’re here to help you make these decisions if the project is complicated.

Question What You Know

What I’m about to disclose may seem weird, or familiar, depending on the kind of person you are.  Last week our Corona Book Club met to discuss our recent pick, and as I sat down to put together my thoughts on it, I was reminded of an ad campaign from my youth.  I googled the slogan (this is not so weird) and couldn’t find the ad online, so instead I pulled out my huge 3-ring binder where I save things like interesting ads and magazine articles and dug out my own personal copy from circa 1995 (that may be weird).

(Even more strange is that I only have a photocopy of this particular ad, so I have no idea where it came from, and it appears to be half of a two-page ad, so I’m not even sure what the ad is for. My guess is it was for Carnival Cruise Lines or Cirque du Soleil, but it could just as easily have been for perfume or Nike or Waterford Crystal – 90’s ads were full of angsty inspirational prose.  In fact, my google search turned up another blogger writing about one of my favorites – yes, also part of my hard copy collection.)

I digress.  As I wasQuestion what you know saying, the ad text, which reads: “What appears to be new may in fact be familiar. What appears to be familiar may in fact be new.  So question what you know … because you may not really know it at all.” resonated with a point made in the book: “Don’t treat everyday life as boring or obvious; do treat ‘obvious’ actions, settings and events as potentially remarkable.”.

The book is David Silverman’s, A very short, fairly interesting and reasonably cheap book about qualitative research.  In addition to his exhortation to pay attention to things that may seem unremarkable, he also encourages researchers to explore other research methods and other sources of data.  He points out, accurately, that most commercial qualitative research is limited to interviews and focus groups.  He suggests we broaden our horizons to consider observational research methods, analysis of natural language, written documents, and so on.  He provides examples to show the value and possibilities in these alternatives.

Having read this book, it seems to me that my binder of nearly-vintage ads could be a useful data source for studying the Gen X persona.  Intrepid grad students can request a copy.

And if you recognize this ad, please tell us about it!

Incorporating Exercises into Focus Groups

At Corona Insights, we are always researching best practices for the work we do.  In the world of qualitative research, this often means best practices for conducting focus groups.  Over the years, we have learned many tips and tricks for conducting focus groups, which includes incorporating exercises into our discussions.  There are a wide range of exercises and activities we use depending on the topic of discussion.  These exercises can include everything from drawing ideas to ranking priorities to testing messaging or ads.

Incorporating exercises into focus groups serves several important purposes:

  • It gives participants an opportunity to think about topics in a different way. It can sometimes be hard for participants to fully think through a topic when they are expected to quickly answer a question.  Allowing them more time to think about the topic in an exercise helps encourage different thinking and promotes answers and opinions that are below top of mind.Light Bulbs
  • It encourages those who are quiet to express their opinions. Some participants are quieter by nature, so it can be a challenge to hear their opinions in a group of 8-10 other people.  Incorporating exercises, and having participants share their thought process for completing the activity ensures that even the most shy of participants are participating.
  • It makes the group more interesting. Sitting in a room listening to someone ask questions for two hours can be exhausting, especially if the group takes place in the evening. Incorporating exercises into a focus group breaks up the model of the moderator asking questions and participants answering, and hopefully makes the group more fun!
  • It helps breaks up group think. Sometimes if there are strong personalities in the room, or if a topic is particularly controversial, participants can act as if they agree with each other on certain topics, even if this is not truly the case. This can also happen if most participants haven’t given the topic a lot of previous thought, and a few participants have more knowledge on the topic than others.  Having the participants work and think individually during an exercise ensures that the group is not being influenced by group think.

So, if you ever attend a Corona focus group, don’t be surprised to see participants doing more than just answering questions (and hopefully having more fun and expressing more thoughtful and insightful opinions because of it).

Wondering if you’re normal? The answer is probably yes.

Katherine Brown is Corona’s 2015 summer intern. She has one year left at Macalester College where she has chosen to study anthropology because it gives her an excuse to talk to people from all over the place and call it work. After living in Germany for a calendar year and Turkey for an academic one, Katherine is able to pass for a German and haggle for coffee sets in Turkish. She’s also well on her way to writing a senior thesis about the huge German Turkish population. When she’s not busy looking for people she’s never met to tell her about their lives in detail, she’s likely spending time with some portion of her Denver-based extended family in restaurants, the mountains, or both. 

Wondering if you’re normal? The answer is probably yes.
Written by Katherine Brown

Early this summer in the midst of looking for jobs and internships, I decided to make a documentary about my extended family. When I mentioned this idea to my eighteen year old cousin, she immediately wanted in. Since I’m all about teamwork and she’s more like a sister than a cousin to me anyway, it didn’t take much convincing at all before we were a small team of two.

Our first (and, I will add, only) meeting was a discussion of how to start the project. It’s hard to make a documentary about your own opinionated and very close family without offending anyone. We decided the best thing to do would be to write up a short questionnaire, send it out to the family, see where people had strong opinions, and go from there. Coming from an anthropology background and hours of training on how to frame questions, how to stay neutral, and how to make people comfortable enough to share information, coming up with five questions for my family seemed like nothing. “What are the biggest generational gaps?” “Do you see major points of contention? What are they?” I wanted to look for patterns, anything we could latch onto and say “YES! That’s it! Let’s dig deeper there!” My cousin, who just graduated from high school, was set on a different approach. She felt that it was crucial to know just one thing.

“Is our family normal? Why or why not?”

It took me a while to figure out why this one question seemed so crucial. Maybe she was looking for someone to confirm her suspicions that our family is not normal. Maybe she wanted everyone to tell her that we are, in fact, normal and that she didn’t need to worry. I mostly just put it off to the fact that she hadn’t had the training in asking questions that I have.

A couple of weeks later, I started working at Corona as an intern for the summer. By then, I’d more or less forgotten about the incident with my cousin. My life became filled with interviews and survey data. It wasn’t until this week that I had a chance to step back again and think about my cousin’s question. Apparently all of this first-hand exposure to research has been teaching me something, because I suddenly understood why she had been so focused on the question of normal. She knew what we were really trying to get at through this documentary.

“Am I normal?”

Through my coursework and experience interviewing people, I’ve always focused on the process. It never occurred to me that all the data analysis (whether from surveys or interviews or focus groups) I’ve done is really just a quest to find “normal”. Of course, each data set is slightly different, focusing on finding normalcies among slightly different populations. But no matter what kind of data I am looking at, people love to emphasize their own ‘normalness’.

In my academic research on German Turks, many interviews will start with people telling me that they are “not really the kind of person I’m looking for”. Their story is different from the ones I’m used to hearing because they don’t come from Eastern Anatolia. Or because they “did not have a choice when they moved to Germany”. Or because they “didn’t even move back to Turkey on purpose”.  And yet, in each interview I hear more or less the same thing. The details are different, of course. But there are notable patterns in what people emphasize when I ask what it means to be a German Turk. These people who present themselves as unique or abnormal almost always end up giving me details about how they are just like all the other German Turks. I hear “Anyone who has had experience in Germany will tell you that…” in almost every interview, including the ones that begin with the phrase “I’m not really the kind of person you’re looking for”. Those normal habits of abnormal people are exactly what I’m looking for when I’m building my senior thesis. Those habits are the ones that matter.

My exposure to qualitative research at Corona has shown me similar results, albeit without the introductory phrase of “I’m not the right person”. Research here is often conducted to find out people’s opinions. I’ve noticed that most people believe that their own ideas are also the most widely-held opinions, the ‘normal’ opinions. Answers to an interview question sometimes begin with pauses implying how ‘obvious’ the answer is going to be. In the next interview someone may pause for the same reason, then completely contradict the last person’s answer. What I’ve learned from this is that everyone is normal. Everyone knows the ins and outs of their industry. Everyone knows what everyone else thinks. At least until you talk to everyone else.

Even quantitative survey data allows room for people to remark upon their normal lives. People will circle survey questions and comment “why are you even asking this?”, either forgetting or oblivious to other people’s perceptions of normal. In open ended responses people will write that they do “the normal things” or go somewhere “right next to where you get off the train”. That version of normal may or may not be the same as mine. Many of these people are filling out the surveys quickly, and do not remember that they’ve never met me. Why would any one person’s experience be anything BUT normal?

I will spoil the end of the unfinished family documentary.  Everyone in my family responded that we are normal. Their responses to my open-ended, non-judgmental, broad, neutral questions varied pretty greatly. And yet, there were patterns. In my experience, following those patterns would lead to a pretty representative profile of a ‘normal’ family with the same education, socio-economic status, geographic location, and generational breakdown as my own. On her quest to find normal, my cousin may be beginning to realize that everything and everyone is normal. On my quest to become a better researcher, I’m beginning to realize that what I’m looking for is the answer to the question my cousin was so set on asking.

“Are you normal? Why or why not?”


It’s All About How You Ask

Last month, I attended the RIVA Training Institute in Rockville, Maryland to get a fresh look at how Corona tackles qualitative research.  It was heartening to find that our qualitative research practices match industry standards, but as with any good training, I came away with some new insights that will undoubtedly help make us better researchers.

One thing in particular that stood out to me was taking a look at how we phrase questions during focus groups and interviews.  Perhaps you have been in a situation where you feel like you’re asking a really valuable question, but aren’t getting comprehensive responses.  Why?  Well, the answer is actually in the question.  The problem may, in fact, be the word, “why”? WHY?

Of course, qualitative research is all about uncovering answers to “why” questions, but using the word “why” may actually prevent you from getting the answers you need.  The word “why” tends to put people on the defensive.  As an example, think back to when you were a teenager, and you were late for your curfew.  What’s the first thing your mom asked?  “Why are you late? Why didn’t you call? Why don’t you show some respect?”  You’re put on the spot and immediately feel the need to defend yourself.  In general, there are fairly negative connotations with the word “why”, so although it might be somewhat subconscious, people may be less likely to open up and give deeper responses if they’re asked to explain themselves in this way.

Of course, we still need to know the answer to these “why” questions, but using a different word choice will likely elicit more responses that are below top-of-mind.  Consider using phrases such as, “What are some reasons…” or “What influenced you to…” as alternatives.   It sounds simple, but it works wonders!  So, next time you ask a “why” question, but don’t get the answers you need, consider rephrasing.  You may be surprised to find it will not only help you get better answers as researcher, but also in other aspects of your life.

Welcome, Mollie!

We are delighted to welcome Mollie Boettcher as the newest Multi-Ethnic Group Of People Holding The Word Welcomemember of the Corona Insights team!

As our newest Associate, Mollie will specialize in qualitative research practices including, but certainly not limited to: recruiting research participants, conducting focus groups and interviews, then analyzing and interpreting qualitative data for clients seeking data-driven guidance.

Mollie attended the University of Winsonsin—La Crosse, where she majored in Business Management and minored in Chemistry. Mollie graduated with her B.S. in 2010.

When Mollie is not hard at work in her office, you can find her out hiking or snowshoeing in the Rocky Mountains. She also likes to explore Denver, including taking her dog on walks in Wash Park, and enjoying the many unique restaurants and breweries that Denver has to offer.

The relevance of reviews

Online reviewsRecently, Corona conducted some focus groups in part to understand how people make an important decision. In analyzing the groups, I found that in seeking resources to help make that decision, people preferred referrals from their family, friends and neighbors over quantitative data that was developed using a sophisticated methodology. Why? Participants viewed referrals as being more trustworthy and valuable than the data we presented.

This finding illustrates the fact that data comes in many forms and the data that people use in their decision making can range from the scientific to the intuitive. Along these lines, the finding from our focus groups wasn’t particularly surprising to me since I often rely on my own “neighbors” to help guide my decision making. Whether I want to find a new dentist, the best coffee in my neighborhood, or a hotel for my next trip, I usually look at customer ratings and reviews to get other people’s opinions first. According to this article, I am what’s been called a “social researcher” since I actively seek out and read customer reviews prior to making a purchasing decision.

It turns out I’m not alone, as 88 percent of consumers say that online reviews influence their buying decisions. Not only does the review content matter, but the number of reviews that a product has can also influence decision making, as consumers tend to buy products that have more reviews. According to the New York Times, reviews from ordinary people have become an “essential mechanism” for selling just about anything. Further, the article says that, in many situations, reviews are replacing marketing departments, press agents, advertisements, word of mouth and professional critics. The importance of reviews has even led businesses to plant reviews or hire people to produce a mass of reviews under different pseudonyms.

I know that using reviews to make decisions isn’t exactly scientific, but in my personal life, I think of customer reviews as a form of qualitative data which often answers questions like “Why did you like (or dislike) this product?” or “How did you use this product?” I’m also part of the eighty-one percent of consumers who use customer reviews to decide between multiple products, or to confirm that their final selection is the right one.

Interestingly, I don’t write many reviews myself. And while reviews are certainly part of my decision making, I know to take them with a grain of salt – especially since unhappy customers are considered more likely than satisfied customers to try and make their voices heard. Overall, however, I’ve found them to be a valuable and reliable resource. Does anyone you know feel the same way?

Yes, that college degree is still worth the cost of attendance

Is college still a good investmentHere at Corona, we help organizations make strategic decisions via our research and strategy services. But individuals can also use research to make strategic decisions.

We read a lot these days about the cost of higher education. While rises in health care costs get more attention, inflation in higher education costs have actually outpaced them, and the unprecedented question has begun arising about whether an education is worth the cost in modern America.

We have researched supply and demand issues, and have seen evidence that college grads are increasingly competing for jobs that traditionally haven’t required a college degree, and that many college grads are displacing workers without degrees in those positions. That’s not quite the door-opening experience that we would hope to see coming with a college degree, but nonetheless we see that, on the whole, people with higher education levels have more earning power and better employment prospects.

We recently did a quick analysis of wage and salary levels by education level for Coloradans between the ages of 30 and 34. At this age, most people have completed their education, and by controlling for age we can eliminate some factors such as older people being less likely to have degrees. We included all people in this age range, including those who aren’t working, to account for the fact that employment opportunities may differ by age level.

EDUThe results were interesting. We saw the following pattern:

As a quick back of the envelope analysis, we looked at the reported resident tuition and fee costs of public schools as reported by the Colorado Commission on Higher Education. We then added the lost earnings of pursuing education beyond high school, assuming two years of lost earnings for an associate degree, four years for a bachelor, and seven years for an advanced degree. We ignored living expenses since you’re presumably buying groceries and housing whether you go to college or not.

If you compare those costs to the income differential, we see a payback period of roughly 6 to 7 years for any type of post-secondary degree, assuming that you get no financial aid during your college years. The earning power of lesser degrees is lower, but the cost and time to get them is lower, and they even out. And obviously there are factors that we didn’t consider in this quick analysis, such as the type of degree a student earns.  But as a general rule, unless you plan to retire at 30, it’s pretty clear in this simple analysis that there’s a positive long-term financial payoff to pursuing higher education.

So for any of our blog readers considering the return on investment of college, we’ve done the data crunching, so you can now use it to make your strategic decision.