Question What You Know

What I’m about to disclose may seem weird, or familiar, depending on the kind of person you are.  Last week our Corona Book Club met to discuss our recent pick, and as I sat down to put together my thoughts on it, I was reminded of an ad campaign from my youth.  I googled the slogan (this is not so weird) and couldn’t find the ad online, so instead I pulled out my huge 3-ring binder where I save things like interesting ads and magazine articles and dug out my own personal copy from circa 1995 (that may be weird).

(Even more strange is that I only have a photocopy of this particular ad, so I have no idea where it came from, and it appears to be half of a two-page ad, so I’m not even sure what the ad is for. My guess is it was for Carnival Cruise Lines or Cirque du Soleil, but it could just as easily have been for perfume or Nike or Waterford Crystal – 90’s ads were full of angsty inspirational prose.  In fact, my google search turned up another blogger writing about one of my favorites – yes, also part of my hard copy collection.)

I digress.  As I wasQuestion what you know saying, the ad text, which reads: “What appears to be new may in fact be familiar. What appears to be familiar may in fact be new.  So question what you know … because you may not really know it at all.” resonated with a point made in the book: “Don’t treat everyday life as boring or obvious; do treat ‘obvious’ actions, settings and events as potentially remarkable.”.

The book is David Silverman’s, A very short, fairly interesting and reasonably cheap book about qualitative research.  In addition to his exhortation to pay attention to things that may seem unremarkable, he also encourages researchers to explore other research methods and other sources of data.  He points out, accurately, that most commercial qualitative research is limited to interviews and focus groups.  He suggests we broaden our horizons to consider observational research methods, analysis of natural language, written documents, and so on.  He provides examples to show the value and possibilities in these alternatives.

Having read this book, it seems to me that my binder of nearly-vintage ads could be a useful data source for studying the Gen X persona.  Intrepid grad students can request a copy.

And if you recognize this ad, please tell us about it!

What your response rate says about engagement

Businessman Filling FormWhen we think about tracking customer satisfaction via surveys, the analysis is almost always on the survey responses themselves: how many said they were satisfied, what is driving satisfaction, and so on. (See a related post on 4 ways to report customer satisfaction.)

Not shocking (and of course we should look at the results to questions we ask), but there is another layer of data that can be analyzed: the data about the survey itself.

First and foremost is response rate.  (Quick review: response rate is the proportion of people who respond to an invitation to take a survey; read more here.) Response rate itself is important to reduce non-response bias (i.e., to reduce our concern that the people who do not respond are potentially very different that those who do respond), but it’s also a proxy for engagement. The more engaged your customers are with your organization, the more likely they will be to participate in your research. Therefore, tracking response rate as a separate metric as part of your overall customer dashboard can provide more depth in understanding your current relationship with customers (or citizens or members or…).

So, you’re probably now asking, “What response rate correlates to high engagement?” Short answer – it depends. Industry, topic matter, type of respondent, sampling, etc. can all make an impact on response rates. So while I’ll offer some general rules of thumb, take them with a grain of salt:

  • Less than 10%: Low engagement
  • 10-20%: Average engagement
  • 20-50%: High engagement
  • 50%+: Rock-star territory

Yes, we’ve had over 50% response to our clients’ surveys.

The important caveat here is to be weary of survey fatigue. If you are over surveying your customers, then response rates will decrease over time as people tire of taking surveys (shocking, right?). What is considered surveying too much will vary depending on the length of the survey and subject matter, but surveying monthly (or more frequently) will almost certainly cause fatigue, while surveying yearly (or less frequently) will probably not cause fatigue. One to 12 months? It’s a gray area. (Feel free to contact us for an opinion on your specific case).

Another potential source of survey meta data that you could use to assess engagement is depth of response to open-ended questions. The easiest way to measure this is to use word count as a proxy  – the more they write, presumably the more they care enough to tell you something.

For example, we did a large voter study for a state agency, and when asked their priorities on the given service topic, we received paragraph responses back. This, combined with other results, showed just how engaged they were with the topic (though not necessarily the agency in this case) and how much thought they had given it. Useful, as well as somewhat surprising, information for our client.

The next time you’re looking at survey data, be sure to look at more than just the responses themselves.


The Challenging In-Between: Bridging the gap among visionaries and operational experts

Over the years I’ve discovered that nonprofit executives and board members typically fall into two main categories: those who are boldly aspirational and those who are decidedly tactical. The first focuses on the big idea and its power to move people. They are lofty, passionate and effervescent. To the operationally-focused person they appear to dodge the important nuts and bolts. On the other hand, the executive whose natural talents lie in operations and the ability to get things done (often in spite of their visionary peers), are naturally challenged to let go and dream big. They seek the known. Their tendency to quickly dive into the weedy details is off-putting to the visionary.

Unfortunately there can be little in common between these divergent thinking styles and they often frustrate the heck out of each other. They use different language, or interpret a common term in opposite ways, and don’t know how to create the connective tissue that binds the two important orientations together.

BridgeWhat is the bridge? Strategy. The essence of strategy lies in charting the unknowns – where the industry is going to be, what customers will need (and demand) in the future, what donors will expect, and how the community will be doing. It also rests in a clear articulation of how the organization uniquely meets its customers’ needs in ways that rivals can’t or don’t. The strategist navigates unknowns and uncertainties. S/he keeps her eye on the 3-5 year horizon as she leads the development of a clear strategy – an articulation of what the organization will focus on over the next 3-5 years based upon well-founded decisions about the objective to be achieved, the scope within which it will work, and the competitive advantage to be leveraged.

I’m a big believer in the power of a strategy statement as described in the classic Harvard Business Review article from 2008 – Can you say what your strategy is? – by David J. Collis and Michael G. Rukstad. It is my go-to resource in this work and cannot recommend it strongly enough.

As a consultant I’m often the bridge builder – the strategy seeker – bringing together the operationally- and aspirationally-oriented executives. This bridge building is iterative. It takes time to spark a ha’s, establish common language, and build a team of executives focused on the same thing – future strategy.

Navigating Time and Space: Why we Include Geography

When I was a kid living in Colorado Springs, my family frequently drove to Denver.  We would go to watch baseball games or visit museums.  It is about 60 miles between Colorado Springs and Denver.  Back then, the speed limit was 60 to 65 miles per hour, and I had fun mentally calculating that since our Jeep was traveling about one mile per minute, then we would reach Denver in about one hour. Of course, this was all before the electronic geographic information revolution.

These days, we can quickly discover the distance from one place to another with just a few clicks on Google Maps or onboard GPS devices, and there are numerous software programs that calculate typical drive, bike or walk times.

Here at Corona, we leverage these geographic information technologies for many of our research projects when we suspect that there is a spatial (i.e., geographic) relationship with our key variables.  In other words, knowing the distance or travel time between two locations can reveal key insights.  For example, let’s say the Colorado Rockies baseball team wanted us to survey fans living in the Metro Area to understand what barriers prevent them from traveling to Coors Field to watch a game.  Our analysis would likely explore their opinions about going to a game (e.g., are tickets well priced, are games played at convenient times, etc), but we might expect that these opinions are influenced, in part, on the distance or travel time between their home and the ballpark. Fans living far away from the ballpark might have stronger convenience barriers than fans living closer to the ballpark.Drivetime Map

To explore this hypothesis, we can use GIS software to plot survey respondents’ homes. Then we decide to analyze by distance or by travel time. This choice depends on the research question we are trying to answer, as well as the context of the research. In a study of opinions about sound or light pollution, analyzing distance clearly makes more sense. If a behavior of interest involves walking or biking, then distance might be more important than travel time, considering walking 100 miles is a significant feat, but people frequently travel that distance by car. Alternatively, a study in a city where all the streets are linear and the speed limits are the same, drive time would be directly related to distance, so the unit of measurement wouldn’t matter.  However, in many of our projects, such as the Coors Field example, we are most interested in drive time.  Using drive times has a big advantage when the transportation system is not linear, which is often the case due to interstate highways, bridges, mountains, canyons, no-travel zones, construction, and a host of other reasons.  Considering drive time during rush hour is likely longer than an early weekend morning, our software allows us to specify a drive time to the day and hour.

So how does including geographic data benefit analysis and improve insights?  Most simply, we create custom segments based on distance, and we create easy-to-understand graphs that cross results to other questions by this variable. Segmentation is a good start, but we rarely stop there.  On many projects, the research demands more rigorous results, in which cases we will convert the data so that we can apply a more advanced analyses that tells us the strength of relationships to other key variables. For example, we can find out the extent that drive time to Coors field predicts fans’ perceived barrier to going there for a game.  In fact, we can explore multiple variables (e.g., ticket prices, fan devotion, and drive time) simultaneously to reveal patterns that would otherwise be difficult to tease apart. In some cases, we calculate drive times to other site, such as other leisure attractions.  We then incorporate that data to the analyses so that the results more closely reflect the real world, where decisions on how to spend leisure time are more complex.

While geography won’t provided all of the answers to our research questions, distance and drive time can be a key variable that helps explain what’s going on.  By using geographic technologies, we can efficiently explore this variable and sharpen our findings and recommendations.  In other words, it helps us paint a more complete picture.

Send us an email if you would like to discuss how analyzing spatial patterns could help answer your most important questions.

Incorporating Exercises into Focus Groups

At Corona Insights, we are always researching best practices for the work we do.  In the world of qualitative research, this often means best practices for conducting focus groups.  Over the years, we have learned many tips and tricks for conducting focus groups, which includes incorporating exercises into our discussions.  There are a wide range of exercises and activities we use depending on the topic of discussion.  These exercises can include everything from drawing ideas to ranking priorities to testing messaging or ads.

Incorporating exercises into focus groups serves several important purposes:

  • It gives participants an opportunity to think about topics in a different way. It can sometimes be hard for participants to fully think through a topic when they are expected to quickly answer a question.  Allowing them more time to think about the topic in an exercise helps encourage different thinking and promotes answers and opinions that are below top of mind.Light Bulbs
  • It encourages those who are quiet to express their opinions. Some participants are quieter by nature, so it can be a challenge to hear their opinions in a group of 8-10 other people.  Incorporating exercises, and having participants share their thought process for completing the activity ensures that even the most shy of participants are participating.
  • It makes the group more interesting. Sitting in a room listening to someone ask questions for two hours can be exhausting, especially if the group takes place in the evening. Incorporating exercises into a focus group breaks up the model of the moderator asking questions and participants answering, and hopefully makes the group more fun!
  • It helps breaks up group think. Sometimes if there are strong personalities in the room, or if a topic is particularly controversial, participants can act as if they agree with each other on certain topics, even if this is not truly the case. This can also happen if most participants haven’t given the topic a lot of previous thought, and a few participants have more knowledge on the topic than others.  Having the participants work and think individually during an exercise ensures that the group is not being influenced by group think.

So, if you ever attend a Corona focus group, don’t be surprised to see participants doing more than just answering questions (and hopefully having more fun and expressing more thoughtful and insightful opinions because of it).

Graphs: An effective tool, but use them carefully

Ahh…the graph.  Where would the business world be without them?  While some of us are just as content looking through a giant spreadsheet full of numbers, graphs can help to illustrate the story more effectively for number geeks and math haters alike.  However, while graphs can be a great tool, there are certainly times when graphs can make interpreting your data even more difficult to understand or (intentional or not) even misleading.  Here are a few things to think about when creating graphs for your data.

Line charts should represent something linear!

Line charts are a very common way of representing data.  However, in most cases, line charts should only be used if there is some sort of linear relationship in the categories displayed on the horizontal axis of your chart.  For example, if you want to see how responses vary by age of respondents, the year the data was collected, or even satisfaction on a numeric scale, a line chart can be a great way of representing this data.

DO: Customer Satisfaction by Year

However, if you are instead dealing with categorical data, using a line chart suggests a relationship between the categories that may not be true.   In the example below, a line chart implies that Colorado is related to Nebraska in the same way that Nebraska is related to Wyoming.  Clearly this isn’t true, so in this case, a bar chart would likely be a more effective way of presenting the data.

DON'T: Customer Satisfaction by State

Pie charts (and split-bar charts) should add to 100%!

Everyone loves pie charts.  Not only are they the best type of graph to use if you want to represent your favorite food (or video game character), they are an excellent way of presenting the distribution of data in which every data point belongs to one category.

DO: Gender

However, pie charts can cause all sorts of problems in interpretation if the data is not mutually exclusive (that is, if a single data point can belong to multiple categories).  In the example below, the pie chart implies that the chart represents the total population in terms of pet ownership, but some people may have multiple types of animals.  Again, in cases such as these, a bar chart would be a much more clear way of presenting this data.

DON'T: Pets at Home

Be careful with the scales you use!

While bar charts can be a good option for a wide variety of data, the scale you use for your charts can cause confusion in interpretation if you aren’t careful.  In particular, if you let your graphing software choose your scale for you, you may end up with results that tell a very different story than reality!  For example, let’s say you wanted to compare customer satisfaction across customer segments.  If you pulled out trusty old Excel and graphed this data with no modifications, here’s the output you would get:

DON'T: Satisfaction with Automatic Scaling

Even with the percentages listed on the graph, it looks like there is a HUGE difference in the satisfaction of Segments A and B compared to the others.  However, the difference in satisfaction between Segments A and E is really only 13 percent.  Here’s the same data, but using a fixed scale from 0 to 100 percent:

DO: Satisfaction with Fixed Scaling

By ensuring that the scale represents the entire range of possible responses, we can more accurately convey the true differences between segments.

Other things to consider

These are, of course, just a few things to consider when presenting your data.  We haven’t even touched on other topics like overall charting philosophies, how newer visualization techniques can result in pretty, but dysfunctional graphs, or the nuances of more advanced types of visualizations, such as cartography.  However, by keeping in mind what your data is meant to represent and ensuring that your approach avoids some of these pitfalls, you’ll be on your way to more meaningful and accurate graphs.

Welcome, Gregory!

We are delighted to introduce Gregory Hornback as the newest Welcome
member of the Corona Insights team! To learn more about Gregory’s role at Corona Insights, check out his bio here.

We also thought it’d be fun to include a little Q&A to help us and our readers learn more about Gregory. Enjoy!

Q. What is your favorite hobby?
A. I have a bunch of odd hobbies – like mechanical keyboards and wet shaving – but my favorite would probably be computers. I built my first computer when I was in high school and have enjoyed messing with the technology ever since. Just like most other computer enthusiasts, my computer really gets put through its paces when I’m enjoying the most recent and demanding games.

Q. What are you looking forward to most now that you live in Colorado?
A. Colorado has been my favorite state ever since I was young. Being able to drive up into the mountains just to ski or snowboard for the day is a lifelong dream of mine, so I would definitely say the mountains. Also, it’s not well known that Indiana is brutally humid, so every day I am looking forward to enjoying the pleasant, low-humidity weather here in Colorado.

Q. What is your favorite book?
A. Despite my love for high-fantasy books (magic, mystical creatures, etc.), my all-time favorite book would have to be The Stand by Stephen King. I found it to be a fascinating portrayal of humanity and survival in a post-apocalyptic world. Though the book might find its roots in horror, it is truly one that everyone should pick up.

Q. Favorite movie?
A. No Country for Old Men definitely takes the cake.

Q. What is your perfect pizza?
A. My perfect pizza has a few key criteria: 1) it must have a fantastic crust that has a nice and crisp, but not burnt, outer edge with a firm yet spongy inside 2) the sauce must have some sweetness to it but not taste overly sweet 3) it should have a saltier taste baked into the cheese and 4) it should contain extra cheese on one half and pepperoni, ham, bacon, and Canadian bacon on the other.

Q. Favorite college course?
A. My favorite undergraduate course was almost definitely my consumer psychology course. It was a course that combined my love of studying people and my interest in consumerism and economics. It contained the context (within the larger field of psychology) that really spoke to me. In my graduate studies, my favorite course was Online Interaction, a course detailing the history of the internet, how communities formed, and everything on how individuals interact with one another via computer-mediated-communication.

Q. Why Market Research?
A. I originally started out my college career majoring in Chemistry. After I quickly discovered chemistry (and to a greater degree, the required physics courses) was not for me, I fell back to a field I knew I loved, but I wasn’t sure how to make a career out of – Psychology. Eventually, after focusing most of my education around social-psychology, I stumbled upon the field of Consumer Behavior. A string of video games were released that absolutely bombed, despite insanely high budgets, and I was extremely confused; I knew these companies could spend even a small percentage of their budgets on researching how people react to certain aspects of their games in development, but they didn’t seem to be. It was then that I found my passion. Studying how people act as consumers has been my focus ever since, eventually leading to a graduate degree in the field. In my mind, market research is a way in which I can utilize my passion and make it a career.

Wondering if you’re normal? The answer is probably yes.

Katherine Brown is Corona’s 2015 summer intern. She has one year left at Macalester College where she has chosen to study anthropology because it gives her an excuse to talk to people from all over the place and call it work. After living in Germany for a calendar year and Turkey for an academic one, Katherine is able to pass for a German and haggle for coffee sets in Turkish. She’s also well on her way to writing a senior thesis about the huge German Turkish population. When she’s not busy looking for people she’s never met to tell her about their lives in detail, she’s likely spending time with some portion of her Denver-based extended family in restaurants, the mountains, or both. 

Wondering if you’re normal? The answer is probably yes.
Written by Katherine Brown

Early this summer in the midst of looking for jobs and internships, I decided to make a documentary about my extended family. When I mentioned this idea to my eighteen year old cousin, she immediately wanted in. Since I’m all about teamwork and she’s more like a sister than a cousin to me anyway, it didn’t take much convincing at all before we were a small team of two.

Our first (and, I will add, only) meeting was a discussion of how to start the project. It’s hard to make a documentary about your own opinionated and very close family without offending anyone. We decided the best thing to do would be to write up a short questionnaire, send it out to the family, see where people had strong opinions, and go from there. Coming from an anthropology background and hours of training on how to frame questions, how to stay neutral, and how to make people comfortable enough to share information, coming up with five questions for my family seemed like nothing. “What are the biggest generational gaps?” “Do you see major points of contention? What are they?” I wanted to look for patterns, anything we could latch onto and say “YES! That’s it! Let’s dig deeper there!” My cousin, who just graduated from high school, was set on a different approach. She felt that it was crucial to know just one thing.

“Is our family normal? Why or why not?”

It took me a while to figure out why this one question seemed so crucial. Maybe she was looking for someone to confirm her suspicions that our family is not normal. Maybe she wanted everyone to tell her that we are, in fact, normal and that she didn’t need to worry. I mostly just put it off to the fact that she hadn’t had the training in asking questions that I have.

A couple of weeks later, I started working at Corona as an intern for the summer. By then, I’d more or less forgotten about the incident with my cousin. My life became filled with interviews and survey data. It wasn’t until this week that I had a chance to step back again and think about my cousin’s question. Apparently all of this first-hand exposure to research has been teaching me something, because I suddenly understood why she had been so focused on the question of normal. She knew what we were really trying to get at through this documentary.

“Am I normal?”

Through my coursework and experience interviewing people, I’ve always focused on the process. It never occurred to me that all the data analysis (whether from surveys or interviews or focus groups) I’ve done is really just a quest to find “normal”. Of course, each data set is slightly different, focusing on finding normalcies among slightly different populations. But no matter what kind of data I am looking at, people love to emphasize their own ‘normalness’.

In my academic research on German Turks, many interviews will start with people telling me that they are “not really the kind of person I’m looking for”. Their story is different from the ones I’m used to hearing because they don’t come from Eastern Anatolia. Or because they “did not have a choice when they moved to Germany”. Or because they “didn’t even move back to Turkey on purpose”.  And yet, in each interview I hear more or less the same thing. The details are different, of course. But there are notable patterns in what people emphasize when I ask what it means to be a German Turk. These people who present themselves as unique or abnormal almost always end up giving me details about how they are just like all the other German Turks. I hear “Anyone who has had experience in Germany will tell you that…” in almost every interview, including the ones that begin with the phrase “I’m not really the kind of person you’re looking for”. Those normal habits of abnormal people are exactly what I’m looking for when I’m building my senior thesis. Those habits are the ones that matter.

My exposure to qualitative research at Corona has shown me similar results, albeit without the introductory phrase of “I’m not the right person”. Research here is often conducted to find out people’s opinions. I’ve noticed that most people believe that their own ideas are also the most widely-held opinions, the ‘normal’ opinions. Answers to an interview question sometimes begin with pauses implying how ‘obvious’ the answer is going to be. In the next interview someone may pause for the same reason, then completely contradict the last person’s answer. What I’ve learned from this is that everyone is normal. Everyone knows the ins and outs of their industry. Everyone knows what everyone else thinks. At least until you talk to everyone else.

Even quantitative survey data allows room for people to remark upon their normal lives. People will circle survey questions and comment “why are you even asking this?”, either forgetting or oblivious to other people’s perceptions of normal. In open ended responses people will write that they do “the normal things” or go somewhere “right next to where you get off the train”. That version of normal may or may not be the same as mine. Many of these people are filling out the surveys quickly, and do not remember that they’ve never met me. Why would any one person’s experience be anything BUT normal?

I will spoil the end of the unfinished family documentary.  Everyone in my family responded that we are normal. Their responses to my open-ended, non-judgmental, broad, neutral questions varied pretty greatly. And yet, there were patterns. In my experience, following those patterns would lead to a pretty representative profile of a ‘normal’ family with the same education, socio-economic status, geographic location, and generational breakdown as my own. On her quest to find normal, my cousin may be beginning to realize that everything and everyone is normal. On my quest to become a better researcher, I’m beginning to realize that what I’m looking for is the answer to the question my cousin was so set on asking.

“Are you normal? Why or why not?”


The more things change…

If you have been reading our blog you probably have noticed that our summer here at Corona has been one of professional development and learning. Granted, we like to think every season is, but maybe it’s the heat that makes staying indoors and reading or conferencing all the more attractive.

Conferences especially are supposed to show off the latest and greatest, and the MRA’s Insights and Strategies Conference certainly had that, from Trend Hunter presenting, to Adobe and other “future of market research” keynotes.

What has been particularly interesting to me, though, is the “back to basics” theme of many conference sessions, books, and webinars as of late. Take for example these takeaways from some recent conference sessions I attended:

  • Staying on top of demographic trends. Census data isn’t always sexy, but it’s reliable and paints the best picture of America and how it’s changing.  Taking a step back to gut check how you think your market is changing to how it actually is changing is invaluable.
  • Phone research isn’t dead. Our clients, I think, are sometimes surprised when we propose a telephone survey over online.  Yes, telephone (including cell phone) still exists, and in fact can be the best sampling means out there depending on your objective (and price competitive too).
  • Know what you’re trying to accomplish before starting the research. Seems obvious, but worth repeating. This was interestingly both the topic at the conference as well as Corona’s most recent book club (see Mollie’s post here). Knowing how you’ll use the research can inform what type of information you need to learn which will then inform the methodology.

Indeed, the more things change the more they stay the same. If only we all got summer vacation again too.