Wondering if you’re normal? The answer is probably yes.

Katherine Brown is Corona’s 2015 summer intern. She has one year left at Macalester College where she has chosen to study anthropology because it gives her an excuse to talk to people from all over the place and call it work. After living in Germany for a calendar year and Turkey for an academic one, Katherine is able to pass for a German and haggle for coffee sets in Turkish. She’s also well on her way to writing a senior thesis about the huge German Turkish population. When she’s not busy looking for people she’s never met to tell her about their lives in detail, she’s likely spending time with some portion of her Denver-based extended family in restaurants, the mountains, or both. 

Wondering if you’re normal? The answer is probably yes.
Written by Katherine Brown

Early this summer in the midst of looking for jobs and internships, I decided to make a documentary about my extended family. When I mentioned this idea to my eighteen year old cousin, she immediately wanted in. Since I’m all about teamwork and she’s more like a sister than a cousin to me anyway, it didn’t take much convincing at all before we were a small team of two.

Our first (and, I will add, only) meeting was a discussion of how to start the project. It’s hard to make a documentary about your own opinionated and very close family without offending anyone. We decided the best thing to do would be to write up a short questionnaire, send it out to the family, see where people had strong opinions, and go from there. Coming from an anthropology background and hours of training on how to frame questions, how to stay neutral, and how to make people comfortable enough to share information, coming up with five questions for my family seemed like nothing. “What are the biggest generational gaps?” “Do you see major points of contention? What are they?” I wanted to look for patterns, anything we could latch onto and say “YES! That’s it! Let’s dig deeper there!” My cousin, who just graduated from high school, was set on a different approach. She felt that it was crucial to know just one thing.

“Is our family normal? Why or why not?”

It took me a while to figure out why this one question seemed so crucial. Maybe she was looking for someone to confirm her suspicions that our family is not normal. Maybe she wanted everyone to tell her that we are, in fact, normal and that she didn’t need to worry. I mostly just put it off to the fact that she hadn’t had the training in asking questions that I have.

A couple of weeks later, I started working at Corona as an intern for the summer. By then, I’d more or less forgotten about the incident with my cousin. My life became filled with interviews and survey data. It wasn’t until this week that I had a chance to step back again and think about my cousin’s question. Apparently all of this first-hand exposure to research has been teaching me something, because I suddenly understood why she had been so focused on the question of normal. She knew what we were really trying to get at through this documentary.

“Am I normal?”

Through my coursework and experience interviewing people, I’ve always focused on the process. It never occurred to me that all the data analysis (whether from surveys or interviews or focus groups) I’ve done is really just a quest to find “normal”. Of course, each data set is slightly different, focusing on finding normalcies among slightly different populations. But no matter what kind of data I am looking at, people love to emphasize their own ‘normalness’.

In my academic research on German Turks, many interviews will start with people telling me that they are “not really the kind of person I’m looking for”. Their story is different from the ones I’m used to hearing because they don’t come from Eastern Anatolia. Or because they “did not have a choice when they moved to Germany”. Or because they “didn’t even move back to Turkey on purpose”.  And yet, in each interview I hear more or less the same thing. The details are different, of course. But there are notable patterns in what people emphasize when I ask what it means to be a German Turk. These people who present themselves as unique or abnormal almost always end up giving me details about how they are just like all the other German Turks. I hear “Anyone who has had experience in Germany will tell you that…” in almost every interview, including the ones that begin with the phrase “I’m not really the kind of person you’re looking for”. Those normal habits of abnormal people are exactly what I’m looking for when I’m building my senior thesis. Those habits are the ones that matter.

My exposure to qualitative research at Corona has shown me similar results, albeit without the introductory phrase of “I’m not the right person”. Research here is often conducted to find out people’s opinions. I’ve noticed that most people believe that their own ideas are also the most widely-held opinions, the ‘normal’ opinions. Answers to an interview question sometimes begin with pauses implying how ‘obvious’ the answer is going to be. In the next interview someone may pause for the same reason, then completely contradict the last person’s answer. What I’ve learned from this is that everyone is normal. Everyone knows the ins and outs of their industry. Everyone knows what everyone else thinks. At least until you talk to everyone else.

Even quantitative survey data allows room for people to remark upon their normal lives. People will circle survey questions and comment “why are you even asking this?”, either forgetting or oblivious to other people’s perceptions of normal. In open ended responses people will write that they do “the normal things” or go somewhere “right next to where you get off the train”. That version of normal may or may not be the same as mine. Many of these people are filling out the surveys quickly, and do not remember that they’ve never met me. Why would any one person’s experience be anything BUT normal?

I will spoil the end of the unfinished family documentary.  Everyone in my family responded that we are normal. Their responses to my open-ended, non-judgmental, broad, neutral questions varied pretty greatly. And yet, there were patterns. In my experience, following those patterns would lead to a pretty representative profile of a ‘normal’ family with the same education, socio-economic status, geographic location, and generational breakdown as my own. On her quest to find normal, my cousin may be beginning to realize that everything and everyone is normal. On my quest to become a better researcher, I’m beginning to realize that what I’m looking for is the answer to the question my cousin was so set on asking.

“Are you normal? Why or why not?”


The more things change…

If you have been reading our blog you probably have noticed that our summer here at Corona has been one of professional development and learning. Granted, we like to think every season is, but maybe it’s the heat that makes staying indoors and reading or conferencing all the more attractive.

Conferences especially are supposed to show off the latest and greatest, and the MRA’s Insights and Strategies Conference certainly had that, from Trend Hunter presenting, to Adobe and other “future of market research” keynotes.

What has been particularly interesting to me, though, is the “back to basics” theme of many conference sessions, books, and webinars as of late. Take for example these takeaways from some recent conference sessions I attended:

  • Staying on top of demographic trends. Census data isn’t always sexy, but it’s reliable and paints the best picture of America and how it’s changing.  Taking a step back to gut check how you think your market is changing to how it actually is changing is invaluable.
  • Phone research isn’t dead. Our clients, I think, are sometimes surprised when we propose a telephone survey over online.  Yes, telephone (including cell phone) still exists, and in fact can be the best sampling means out there depending on your objective (and price competitive too).
  • Know what you’re trying to accomplish before starting the research. Seems obvious, but worth repeating. This was interestingly both the topic at the conference as well as Corona’s most recent book club (see Mollie’s post here). Knowing how you’ll use the research can inform what type of information you need to learn which will then inform the methodology.

Indeed, the more things change the more they stay the same. If only we all got summer vacation again too.

Hey honey, wanna help me with a big project?

Facilitating success over 15 years of love and consulting.

In my mind, the question with milestones is this, which one is significant enough to celebrate? First year, 10, 20 or 50 – no brainer. What about 15? Sometimes the mid-decade mark is an important one. That’s the case for me this year as I celebrate 15 years with my company, Corona Insights. Oh, lots has changed over those years. I’m no longer officing out of the spare bedroom with the double bed that was great for holding files and for taking naps. Today I can catnap in my comfy office chair. Oh, and then there is air conditioning instead of a swamp cooler. Plus I have more colleagues than the one who is forever first for me, Kevin Raines.

We don’t do payroll in-house anymore off the 1st Bank checkbook. Now we have a payroll service and a super talented office manager to handle all of that stuff. Like many a start-up there was no salary for the first six months.

Those first six months of 2000 were a bit like the austerity period when Kevin and I were saving to become homeowners in 1994. No frills, no fluff. Our old house would just have to wait a bit as we saved for future remodels, especially since Raines Manor also served as Corona world headquarters. Who knew that the habits we’d built as first-time home buyers, and later remodelers, would serve us well as we had to chart the financial course for our company?

I’ve had the joy and Now, Future, Pastpleasure of building my consulting practice from scratch since 2000. I’ve also had the joy and pleasure of helping to build a small business too. Of course, if I’m honest, there has been some pain too, maybe quite a bit depending on the day, month or year. 2010, in the depths of the recession, many a painful moment. Certain challenging consulting gigs required many hours of free therapy from fellow consultants and colleagues. But I get ahead of myself once again.

Over the next several blogs I will reflect on my early years as a Jane of all trades, back in the day when I staffed research projects. (Some of you will recall the observational seat belt research we conducted on Denver street corners.) Then I’ll focus on the clients and projects that served as the crucible for my consulting practice. Finally, I’ll touch on what keeps me fresh and motivated today.

In the meantime it is interesting to note how much 2015 is like 2000. The nonprofit strategic consulting market is as fragmented today as it was then, possibly even more so. It remains largely populated by individual practitioners and smaller firms, many of which are still going today, although a few of my consultant peers have left the field to pursue other dreams and new folks have come into the market.

Business is still largely derived from repeat and referral sources. Friends and colleagues mattered then and matter now. What also matters is deep passion and love for my craft and the people who have helped me master it. There are too many of you to mention by name – and I thank you from the bottom of my heart.

Like they did in 2000, 2005 and 2010, prospective clients often begin their call or email with “I’d like to speak with you about facilitating our strategic planning process.”

Fortunately, some things never change.

I remain deeply indebted to my beloved who said, “Hey, I won a big project. Do you wanna help?” Who knew love and consulting would go so well together?


Adventures in Book Club: A Research Story

As it finally starts to feel like summer, one of my favorite activities is sitting outside on the patio reading a good book.  My fellow Coronerds must agree with this sentiment as we recently kicked off summer with our very own Corona Book Club, fearlessly led by Senior Associate, Matt Bruce.  As we started discussing this quarter’s book, I got to thinking that the ingredients for a successful book club really mirror the ingredients for a successful research project.

Coronerd Book Club

Those of you who may have participated in a book club on your own personal time may have noticed that some book clubs thrive, while others flop.  I would argue that success starts with clearly defining a purpose.  Is the purpose of the book club to socialize, to learn new writing techniques or to learn about a specific topic?  If all members of the club don’t agree on a purpose, it probably won’t be successful.   Imagine the chaos that may ensue if half of the members haven’t read the book because they came just to socialize, while the other half planned to have an in-depth discussion about literary techniques.  If you don’t have a clear purpose for the book club, you won’t be able to agree on a book or have a meaningful discussion.

The same is true for research projects.  We’ve talked a lot in the past about beginning with the end in mind—that is, being clear about why the research is being conducted in the first place, and how it will be used once complete.  When our clients are clear about the purpose behind the research project, and all stakeholders are on the same page, we are better able to guide the research by creating research goals, asking the right questions and using appropriate analysis techniques so that the research is actionable.

Wondering about the purpose of Corona’s book club?  It’s to continually learn about ways to improve our processes to make our work more useful to our clients, of course!

Tax Burden by County in Colorado

We saw an article in the Durango Herald the other day noting that La Plata County has the fourth-lowest tax burden in the state, and it made us curious.  We found the web site where the figures were posted.

The study used assumptions about a typical household, and calculated its cost in state income tax, property tax, sales tax, and fuel tax.  Because Colorado has a flat-rate income tax, that’s a wash throughout the state.

If one examines only the other three costs, the lowest tax for this hypothetical household is in Las Animas County at $1,337.  The next lowest rates are found in Gilpin County ($1,370), San Miguel County ($1,467), La Plata County ($1,676), and Gunnison County ($1,777).  The median among the 58 counties measured was $2,292.  At the bottom, the most heavily taxed Colorado residents live in Adams County, where the same household will pay $3,635, almost 60 percent above the median and nearly triple the Las Animas rate.  The difference from the median in Adams County is almost exclusively due to higher property taxes.

Tracking H2O

As a skiing and a river rafting enthusiast, I’m interested in snow levels of our nearby Rocky Mountains and of water levels in our creeks and rivers.  Early summer is therefore an exciting time to monitor snowpack and river levels because changes are dramatic. Luckily, two websites provide current data on snowpack and river levels that illustrate data in interesting ways.

To understand snowpack levels, I surf over to the National Resource Conservation Service (NRCS) SNOTEL website, which tracks snowpack across eight watersheds in Colorado. This website is easy to use and the data is updated about every four to six days.

Since I live in the South Platte River Basin, I frequently check out our time series snowpack summary, which displays snowpack estimates for the current year as well as the previous three years. As you can see below, the average and median (called normal) snowpack levels (red lines) are close to identical.  Looking at these lines, you can see that May 1st is a typical “tipping point” that leads to a downward melting trend.

We bucked the trend in 2015.  This year (dark blue line) was pretty close to normal until March when we went through a six-week dry spell. Storms and cool weather in April and May helped pull the snowpack upwards until we peaked on May 24, which is about a month after our normal peak day.  Although we are now sliding into snowmelt season, you can see that our current snowpack, as a percent, is 309% greater than a normal year.

South Platte River Bason Time Series Snowpack Summary Graph

After looking at our snowpack levels, I frequently cruise over to the United States Geological Survey (USGS) Water Information System website, which tracks real-time water levels in rivers and creeks nationwide.  Their top-level map depicts river and creek water levels as a percentile of that day over a 30-year average.  Just as any good data-driven exhibit should do, their national map makes it easy to see major trends and patterns. Rivers in the Pacific Coast states are very low, with many running below the 10th percentile.  The Great Plains states have experienced a very wet spring and early summer, and rivers here are generally running very high compared to normal.  The Appalachia and Atlantic Coast rivers are normal or low.

River Levels Map

Zooming into the Denver area, we see our river levels are very high, with most above the 90th percentile.  By clicking on a dot (which represent water level gauges), you can get information about a specific sites.  For example, the chart below shows water levels of the South Platte River in Englewood, CO, a suburb of Denver. You can see two steep spikes on June 5th, one in the early morning and one in the evening, that were caused by strong storms. Evidence of the first storm was obvious to the residents in Southeast Denver, who woke up that morning to the sound of front-end loaders moving four feet of hail off their streets.

South Platte River Levels_June

While the June 5th spikes are related to localized rainstorms, the melting snowpack is also influencing water levels.  If we extend the graph from seven days to 60 days, we see that water levels have consistently climbed since early April, due to spring rainstorms and melting snowpack.  The spikes of June 5th, which looked so dramatic in the top graph, become just one of many steep spikes the river has seen in the past couple of months.

South Platte River Levels_Spring

So who cares about all this anyway?  Well, you might care if you operate a tourism business in Colorado.  For example, organizers of running races scheduled to take place in the high mountains changed their routes due to snow covered trails, and two popular driving roads, Trail Ridge Road in Rocky Mountain National Park and Mt. Evans Road in the Arapahoe-Roosevelt National Forest, were both closed over Memorial Day weekend (their traditional opening day).  Besides the recreation and tourism industry, other professionals such as utility managers, city planners, emergency respondents, water companies, and farmers all depend on snowpack and river levels.  While Coloradoans will continue to experience wild weather, these two websites can help us all understand and react to it.


Corona Summer Camp 2015: AAPOR

Ah, summer camp. For those of us who were generally allergic to the outdoors as kids, summer camp did not necessarily mean cabins, building fires, and outdoor recreation. In my case, summer camp usually meant summer orchestra, which was the best. Not only did I get to play music for hours every day, I also got to spend time with a bunch of other orchestra kids who would not even bat an eye if I wanted to discuss which Suzuki book they were on or the pros and cons of catgut strings.

In many ways, AAPOR feels similar to those years of summer orchestra—conference attendees discuss sampling issues and response rates as if they were discussing the weather. This year over one thousand people met at AAPOR to discuss all the intricacies of public opinion research. Beth and I had a really difficult time choosing which talks to go to because they all sounded so relevant and interesting.

This conference in particular gives us a good sense of the changing landscape of survey research and of the big issues in the field. One big issue was the mode to use for survey research. Phone surveys have been the gold standard for opinion research for a while, but with the increasing rate of cell phone use (especially cell phone only households) and the decreasing response rates, it has become harder and more expensive to maintain phone survey quality. There was at least one presentation about a survey that has transitioned over to cell phone only. Other talks discussed the quality of online panel surveys or using multiple survey modes (e.g., a paper survey with phone, an online survey with paper, etc.) to increase response rates.

Another big issue was incorporating all the available data into public opinion research. New research has started looking at whether social media and other big datasets can be analyzed as another way of measuring public opinion.  A final big issue had to do with transparency in survey research. Although there is a ton of survey data floating around, it is not always clear who is following correct and ethical survey methodology standards and who is not. Moreover, a lot of survey data reports do not include enough information to even judge what standards are being used. AAPOR’s Transparency Initiative encourages organizations to include with public release of data certain survey methodology information that allows people to judge the quality of the data.

Beth and Kate w Nate Silver at AAPOR 2015On top of all the great discussions of survey methodologyy and of the future of public opinion research, Beth and I also got to meet (briefly) Nate Silver, who received an award from AAPOR this year. (Someone may have texted this photo to her mom.) After the awards reception, AAPOR held a casino night to raise money for student awards. Silver participated in the poker tournament and kindly posed for many photos. He also has posted his own thoughts  about some issues raised at the conference.


The Lifesavers Conference

As with any industry, it is important in market research to keep up with the latest thinking and practices by regularly attending workshops and conferences.  For this reason, members of the Corona staff can occasionally be found at conferences put on by the American Association of Public Opinion Research (AAPOR), Market Research Association (MRA), and American Marketing Association (AMA).  However, here at Corona, we have an additional challenge of keeping up not only on the latest and greatest research practices, but also on the issues most important to our clients.  For that reason, we also try and occasionally make an appearance at conferences focused on subject matters such as parks and recreation, traffic safety, and more.

Kevin Presenting @ Lifesavers Conference

In March of this year, Kevin and I made the trip to Chicago to participate in this year’s Lifesavers Conference.  This conference has been conducted annually for decades and brings together individuals from around the country whose jobs are dedicated to keeping Americans safe on our nation’s highways.  The minds present at these conferences have been instrumental at making changes both legislatively and in communications with the public to dramatically reduce traffic fatalities over the years, including laws aimed at requiring child safety seats and punishing drunk drivers, and communications aimed at increasing seat belt usage, reducing impaired driving, and more recently, reducing distracted driving.

The things we learned at this year’s conference were enlightening to say the least, and it would require a whole series of blog posts to cover them all.  However, a few highlights included:

  • Learning about how state traffic safety departments are effectively using social media to reach a new generation for whom traditional television advertising simply isn’t effective.
  • Learning about how small nonprofit organizations can conduct their own evaluations to make the case to funders that their work is making a true impact.
  • Understanding the efforts being made by law enforcement in Colorado and Washington to keep drivers safe in the age of legalized marijuana.

This year’s conference was particularly special for us, as Kevin had the opportunity to present the results of research we conducted with the Minnesota Office of Traffic Safety aimed at better understanding some of the characteristics of high-risk drivers (those who exhibit a combination of risky traffic behaviors, including drinking and driving, speeding, texting while driving, and not wearing a seat belt).  A few of the key findings of that research included:

  • High risk drivers tend to overestimate how common their behaviors are among their peers (drinking and driving isn’t near as common as one might think), overestimate their own driving ability (almost everyone believes they are “above average”), and underestimate the risk of their driving behaviors (those who speed regularly are considerably more likely to be in a crash than those who do not).
  • Those who text and drive know they shouldn’t and worry about being in an accident, but they do it anyway. (Another presentation at the conference suggested that texting and driving should be treated as an addition rather than a rational decision.)  On the other hand, those who speed regularly are relatively unlikely to think their behavior is a problem and are more worried about getting a ticket than being in a crash.

Overall, the conference was a great chance to catch up on some of the things going on in traffic safety and lend our own expertise as well, so we hope we have the opportunity to attend again in the future!

Total Trivia Crack Scores by Game and Duration – The final post is a series of blogs analyzing Trivia Crack

We looked at total Trivia Crack scores in our previous blog of this series, and now we’ll close up by delving a little bit deeper.  Because, you see, not all Trivia Crack games are the same.

In our methodology section (Blog 4 of this series), we made a comment that players in one-on-one games tend to have higher scores than players in challenges.  Let’s take a look at that comparison.  (These graphs are also mathematically smoothed to enhance readability.)

Challenges vs One on One Scores

We can see that Challenge gamers are far more likely to have scores below approximately 390, while One on One gamers are more likely to have scores in the 410 to 430 range, and far more likely to have scores in the 460 to 490 range.  Interestingly, you’re more likely to encounter the elite players with scores over 490 in Challenge games, which runs counter to the pattern.

Overall, the median score for Challenge gamers is 411, while the median score for One on One gamers is 434.  This is a pretty big difference, because it means that the average One on One gamer is scoring about 4 points higher in each of the six categories.

And as we close out our analysis of Trivia Crack, we’ll take one final look at stamina.  If you’ve made it through all nine blogs in this series, you’re probably interested in that.

In our fourth blog on methodology, we examined how scores change as people answer more questions.  Our competing theories were that more frequent players may have higher scores because they’re good at the game and having fun, or they may have lower scores because it’s hard to maintain consistent high scores over time.  In that blog, we showed that scores actually tend to be higher as players answer more questions, rising approximately 6 points (1 per category) for every 1,000 questions answered.  So our theory was proven that stronger players play more often.

But actually, both theories are true.  When we examine the score distribution, we find that players are much less likely to have elite scores as they answer more questions.

The three smoothed curves below show the distribution of scores of players who have answered differing numbers of questions per category, with the yellow bar showing the most frequent players and the gray bar showing the newer or less frequent players.  If you look at the far right side of the graph, you’ll note that we had no frequent players in our sample who scored more than a 495, while a small percentage of our newest players did.  While it may be a statistical fluke it would appear that it’s difficult to maintain an ultra-high score as one plays more and more games.

If that’s the case, though, why do we see average scores rise as more games are played?  The answer may lie in two places on the graph.  First, we can take a look at the middle (blue) bar of those who have answered between 1,000 and 2,000 questions per category.  These people are quite strong, with a notable percentage having scores exceeding 490.  And second, we can see in the most veteran players (yellow bar) that a goodly proportion are well above average, with scores ranging from 460 to 490.

Questions Answered per Category

And with that, we conclude our examination of the game of Trivia Crack.  If you want to challenge me, look me up at @kevin.802.562 and we’ll make a game of it.  But I have to warn you, we take the “Bright Thinking, Brilliant Guidance” motto seriously at Corona Insights, so be prepared for a battle.

Total Trivia Crack Scores – The eighth in a series of posts analyzing Trivia Crack

In our previous six Trivia Crack blogs, we’ve taken a look at each of the individual categories as well as talking a bit about the methodology of collecting and analyzing scores.  Now let’s move on to the capstone of our little Trivia Crack project, which is a look at total scores.

We calculated total scores by simply adding the scores of each of the categories together.  Here’s the distribution that we saw.  Note that this graph includes a smoothing algorithm or otherwise it would be incredibly jagged and hard to read.

Total Scores Graph

From the above graph, we see that it takes a score of 393 to rank about 25 percent of Trivia Crack players, and a total score of 425 to rank in the top half.  The top 10 percent of players are scoring 477 or higher, and if you want to be a 1 percenter you’re going to need a 505 or higher.  Good luck!

If you’d like to know exactly where you stand, you can check out the following table for exact scores.

Table of Exact Scores

Alright, it’s time for that final blog post. Let’s dig a bit deeper into these scores by game and duration.