Hey honey, wanna help me with a big project?

Facilitating success over 15 years of love and consulting.

In my mind, the question with milestones is this, which one is significant enough to celebrate? First year, 10, 20 or 50 – no brainer. What about 15? Sometimes the mid-decade mark is an important one. That’s the case for me this year as I celebrate 15 years with my company, Corona Insights. Oh, lots has changed over those years. I’m no longer officing out of the spare bedroom with the double bed that was great for holding files and for taking naps. Today I can catnap in my comfy office chair. Oh, and then there is air conditioning instead of a swamp cooler. Plus I have more colleagues than the one who is forever first for me, Kevin Raines.

We don’t do payroll in-house anymore off the 1st Bank checkbook. Now we have a payroll service and a super talented office manager to handle all of that stuff. Like many a start-up there was no salary for the first six months.

Those first six months of 2000 were a bit like the austerity period when Kevin and I were saving to become homeowners in 1994. No frills, no fluff. Our old house would just have to wait a bit as we saved for future remodels, especially since Raines Manor also served as Corona world headquarters. Who knew that the habits we’d built as first-time home buyers, and later remodelers, would serve us well as we had to chart the financial course for our company?

I’ve had the joy and Now, Future, Pastpleasure of building my consulting practice from scratch since 2000. I’ve also had the joy and pleasure of helping to build a small business too. Of course, if I’m honest, there has been some pain too, maybe quite a bit depending on the day, month or year. 2010, in the depths of the recession, many a painful moment. Certain challenging consulting gigs required many hours of free therapy from fellow consultants and colleagues. But I get ahead of myself once again.

Over the next several blogs I will reflect on my early years as a Jane of all trades, back in the day when I staffed research projects. (Some of you will recall the observational seat belt research we conducted on Denver street corners.) Then I’ll focus on the clients and projects that served as the crucible for my consulting practice. Finally, I’ll touch on what keeps me fresh and motivated today.

In the meantime it is interesting to note how much 2015 is like 2000. The nonprofit strategic consulting market is as fragmented today as it was then, possibly even more so. It remains largely populated by individual practitioners and smaller firms, many of which are still going today, although a few of my consultant peers have left the field to pursue other dreams and new folks have come into the market.

Business is still largely derived from repeat and referral sources. Friends and colleagues mattered then and matter now. What also matters is deep passion and love for my craft and the people who have helped me master it. There are too many of you to mention by name – and I thank you from the bottom of my heart.

Like they did in 2000, 2005 and 2010, prospective clients often begin their call or email with “I’d like to speak with you about facilitating our strategic planning process.”

Fortunately, some things never change.

I remain deeply indebted to my beloved who said, “Hey, I won a big project. Do you wanna help?” Who knew love and consulting would go so well together?


Adventures in Book Club: A Research Story

As it finally starts to feel like summer, one of my favorite activities is sitting outside on the patio reading a good book.  My fellow Coronerds must agree with this sentiment as we recently kicked off summer with our very own Corona Book Club, fearlessly led by Senior Associate, Matt Bruce.  As we started discussing this quarter’s book, I got to thinking that the ingredients for a successful book club really mirror the ingredients for a successful research project.

Coronerd Book Club

Those of you who may have participated in a book club on your own personal time may have noticed that some book clubs thrive, while others flop.  I would argue that success starts with clearly defining a purpose.  Is the purpose of the book club to socialize, to learn new writing techniques or to learn about a specific topic?  If all members of the club don’t agree on a purpose, it probably won’t be successful.   Imagine the chaos that may ensue if half of the members haven’t read the book because they came just to socialize, while the other half planned to have an in-depth discussion about literary techniques.  If you don’t have a clear purpose for the book club, you won’t be able to agree on a book or have a meaningful discussion.

The same is true for research projects.  We’ve talked a lot in the past about beginning with the end in mind—that is, being clear about why the research is being conducted in the first place, and how it will be used once complete.  When our clients are clear about the purpose behind the research project, and all stakeholders are on the same page, we are better able to guide the research by creating research goals, asking the right questions and using appropriate analysis techniques so that the research is actionable.

Wondering about the purpose of Corona’s book club?  It’s to continually learn about ways to improve our processes to make our work more useful to our clients, of course!

Tax Burden by County in Colorado

We saw an article in the Durango Herald the other day noting that La Plata County has the fourth-lowest tax burden in the state, and it made us curious.  We found the web site where the figures were posted.

The study used assumptions about a typical household, and calculated its cost in state income tax, property tax, sales tax, and fuel tax.  Because Colorado has a flat-rate income tax, that’s a wash throughout the state.

If one examines only the other three costs, the lowest tax for this hypothetical household is in Las Animas County at $1,337.  The next lowest rates are found in Gilpin County ($1,370), San Miguel County ($1,467), La Plata County ($1,676), and Gunnison County ($1,777).  The median among the 58 counties measured was $2,292.  At the bottom, the most heavily taxed Colorado residents live in Adams County, where the same household will pay $3,635, almost 60 percent above the median and nearly triple the Las Animas rate.  The difference from the median in Adams County is almost exclusively due to higher property taxes.

Tracking H2O

As a skiing and a river rafting enthusiast, I’m interested in snow levels of our nearby Rocky Mountains and of water levels in our creeks and rivers.  Early summer is therefore an exciting time to monitor snowpack and river levels because changes are dramatic. Luckily, two websites provide current data on snowpack and river levels that illustrate data in interesting ways.

To understand snowpack levels, I surf over to the National Resource Conservation Service (NRCS) SNOTEL website, which tracks snowpack across eight watersheds in Colorado. This website is easy to use and the data is updated about every four to six days.

Since I live in the South Platte River Basin, I frequently check out our time series snowpack summary, which displays snowpack estimates for the current year as well as the previous three years. As you can see below, the average and median (called normal) snowpack levels (red lines) are close to identical.  Looking at these lines, you can see that May 1st is a typical “tipping point” that leads to a downward melting trend.

We bucked the trend in 2015.  This year (dark blue line) was pretty close to normal until March when we went through a six-week dry spell. Storms and cool weather in April and May helped pull the snowpack upwards until we peaked on May 24, which is about a month after our normal peak day.  Although we are now sliding into snowmelt season, you can see that our current snowpack, as a percent, is 309% greater than a normal year.

South Platte River Bason Time Series Snowpack Summary Graph

After looking at our snowpack levels, I frequently cruise over to the United States Geological Survey (USGS) Water Information System website, which tracks real-time water levels in rivers and creeks nationwide.  Their top-level map depicts river and creek water levels as a percentile of that day over a 30-year average.  Just as any good data-driven exhibit should do, their national map makes it easy to see major trends and patterns. Rivers in the Pacific Coast states are very low, with many running below the 10th percentile.  The Great Plains states have experienced a very wet spring and early summer, and rivers here are generally running very high compared to normal.  The Appalachia and Atlantic Coast rivers are normal or low.

River Levels Map

Zooming into the Denver area, we see our river levels are very high, with most above the 90th percentile.  By clicking on a dot (which represent water level gauges), you can get information about a specific sites.  For example, the chart below shows water levels of the South Platte River in Englewood, CO, a suburb of Denver. You can see two steep spikes on June 5th, one in the early morning and one in the evening, that were caused by strong storms. Evidence of the first storm was obvious to the residents in Southeast Denver, who woke up that morning to the sound of front-end loaders moving four feet of hail off their streets.

South Platte River Levels_June

While the June 5th spikes are related to localized rainstorms, the melting snowpack is also influencing water levels.  If we extend the graph from seven days to 60 days, we see that water levels have consistently climbed since early April, due to spring rainstorms and melting snowpack.  The spikes of June 5th, which looked so dramatic in the top graph, become just one of many steep spikes the river has seen in the past couple of months.

South Platte River Levels_Spring

So who cares about all this anyway?  Well, you might care if you operate a tourism business in Colorado.  For example, organizers of running races scheduled to take place in the high mountains changed their routes due to snow covered trails, and two popular driving roads, Trail Ridge Road in Rocky Mountain National Park and Mt. Evans Road in the Arapahoe-Roosevelt National Forest, were both closed over Memorial Day weekend (their traditional opening day).  Besides the recreation and tourism industry, other professionals such as utility managers, city planners, emergency respondents, water companies, and farmers all depend on snowpack and river levels.  While Coloradoans will continue to experience wild weather, these two websites can help us all understand and react to it.


Corona Summer Camp 2015: AAPOR

Ah, summer camp. For those of us who were generally allergic to the outdoors as kids, summer camp did not necessarily mean cabins, building fires, and outdoor recreation. In my case, summer camp usually meant summer orchestra, which was the best. Not only did I get to play music for hours every day, I also got to spend time with a bunch of other orchestra kids who would not even bat an eye if I wanted to discuss which Suzuki book they were on or the pros and cons of catgut strings.

In many ways, AAPOR feels similar to those years of summer orchestra—conference attendees discuss sampling issues and response rates as if they were discussing the weather. This year over one thousand people met at AAPOR to discuss all the intricacies of public opinion research. Beth and I had a really difficult time choosing which talks to go to because they all sounded so relevant and interesting.

This conference in particular gives us a good sense of the changing landscape of survey research and of the big issues in the field. One big issue was the mode to use for survey research. Phone surveys have been the gold standard for opinion research for a while, but with the increasing rate of cell phone use (especially cell phone only households) and the decreasing response rates, it has become harder and more expensive to maintain phone survey quality. There was at least one presentation about a survey that has transitioned over to cell phone only. Other talks discussed the quality of online panel surveys or using multiple survey modes (e.g., a paper survey with phone, an online survey with paper, etc.) to increase response rates.

Another big issue was incorporating all the available data into public opinion research. New research has started looking at whether social media and other big datasets can be analyzed as another way of measuring public opinion.  A final big issue had to do with transparency in survey research. Although there is a ton of survey data floating around, it is not always clear who is following correct and ethical survey methodology standards and who is not. Moreover, a lot of survey data reports do not include enough information to even judge what standards are being used. AAPOR’s Transparency Initiative encourages organizations to include with public release of data certain survey methodology information that allows people to judge the quality of the data.

Beth and Kate w Nate Silver at AAPOR 2015On top of all the great discussions of survey methodologyy and of the future of public opinion research, Beth and I also got to meet (briefly) Nate Silver, who received an award from AAPOR this year. (Someone may have texted this photo to her mom.) After the awards reception, AAPOR held a casino night to raise money for student awards. Silver participated in the poker tournament and kindly posed for many photos. He also has posted his own thoughts  about some issues raised at the conference.


The Lifesavers Conference

As with any industry, it is important in market research to keep up with the latest thinking and practices by regularly attending workshops and conferences.  For this reason, members of the Corona staff can occasionally be found at conferences put on by the American Association of Public Opinion Research (AAPOR), Market Research Association (MRA), and American Marketing Association (AMA).  However, here at Corona, we have an additional challenge of keeping up not only on the latest and greatest research practices, but also on the issues most important to our clients.  For that reason, we also try and occasionally make an appearance at conferences focused on subject matters such as parks and recreation, traffic safety, and more.

Kevin Presenting @ Lifesavers Conference

In March of this year, Kevin and I made the trip to Chicago to participate in this year’s Lifesavers Conference.  This conference has been conducted annually for decades and brings together individuals from around the country whose jobs are dedicated to keeping Americans safe on our nation’s highways.  The minds present at these conferences have been instrumental at making changes both legislatively and in communications with the public to dramatically reduce traffic fatalities over the years, including laws aimed at requiring child safety seats and punishing drunk drivers, and communications aimed at increasing seat belt usage, reducing impaired driving, and more recently, reducing distracted driving.

The things we learned at this year’s conference were enlightening to say the least, and it would require a whole series of blog posts to cover them all.  However, a few highlights included:

  • Learning about how state traffic safety departments are effectively using social media to reach a new generation for whom traditional television advertising simply isn’t effective.
  • Learning about how small nonprofit organizations can conduct their own evaluations to make the case to funders that their work is making a true impact.
  • Understanding the efforts being made by law enforcement in Colorado and Washington to keep drivers safe in the age of legalized marijuana.

This year’s conference was particularly special for us, as Kevin had the opportunity to present the results of research we conducted with the Minnesota Office of Traffic Safety aimed at better understanding some of the characteristics of high-risk drivers (those who exhibit a combination of risky traffic behaviors, including drinking and driving, speeding, texting while driving, and not wearing a seat belt).  A few of the key findings of that research included:

  • High risk drivers tend to overestimate how common their behaviors are among their peers (drinking and driving isn’t near as common as one might think), overestimate their own driving ability (almost everyone believes they are “above average”), and underestimate the risk of their driving behaviors (those who speed regularly are considerably more likely to be in a crash than those who do not).
  • Those who text and drive know they shouldn’t and worry about being in an accident, but they do it anyway. (Another presentation at the conference suggested that texting and driving should be treated as an addition rather than a rational decision.)  On the other hand, those who speed regularly are relatively unlikely to think their behavior is a problem and are more worried about getting a ticket than being in a crash.

Overall, the conference was a great chance to catch up on some of the things going on in traffic safety and lend our own expertise as well, so we hope we have the opportunity to attend again in the future!

Total Trivia Crack Scores by Game and Duration – The final post is a series of blogs analyzing Trivia Crack

We looked at total Trivia Crack scores in our previous blog of this series, and now we’ll close up by delving a little bit deeper.  Because, you see, not all Trivia Crack games are the same.

In our methodology section (Blog 4 of this series), we made a comment that players in one-on-one games tend to have higher scores than players in challenges.  Let’s take a look at that comparison.  (These graphs are also mathematically smoothed to enhance readability.)

Challenges vs One on One Scores

We can see that Challenge gamers are far more likely to have scores below approximately 390, while One on One gamers are more likely to have scores in the 410 to 430 range, and far more likely to have scores in the 460 to 490 range.  Interestingly, you’re more likely to encounter the elite players with scores over 490 in Challenge games, which runs counter to the pattern.

Overall, the median score for Challenge gamers is 411, while the median score for One on One gamers is 434.  This is a pretty big difference, because it means that the average One on One gamer is scoring about 4 points higher in each of the six categories.

And as we close out our analysis of Trivia Crack, we’ll take one final look at stamina.  If you’ve made it through all nine blogs in this series, you’re probably interested in that.

In our fourth blog on methodology, we examined how scores change as people answer more questions.  Our competing theories were that more frequent players may have higher scores because they’re good at the game and having fun, or they may have lower scores because it’s hard to maintain consistent high scores over time.  In that blog, we showed that scores actually tend to be higher as players answer more questions, rising approximately 6 points (1 per category) for every 1,000 questions answered.  So our theory was proven that stronger players play more often.

But actually, both theories are true.  When we examine the score distribution, we find that players are much less likely to have elite scores as they answer more questions.

The three smoothed curves below show the distribution of scores of players who have answered differing numbers of questions per category, with the yellow bar showing the most frequent players and the gray bar showing the newer or less frequent players.  If you look at the far right side of the graph, you’ll note that we had no frequent players in our sample who scored more than a 495, while a small percentage of our newest players did.  While it may be a statistical fluke it would appear that it’s difficult to maintain an ultra-high score as one plays more and more games.

If that’s the case, though, why do we see average scores rise as more games are played?  The answer may lie in two places on the graph.  First, we can take a look at the middle (blue) bar of those who have answered between 1,000 and 2,000 questions per category.  These people are quite strong, with a notable percentage having scores exceeding 490.  And second, we can see in the most veteran players (yellow bar) that a goodly proportion are well above average, with scores ranging from 460 to 490.

Questions Answered per Category

And with that, we conclude our examination of the game of Trivia Crack.  If you want to challenge me, look me up at @kevin.802.562 and we’ll make a game of it.  But I have to warn you, we take the “Bright Thinking, Brilliant Guidance” motto seriously at Corona Insights, so be prepared for a battle.

Total Trivia Crack Scores – The eighth in a series of posts analyzing Trivia Crack

In our previous six Trivia Crack blogs, we’ve taken a look at each of the individual categories as well as talking a bit about the methodology of collecting and analyzing scores.  Now let’s move on to the capstone of our little Trivia Crack project, which is a look at total scores.

We calculated total scores by simply adding the scores of each of the categories together.  Here’s the distribution that we saw.  Note that this graph includes a smoothing algorithm or otherwise it would be incredibly jagged and hard to read.

Total Scores Graph

From the above graph, we see that it takes a score of 393 to rank about 25 percent of Trivia Crack players, and a total score of 425 to rank in the top half.  The top 10 percent of players are scoring 477 or higher, and if you want to be a 1 percenter you’re going to need a 505 or higher.  Good luck!

If you’d like to know exactly where you stand, you can check out the following table for exact scores.

Table of Exact Scores

Alright, it’s time for that final blog post. Let’s dig a bit deeper into these scores by game and duration.

The Arts Category – The seventh in a series of posts analyzing Trivia Crack

Arts is our final individual Trivia Crack category to analyze, and the seventh blog in our series.  We’ll have two more blog posts after this where we look at total combined scores for all categories, so stay tuned!

Arts is the bane of many a player, and is far and away the toughest category.  Let’s take a look at scoring patterns in this realm of painting and literature.

As usual, we gathered and analyzed the scores of over 400 random Trivia Crack players by participating in both one and one and group challenges.  Here’s the distribution of scores that we developed.

Arts Category Scores

Arts is definitely the toughest category overall.  It only takes a score of 59 to beat 25 percent of scores, and a score of 66 will put you in the top half of Arts contestants.  This is much lower than we see in other areas.  A score of 77 will put you in the 90th percentile, which is also the lowest of all the categories.  At the top end of the scale, it only takes an 83 to land in the top 1 percent.  So a lot of people are scoring lower in Arts than in other categories, and relatively few people also have high scores.

So how do Arts scores compare to other categories?    On average, it’s players’ weak link of the six categories.  Arts is the top category for only about 1 percent of players, and it’s the worst category for a whopping 42 percent.  If Arts is your best category, you’re a rare species.

Arts Rank Compared to Other Categories

If you’re good at Arts, does that predict that you’re a good overall player?  And if you’re weak at Arts, are there other categories where you’re strong or weak?  We did a correlation to test that analysis.  We won’t reveal our analysis of total scores until our next blog, but we can look at correlations.

First off, we should note that there’s a strong correlation between every category and total score.  We would expect that, because your total score is just the sum of all of your individual category scores.  However, there’s a stronger correlation in some categories than others, and Arts is a pretty strong predictor of overall score.  It has a Correlation factor of 0.899, where 0 means no relationship at all and 1 means that it’s a perfect predictor.  This is the third-strongest correlation among the six categories, which means that your Arts score is a very good indicator of your total score.

And what category most closely aligns with Arts?  Another simple correlation analysis shows that the scores that run most closely in parallel to Arts scoring is Geography scoring, but it doesn’t particularly stand out among most of the other categories.  What does stand out is the low correlation with sports scores.  I guess you should expect to see a lot of people in hockey jerseys at your local art museum.

Correlations of Category Scores

If you’d like to know exactly where you stand on Arts, here’s a chart that estimates the proportion of players who score at or above a particular score, based on our modeling.

Score Modeling

Now let’s finally take a look at the total scores. 

The History Category – The sixth in a series of posts analyzing Trivia Crack

History is our fifth Trivia Crack category to analyze, and the sixth blog in our series.  Let’s take a look at scoring patterns in this tough category.

We gathered and analyzed the scores of over 400 random Trivia Crack players by participating in both one and one and group challenges.  Here’s the distribution of scores that we developed.

History Category Scores

History is clearly a challenging category.  It only takes a score of 63 to beat 25 percent of scores, and a score of 70 will put you in the top half of Entertainment contestants, which is lower than the figures that we’ve seen in other categories so far.  But you need a 83 to break the 90th percentile, which is higher than most of the other categories we’ve seen so, and it takes an 89 to land in the top 1 percent.  So more people have low scores in history compared to most other categories, but more people also have high scores.  This category therefore sees a broader spread of scores than most categories.

So how does History compare to other categories?    On average, it’s a distant 5th.  History is seldom a player’s weakest category, but also seldom the strongest category, and is more likely to rank in their bottom three than their top three.

History Rank Compared to Other Categories

If you’re good at History, does that predict that you’re a good overall player?  And if you’re weak at History, are there other categories where you’re strong or weak?  We did a correlation to test that analysis.  We won’t reveal our analysis of total scores until later – you know, to build up suspense – but we can look at correlations.

First off, we should note that there’s a strong correlation between every category and total score.  We would expect that, because your total score is just the sum of all of your individual category scores.  However, there’s a stronger correlation in some categories than others, and History is a very strong predictor of overall score.  It has a Correlation factor of 0.928, where 0 means no relationship at all and 1 means that it’s a perfect predictor.  This is the second-strongest correlation among the six categories, almost as strong as Geography.  Your History score is therefore a very good indicator of your total score.

And what category most closely aligns with History?  Another simple correlation analysis shows that the scores that run most closely in parallel to History scoring is Geography scoring.  As we noted in our earlier Geography blog, you apparently learn a lot of geography when you learn history, or perhaps vice versa.  Arts also has a high correlation, and as we’ve seen with other categories, Entertainment is typically a different skill set.

Correlations of Category Scores

If you’d like to know exactly where you stand on History, here’s a chart that estimates the proportion of players who score at or above a particular score, based on our modeling.

Score Modeling

Art is up next, and it is our final individual Trivia Crack category to analyze.