A fresh start with your customer feedback

Spring is in the air – though if you’re from our hometown it’s arguably been in the air since January – and what better season is there for a fresh start?  As we often advocate with our clients, a periodic review of your research, evaluation, or strategic plans can help keep them relevant and maximize the benefits gained.

Customer feedback, whether from a formal voice of the customer program or through informal feedback, is one area we have seen that can grow stale without regular attention.

So, to help you maximize your customer feedback program this year, we offer our spring cleaning checklist:

Fresh Start for Customer Satisfaction
Click to enlarge.


  • Make the most of the data you have.  Look at your current results differently, perhaps by analyzing new segments or trend data.
  • Link up multiple data sources.  Link up customer feedback to actual behavior via account data. Capturing actual behavior is always better than asking the customer about it. Similarly, consider outside data sources that can be overlaid on your customer feedback, such as economic information or even weather (weather, for instance, can be a big driver in retail sales).
  • Add fresh questions.  New questions can lead to fresh insights. Update your survey questions based on previous findings – what else do you want to learn?
  • Conduct a driver analysis.  We often capture feedback on many attributes, but which ones actually matter?
  • Follow-up directly. In our world of collecting and reporting on customer satisfaction we often focus on metrics and big picture areas that need attention.  However, don’t forget that sometimes the best thing you can do is to reach out to your customers personally. You’ll need to be up front with them as to whether their feedback is anonymous or not (otherwise you may have an even more upset customer). Some issues are very specific to the individual and can only be addressed one-on-one. Create a system that can identify and route these customers to the right contact for personal attention. (In fact, with hotels, a recent study found that responding to online reviews boosted their bottom line).

It takes work, but at least it’s not cleaning out the garage.

Be sure to check out our previous posts as well on 4 ways to summarize customer feedback results and the different types of customer satisfaction surveys.

It’s All About How You Ask

Last month, I attended the RIVA Training Institute in Rockville, Maryland to get a fresh look at how Corona tackles qualitative research.  It was heartening to find that our qualitative research practices match industry standards, but as with any good training, I came away with some new insights that will undoubtedly help make us better researchers.

One thing in particular that stood out to me was taking a look at how we phrase questions during focus groups and interviews.  Perhaps you have been in a situation where you feel like you’re asking a really valuable question, but aren’t getting comprehensive responses.  Why?  Well, the answer is actually in the question.  The problem may, in fact, be the word, “why”? WHY?

Of course, qualitative research is all about uncovering answers to “why” questions, but using the word “why” may actually prevent you from getting the answers you need.  The word “why” tends to put people on the defensive.  As an example, think back to when you were a teenager, and you were late for your curfew.  What’s the first thing your mom asked?  “Why are you late? Why didn’t you call? Why don’t you show some respect?”  You’re put on the spot and immediately feel the need to defend yourself.  In general, there are fairly negative connotations with the word “why”, so although it might be somewhat subconscious, people may be less likely to open up and give deeper responses if they’re asked to explain themselves in this way.

Of course, we still need to know the answer to these “why” questions, but using a different word choice will likely elicit more responses that are below top-of-mind.  Consider using phrases such as, “What are some reasons…” or “What influenced you to…” as alternatives.   It sounds simple, but it works wonders!  So, next time you ask a “why” question, but don’t get the answers you need, consider rephrasing.  You may be surprised to find it will not only help you get better answers as researcher, but also in other aspects of your life.

Corona Insights is hiring

Corona Insights is looking for a bright mind to join our team. Specifically, we’re looking for an Associate with a quantitative market research focus. join our team

If you love asking and answering questions, finding insights buried in data, and have a desire to help organizations with their challenges, then we want to hear from you.

View job descriptions and apply here.

Is tomorrow the first real Pi day?

From Wikipedia
From Wikipedia

Happy Pi Day! As you have undoubtedly noticed thanks to pictures and references to the culinary type of pie in your social media streams, tomorrow is March 14, or 3.14 the (rounded) mathematical constant representing the ratio of a circle’s circumference to its diameter.

But wait, there’s more!  Not only is tomorrow 3.14, it’s 3.14.15, which is even more auspicious, as the next two decimals after 3.14 in the ratio are “15”.  A once in a lifetime event for mathematical nerds for sure.

This got us thinking though, was there ever an even crazier (relatively speaking) celebration possible?  What about 3/14/1592?

The answer –

First, was pi even a “thing” in 1592?

That one is easy – Yes.

The earliest approximations of pi date to Egyptian times (though the actual figure was not as accurate in Egyptian times so Egyptians may have been celebrating on a different day).

Second, was pi 3.141592 in 1592?

The answer again – yes.  Though more of a meandering yes.

Around 480 AD, a Chinese mathematician correctly calculated the first 7 decimals, which would be cause for celebration nearly 1100 years later in 1592, but others after him actually calculated it less accurately.  Around the 15th century, the calculations switched from a geometrical technique to infinite series techniques which allowed pi to be computed with a lot greater precision. Long story short, 3.141592 was “real” in 1592.

But what about pi itself.  Was it called pi in 1592?

Because without pi, we would have no pi day. The answer – no.  While pi as a mathematical constant had been around for centuries, it wasn’t actually represented by the pi symbol until the early 1700s.


In reality, without social media to spread the pi love, probably no one outside a small group of mathematicians would have ever celebrated.  But now it’s a “thing.” So tomorrow, celebrate pi and eat some too.

Reference: History of Pi on Wikipedia.

Fresh Evaluation Ideas for Spring

As an evaluator, it’s really easy to draft a thousand lines of questioning to capture every nuance in every conceivable outcome that might result from a particular program.  I want to know everything, and I want to understand everything deeply, and so do the organizations I work with.

DataYet collecting too much data burdens both program participants and the evaluation team, (and in some cases can change how participants respond to particular items – see details here). The hard part of evaluation work is distilling the goals down to their essence and choosing the highest impact measures.

This is especially important in situations where frequent measures are needed because services are adapting to meet changing needs.  A recent NPR story illustrates this, also showing how technology makes continuous monitoring possible.  The story points out that organizations providing aid in disasters often decide what to do at the outset, and then don’t have much information about how it went until they evaluate it formally after the program ends.  But in a few recent cases, including the international response to the Ebola epidemic, a group has implemented a short survey administered weekly by text messages to cell phones of residents in affected communities.  The survey measures only around five key objectives of the Ebola response (e.g., impacts on travel, trust in communications, etc.), and because it is implemented weekly, it provides ongoing updates on progress toward the desired objectives.  The data helps steer the program activities to best meet the most pressing needs.

It is both useful and inspiring to learn about thoughtful, creative solutions that other evaluators have developed to help organizations reach their goals.  We’re always looking to learn and grow so that we can best serve organizations who are themselves continuously improving as they work to make the world a better place.


Brand Tracking for Sustained Growth

Surveys can be used to guide a plethora of business decisions.  If you’re considering launching a new product or service, a survey can help you get feedback on not only the product or service itself, but your messaging and collaterals as well.  If you’re looking to grow into a new market, a survey can help you understand the lay of the land so that you can maximize the return on your marketing investments.   Or maybe you just haven’t checked in with your customers in a while and want to know how they feel; a survey can be an effective tool for that as well.

However, as any good strategic planner will tell you, businesses looking to grow and develop long-term sustainability need to set their sights higher and focus on the long-term goals of their organizations, as well as the strategies and tactics that it will take to get there.  In these situations, an ongoing brand tracking survey can be a very effective tool to help guide an organization’s decisions as it grows.

Brand Tracking for Sustained Growth

Brand tracking studies are typically conducted on a regular basis.  (Once a year is common, but an individual organization’s needs could suggest that either more frequent surveys or less frequent surveys would be most effective.)  Brand tracking studies focus on measuring an individual brand in terms of a variety of key performance indicators, such as:

  • Brand awareness
  • Past purchase
  • Current ownership or use
  • Brand favorability
  • Future purchase intent
  • Likelihood of recommending (Net Promoter Score)
  • Brand association with desired brand imagery

Using a one-time survey, you can effectively quantify how your brand is perming in all of these measures and, if necessary, take actions to improve your brand’s position.  For example, if awareness is low but other metrics are high, you may want to focus your marketing efforts on just getting your name out there.  If awareness is high, but favorability is low, it may be worth looking at your product and making sure it’s living up to your brand promise.  And if you’re missing the mark on key aspects of your brand that you are trying to convey in your marketing messages, it may be worth re-evaluating your advertising and collaterals to make sure they are as effective as you’d like them to be.

A one-time survey can also be effective at understanding how your brand’s performance measures up against some of your key competitors.  By using these competitors as benchmarks, you can understand where you’re ahead of the pack and where you could look to improve.

The real power in brand tracking surveys, however, lies in plotting how your key performance indicators change over time.  If you set a goal of increasing awareness of your brand by 10% after seeing it is low in your initial survey, a tracking survey can serve to see if your efforts are paying off.  Over the years, you can develop a deep understanding of how your brand’s position in the marketplace has changed over time and not only identify areas for ongoing improvement, but areas to celebrate your successes as well.

So the next time you’re wondering if you need a survey to help you understand what your customers think of your organization, consider making a long-term commitment to tracking your brand’s performance.  You may be surprised at the power brand tracking can give you while plotting and monitoring future growth.

Research To Do List

Recently, I came across this article about recommendations for the U.S. Census data. While I found the article interesting overall, I was really struck by this quote: “Meanwhile, the academy panel said, the job of producing more than 11 billion estimates a year from the survey is stretching the [U.S. Census] bureau’s capacity. The agency has never evaluated which data products are most useful. The report suggested that the bureau do such an evaluation and consider trimming the number of data tables it publishes” (added emphasis). Despite calculating many statistics that are used to evaluate different aspects of U.S. society, the Census apparently has not done much self-evaluation as an organization!

Although it might be easy to criticize the Census for failing to evaluate their organization, especially given how data-oriented they are, it also might be easy to sympathize with them. We’ve all had that year where we didn’t do enough initial planning and evaluation, so we ended up doing a ton of work because we weren’t sure what, if anything, was helping our organization grow. Well, the year is young, and with a little planning, it is possible to incorporate some performance and evaluation metrics for your organization into the year.

President of Nerds

At Corona, we not only help organizations evaluate and plan, we also help ourselves evaluate and plan. (Yes, our company is not just a research and consulting firm; we’re also a client of our own research and consulting firm.) So about a month ago, we had our annual retreat. One of the best parts of retreat is looking at the data we collected during the year and making plans for the upcoming year—something that we love to do both in our personal and professional lives. Below are some questions that Corona and maybe your organization might want to think about when planning for the year:

  • Are you collecting enough demographic information about your members, customers, donors, visitors, etc.? Maintaining good records of who supports your organization can be useful when deciding whether to target new markets, evaluating the success of different tactics used to attract those markets, assessing how your organization is or is not growing, etc.
  • Are you looking at the right types of internal metrics? Are there metrics beyond financials that you should be examining? Are the metrics that you currently collect helping you plan or answer questions? Or are they just sitting on graphs?
  • Although it is often useful to do a large survey of your members, customers, donors, visitors, etc., a shorter survey sent out more frequently may better serve your needs. Do you need to understand an issue in depth? If so, a longer survey might be better. Did you already investigate that issue last year and are now ready to engage in a series of steps to address it? If so, more frequent, shorter surveys might be better because they can track changes happening in response to actions that your organization takes.
  • Can you use old data to answer new questions? By applying a different analysis technique or segmenting your data in a different way, you may be able to answer a question without gathering completely new data.

It may already be March, but it’s not too late to think about where you want to be, as an organization, at the end of the year and how you will know if you have made it.

Is your Neighbor an Engineer?

While Kevin has an engineering degree, I do not—my degree is in social sciences.  After reading Kevin’s blogs about income patterns of folks with engineering degrees, I was inspired to take a fresh look at degrees from a spatial perspective. I wondered where engineers are most likely to live, where social scientists are likely to live, and is there is a relationship between them?  What better place to explore than our own back yard.

Since Kevin and I both like maps, I pulled some data from the American Community Survey into our mapping software to take a look. The universe of this data is all adults, 25 years or older, who have obtained a bachelor’s degree or higher.

Interestingly, in terms of raw numbers, the census tracts with the greatest number of engineering degrees are on the south to northwest outskirts of the Denver area, especially around the Boulder area (see map 1).  Maybe people with engineering degrees like to live near the foothills? Out of adults with a bachelor’s degree or higher in the Denver area, about eight percent have an engineering degree.

Adults with Engineering Degrees












However, some of these census tracts are rather large, so I looked at the density of people with engineering degrees by tract (see map 2). When we look at the number of engineering degrees per square mile, we start seeing dense pockets in the heart of Denver and Boulder, but still some good representation in the Southern suburbs.  In case you are curious, there is an average of 106 people with engineering degrees per square mile in Denver and its immediate suburbs (i.e., area within the blue box).

Density of Engineering Degrees












As I previously mentioned, I have a degree in social sciences, and I wanted to know if people with my degree were likely to live near people with engineering degrees? First, I mapped the number of social science degrees (see map 3). Around Denver, about nine percent of people with a bachelor’s degree have one in social sciences.   By visually comparing Maps 1 and 3, I didn’t see any strong similarities in number of degrees by census tract. Social science degrees appear to be most numerous on the southeast and east side of Denver, with some in Boulder too.

Adults with Social Science Degrees












What about density of social science degrees?  There is an average of 139 people with social science degrees per square mile in the Denver area (see map 4). We see some dense areas near downtown Denver, and I can visually start to pick out similarities between densities on maps 2 and 4. Remarkably, the census tract with the greatest density of social science degrees is just east of the Capital building.  Considering this census tract is just a few blocks from our office, we seem to have a nice supply of potential labor nearby.

Density of Social Science Degrees










So what does this tell us?  We know that, on average, there is a slightly greater proportion of people with social science degrees than engineering degrees (i.e., 9% vs. 8%), and there is a greater average density of social scientists than engineers.  I guess engineers like room to spread their elbows and social scientists like to live near other people.

Are engineers likely to be neighbors with social scientists?   When analyzed by raw number, there is a positive correlation between the two degrees. For about every one step up in the number of social science degrees by census tract, there is about a half-step up in the number of engineering degrees.

Does density play a role?  It appears so.  When looking at the relationship between degrees based on density (i.e., number of degrees per square mile), we see that the correlation is stronger than the correlation based on raw numbers of degrees.  This means social scientists and engineers are more likely to live in the same census tract in dense urban areas than in rural areas.

Now that I’ve answered this question, its on to my next project where I aim to prove that my proximity to a doughnut shop has a positive and strong correlation with my personal happiness.

The Wages of Engineering Grads

In two previous blogs (Once an Engineer, Always an Engineer? & The Career Evolution of Engineers),

we examined the long-term occupational patterns of people with engineering degrees, by looking at a sample of 1,250 Colorado residents who hold bachelor’s degrees in engineering.

We saw that many people with engineering degrees do not actually become working engineers.  In fact, only about half of engineering degree holders are working in technical fields.  So the question arises of whether they sacrifice income to do so, or whether they thrive in other occupations.

We conducted an analysis of pay by occupation and degree type (engineer/non-engineer) to explore this issue.  In order to develop as close a comparison as possible, we considered only full-time workers (40+ hours per week) in Colorado who reported making more than minimum wage and who hold a bachelor’s degree or higher.  This eliminates workers in family businesses, entrepreneurial startups, retired people, and workers who choose to work less than full-time.  By limiting the analysis to people who hold college degrees,  we also provide a comparison of the value of an engineering degree against other degrees in fields other than engineering.

So we can now address the question of “Is an engineering degree portable if a graduate decides not to pursue engineering as an occupation?”

The following exhibit shows the mean wage of full-time degreed workers in different major occupational categories.  Partway down the table, we see that full-time engineering grads working in engineering occupations have a mean income of slightly less than $106,000 per year.  We can draw several other conclusions by comparing this figure to other occupations.

While the available data are very limited in numbers in many cases, we can still see patterns.

  • First, engineering grads make similar incomes or even higher in several other fields. Engineers who have gone into business, computers, construction, legal fields, management, medical fields, personal services, and sales have managed to secure similar incomes as they would have in engineering.  So in some cases there’s no financial sacrifice to leave engineering for another career.
  • Second, when engineering grads go into lower-paying fields, they tend to out-earn their coworkers who hold other types of degrees. This is particularly intriguing, because one would theorize that engineers are not receiving specialized training for those fields, while at least some other degree holders are.
  • Third, there are some fields where engineering degrees are not portable, but for the most part these are field where college degrees do not add as much value. Cleaning, protection (security), and food service are the three fields where the engineering graduates actually earned less than their other degreed peers.
  • Fourth, we see a large body of people who list their professions as engineers, and yet do not hold engineering degrees (or at least not bachelor’s degrees). These people have notably lower wages than those who hold engineering degrees.  Given the specificity of the field, it’s possible that this is an indicator of other occupations co-opting the term “engineer” when they aren’t engineers in a technical sense.

So what conclusions can we draw from this?  For people like myself who have left engineering careers, and for others who are considering leaving the field, the results are heartening.  We can conclude that engineering degrees are highly portable to other fields.  In many fields, an engineering grad can completely replace his or her engineering income, leveraging those skills in new arenas.  And even if an engineering grad decides to pursue a field that pays less than engineering does, the degree itself is still a good thing, rewarding them with higher earnings compared to our peers in the new profession.

Overall, it appears that you can’t really go wrong with an engineering degree, so I’ll continue to display my diploma proudly.

The career evolution of engineers

In a previous blog, I examined the long-term occupational patterns of people with engineering degrees.  Looking at a sample of 1,250 Colorado residents who hold bachelor’s degrees in engineering, I examined their current occupations.

Now, one might expect that career choices might grow more diverse with age.  My theory before examining the data was that younger engineering grads would be more likely to work in classic engineering jobs, while older grads are more likely to find other opportunities as their interests evolve and they become exposed to other opportunities.

Annnnd, my theory was wrong.  When we examine the proportion of jobs held by engineering graduates of different ages, we don’t see a strong pattern.  Regardless of age, roughly half of engineering grads are working directly in technical fields.

Technical Occupation of CO Engineers by AgeNote  – Data includes only employed persons.

While employment in technical occupations doesn’t vary by age, there are some minor differences, as seen above.  Younger grads are more likely to be classic engineers, and haven’t worked up to management levels yet.  In fact, their profile is very similar to those age 50 to 64 if we assume that rises to management levels occur over time.

Those in the 65+ age category are similar to the next generation after them, but without a significant presence in computer fields.

Perhaps the most interesting group is the 30-44 age group.  They are more likely than any other age group to be in computer occupations, and less likely to be a classic engineer.  Is th is a function of their generation in particular, or will the younger engineers migrate toward computer occupations as they age?  Only time will tell.

Most Common Occupational Cat of CO Engineers by AgeNote:  Figures include both working and non-working individuals.  Computer and manager occupations in this table do not differentiate between technical and non-technical occupations.

The above table shows the rise to management among some engineering grads, and also movement into sales for those in the middle age categories, as well as the bulge in computer workers in the 30 to 44 group.  Among the Under 30 group, we see a significant portion not in the work force, which is primarily students going straight to graduate school, as well as teachers, mostly at the college level.