We get a lot of inquiries about how to join our panel or participate in our focus groups, and consequently we spend a lot of time explaining that we don’t maintain this kind of recruiting list for participants. (We custom recruit for almost all our groups. We’ll explain why below.) Some questions come from people who have just participated in research for the first time and are shocked that it wasn’t a scam, and enjoyed being paid to share their thoughts. Others are old hands at research participation and consider it a form of employment. Hopefully this post will help both potential participants and those who commission and conduct market research understand why recruiting from a list of interested parties is undesirable.
In a quick search, I easily found several sites helping people get on a list (here and here). I personally like the photos of happy people in the video and the association of market research with telemarketers in the last link…arghh! For another 2.1 million links for paid surveys, click here.
Now, I’m not against the entrepreneurial spirit of earning a buck, but professional respondents are not good for our industry and therefore not good for our clients. Especially when they start to misrepresent themselves in order to participate. WHY? Completing the survey (or other research mode) and earning the incentive becomes their only objective. The problem is only compounded when respondents outright lie to qualify for the survey. And as respondents take more and more surveys, they become more skilled in learning how to ensure they’ll make it past the screener questions and qualify. Additionally, there is potential concern that respondents who become too skilled at taking surveys – even when not cheating – may not give quality responses due to lack of focus. Or they may just become “tuned” to marketing in their everyday lives, and in a sense be too sophisticated to represent the “average person” targeted by the marketing campaign.
This goes back to the core tenet of survey research – sampling. If your sample is not representative of your target population, then accurate conclusions cannot be drawn. The only group that cheaters represent are cheaters themselves (and even then they wouldn’t fill out the survey correctly!). Even when not cheating, similar problems arise from using “professional participants” who are not representative because they’ve become “experts” at awareness of marketing.
And this isn’t just some methodology-obsessed research firm speaking either. Big companies are having concerns too. A recent BusinessWeek article on the quality of online polling noted that P&G is enforcing stricter guidelines when conducting Web polling. The article cites one instance in which two different surveys came to two completely different results regarding the attractiveness of a product.
So what can we do? Stay tuned for an upcoming post on ways to limit professional respondents, cheaters, or respondents who are just plain lazy.
For those of you in our area – and especially if you commute using RTD’s Eco Pass – you may have heard about RTD’s proposed plan to eliminate the pass for small businesses. (As background for those of you not acquainted with Denver Transit, RTD is the local transit authority and the Eco Pass is a pass employers can purchase for their employees.)
This issue has been causing quite the buzz lately and Corona thought it would do its own analysis to see if it really made sense and what the impacts really are (hey, its what we do). To check our analysis out as well as other information on the topic visit the ecopassforum. Educate yourself on the issue and then make your voice heard.
(Disclaimer: Corona has fewer than 50 employees and provides the Eco Pass to them as a benefit)
Our offices are located just a stone’s throw from the convention center here in Denver (not that we’ll be throwing any stones due to security), and as such, we were right in the middle of it all. So we thought we would share our own [fun] observations. (We’ll leave the political commentary for everyone else.)
- The city has definitely seemed prepared and at least from our vantage point, everything seemed to go as smooth as could be expected with such an event.
- Denverites really seemed to take it in. We were essentially tourists in our own city checking out the police in their body armor like a tourist would check out the guards outside of Buckingham Palace in London. Or crowding around protests like street performers on Pier 39 in San Francisco. Nothing attracts a crowd like a crowd.
- People were also chipping in to help. Giving directions, recommendations and tips seemed standard. Our own analyst, Geoff, had already assisted several delegates and news reporters with directions by only 8am on the first day of the convention. We didn’t even need an ad campaign to make us do it.
- Just like any area with crowds, there were street vendors. Buttons by the thousands, t-shirts, bobble-head dolls.
- In addition to the spectacle on the street, there was also great art to be seen.
- The crowds were so big along the 16th Street Mall that the buses could hardly get by. It was probably quicker to walk, but with such great weather, who would mind?
- With all the celebrities, it felt like we were on the west coast.
- But it felt decidedly dressier too; don’t they know Denver has a casual dress policy?
- And our local shoe shine guy, Claude, outside our office kept everyone looking good – he even brought out the big shoe shine chair.
And of course the security. Wow. You knew it would be high, but who knew there were even this many law enforcement officials in the area?
- Wondering what hotels the “important” people were staying at? Just look for the uniformed officers on the corners surrounding it, or the police vehicles lined up for a quick evac. Of course, some people missed this.
- Don’t leave your backpack laying around for even a second.
- Even the horses had riot gear.
- The police were filming the protestors, the protestors were filming the police, and the tourists (or just locals) were filming both.
Our CEO, Kevin Raines has been chosen to present on the topic of Do It Yourself Survey Results for the Denver Metro Chamber of Commerce and the Denver Metro Small Business Development Center at the Business Excellence Forum. His presentation will explore:
- The advantages and disadvantages of survey research;
- The pros and cons of different types of surveys;
- How to avoid the most common surveying mistakes; and
- Ten simple steps for conducting an effective survey.
From The Denver Metro Chamber of Commerce:
The Denver Metro Chamber of Commerce and the Denver Metro Small Business Development Center have assembled 12 of the state’s top business experts to help you meet today’s business challenges. This is a unique opportunity to gain a wide variety of expertise all in one day! Choose 4 of 12 powerful workshops, all designed to help your business grow during these turbulent times.
The forum will be held at Regis University on Wednesday, August 20 from 8 am to 5 pm.
The survey we recently completed for the Denver Office of Cultural Affairs has received two nice write ups in the local press after a great public presentation of the results by DOCA director Dr. Erin Trapp.
This extensive survey of Denver residents consisted of 814 interviews with residents, including 205 with self-identified African Americans and 204 with self-identified Latino Denverites. The final survey answers for the entire city were demographically weighted to ensure they are representative of the population of Denver.
The results show both positives and negatives for the performing arts in Denver. Primary among the positives is that 80 percent of residents are interested in live performing arts performances and a good number actually attend them, as within the past year 58 percent attended live theater, 41 percent went to a festival, 34 percent saw a live musical concert, and 11 percent attended a dance performance.
To read more about the results and their implications for performing arts in Denver click over to the Denver Post article* or to the article in the Rocky Mountain News. In addition, DOCA has released selected findings from the survey into a report available on the Denver City website.
*The Denver Post article begins “You can’t always trust surveys commissioned by people with a vested interest in the results.” We completely agree! When consuming data and survey results, you always need to be aware of who commissioned the research, who completed the research, and how they carried it out. And when you conduct research, this is why it is important to have someone (like us!) who is aggressively neutral, ethically unimpeachable, and methodologically sound.