This year SampleCon 2022 was again hosted at the beautiful Langham Hotel in Pasadena CA. The conference brought together a record number of attendees that included a diverse group of sample and market research professionals. It was pretty hard not to have an interesting conversation at any point during the conference, which had great speaking sessions and fun social events hosted conveniently on the property.
2022 State of the Sampling Industry
To kick off Day 1, the first session was the “General Session: SampleCon State of the Industry”, which included Industry leaders; Armen Adjemian, Ryan Barry, Katie Gross, Steve Schlesinger, and Sima Vasa. One very interesting point in this discussion was focused on the people that work in the MR industry. Specifically, the competition for people, paying appropriate wages, and how to acquire talent. The panel shared views on the health of the industry and what the future opportunities and challenges we would need to solve could look like. After attending this discussion, it was hard not to feel optimistic whether you are on the vendor or agency side!
Data Quality: Industry Perspective
Of course, with so many researchers and sample/data collection agencies in one place, the discussion of data quality is always a hot topic. This year, it seemed like a big theme was that our industry is lacking metrics on the sample side of data quality/collection. It was acknowledged that many companies are doing a good job individually in helping solve this issue, such as PureSpectrum’s PureScore™ and licensable products such as Research Defender, but it would need to take a larger collective effort to drive larger change. There was a call to action of needing to organize a committee from many of the thought leaders in the room, with Ben Coats raising his hand and calling out to the room in the “Fireside Chat with Tomorrow’s Industry Leaders” session to get the conversation started after the conference ended.
Data Quality: Respondent Perspective
I felt like one consistent theme around data quality was presenting itself, which is that respondents seem to be in one of two buckets. Group one is ever-evolving fraudsters, bots, and people trying to game the system in a multitude of ways. Group two is actual people that are either giving good or bad responses depending on the survey conditions. Which led us to what was probably the most interesting session of the week, “Straight From the Source: Live Respondent Conversation” which was an on-stage panel of actual respondents.
It was interesting to learn that respondents, aka “group two” are well aware of the “group one” data quality issues plaguing our industry. They see this in the number of red herring questions they have to answer, they know they are being passed off from panel to panel by the number of times they have to enter their basic demographics and all agreed that the fraud problem is affecting their survey taking experiences. They went into detail about panelist Facebook groups and Reddit groups where they discuss their experiences as a community. All panelists agreed that fraudulent behavior not only hurts the companies performing market research but that it is harming the panelists’ experience as well.
When panelists were asked about their survey participation experiences, the following feedback was provided, and to keep in context, the views are self-reported.
Incentives
- Incentives are too low and have not increased at all
- Fair incentives in cash, gift cards are ok but Crypto is not interesting
- Fast and immediate payout. Even one week is just way too long
Respondent Journey
- Be honest about the LOI
- Avoid asking the same questions over again
- Put the term points at the beginning of a survey
- Terming on over quotas at the end of a survey is an issue
- Shorter surveys are preferred, longer surveys are acceptable as long as it is interesting and the LOI expectation is accurate
Privacy
- Birthdate, panelists prefer you ask age and not specific birthdate
- Don’t ask for their pets name, even that is too personal
Some of these may seem obvious, and some of them might be surprising. In this list, there are many things that sample providers can work to correct with better recruitment, but also many things researchers can correct in the way a survey is scripted. In the “Fireside Chat with Tomorrow’s Industry Leaders”, Max Nanis seemed to feel that technology can solve many challenges we are facing and that there is still a long way to go in implementing technology. Why can’t incentives be handled differently to engage more respondents? Quota fulls can be handled by mapping sample traffic directly to surveys, so why is this still so limited to only a handful of use cases?
Looking Ahead to SampleCon 2023
Overall, SampleCon is a great conference to attend if you want to understand what is happening not just in sample, but overall with online data collection. Furthermore, if you want to have an impact on the industry and feel you want your voice to be heard, this is certainly a conference you should be attending. One important voice that continues to grow every year is the attendance of Market Research Agencies. Technology continues to evolve and many ideas need to come from not only the companies providing sample but from the people buying sample as well. Where we go from here until the next SampleCon is up to the attendees and the initiatives they put in place. This progress will also undoubtedly create a continued lively discussion for next year’s conference!
What are your thoughts? Connect with our team and let us know: