Blog

Beware: Not All Panel Providers Are Equal

By David Kidd, BPR

Panel providers are responsible for recruiting and managing panels of survey respondents. Whether in the North America, the UK, Australasia, Europe, or the Middle East, one thing is for certain…. they are not all equal in the quality of data they obtain.

Here are some reasons why panel providers are not all equal:

• Panel Quality. Panel providers vary in terms of the quality of the panels they maintain. Some providers may have stringent recruitment processes and continuously validate their panellists to ensure they meet specific demographic criteria and are engaged in providing accurate responses. On the other hand, some providers may have looser recruitment methods, leading to lower-quality panellists who may not be representative of the target population.

• Panel Size and Diversity. The size and diversity of a panel can significantly impact the reliability and representativeness of the data collected. A panel provider with a larger panel size and diverse pool of respondents can offer more robust insights across various demographic segments. Providers with smaller panels or limited diversity may result in limited sample sizes or biases in the data collected.

• Target Audience Coverage. Different panel providers specialize in different target audiences or industries. Some may focus on specific demographics or geographic regions, while others may have expertise in particular sectors like healthcare, technology, or consumer goods. Depending on the research objectives, certain panel providers may be better suited for specific target audiences or industries.

• Panellist Engagement and Response Rates. The level of engagement and response rates of panellists can vary between providers. Higher engagement and response rates indicate that panellists are more active, attentive, and willing to participate in surveys. Providers that invest in maintaining an engaged panel and have strategies to incentivise and retain panellists tend to yield better response rates and data quality.

• Survey Quality Control. Panel providers employ various measures to ensure survey quality control. This includes monitoring for “speeders” (respondents who complete surveys too quickly), “straight-liners” (respondents who provide consistent but possibly inaccurate responses), the use of “bots” to complete surveys and other fraudulent or low-quality behaviour. Providers with robust quality control mechanisms can offer more reliable and trustworthy data.

• Data Validation and Accuracy. Panel providers may differ in their data validation practices. Some providers validate responses through methods such as duplicate response checks, consistency checks and data verification, ensuring the accuracy and reliability of the data. Providers that prioritize data validation and implement rigorous checks can deliver more accurate insights.

At BPR, we not only continually stress test our panel providers to ensure the veracity of our clients’ data but we take the process one step further. We apply our own internal diagnostic procedures to further validate the data to guarantee the quality of the insights we present to our clients.