Three Essential Panel Quality Checks

In GMI's latest email Newsletter, three essential panel quality checks are given: a handy guide for any user of online panel as a source of market research. In fact, the three checks are probably more comprehensive than using the more complete, yet bulky ESOMAR 25 questions ESOMAR has defined for online research buyers on panel quality. On the other hand, I think that GMI document is covering all the notional basic checks regarding panel / data quality, but I think I'll dedicate a futre post on that topic... Here are their three checks: CHECK 1: PANEL RECRUITMENT AND INCENTIVES

• Ensure minimum response rates. Your panel provider should require their recruitment partners to provide willing, active panelists who will secure a minimum response rate for your research studies.
• Pay only for good panelists. Your panel provider should not compensate their recruitment partners for fraudulent panelist registrations. They should have systems in place to track which recruiting partners provide fraudulent panelists, and strictly enforce contractual measures to demand a refund of the recruiting fees paid.
• Review panelist redemption files. Your panel provider should regularly create and thoroughly review panelist redemption files to detect any suspicious member account activity.
• Engage panelists with profiling surveys. Your panel provider should regularly revamp each of their profiling surveys, and provide an incentive to their panelists for completing each one. This not only contributes to keeping panel profiling information up-to-date for future research studies, but also helps continuously engage the panel.

CHECK 2: PANEL REGISTRATION

• Enforce panelist login restrictions. Your panel provider should require their panelists to log in using their email address, and create a strong password with a particular type of / minimum set of characters.
• Prevent panelists from changing personal information during registration. Your panel provider should not allow their panelists to change some of the personal information they provide during the registration process. This will prevent fraudulent panelists from creating accounts in multiple geographic regions with varying demographic attributes.
• Store and compare account creation and changes. Your panel provider should be in a position to record a snapshot of every account created upon registration, and store that information in a secure database. Subsequent account changes made by panelists to their personal information should prompt another snapshot, so information can be compared between the two steps.

CHECK 3: OVERALL PANEL HEALTH

• Strictly enforce panelist account removal policy. Your panel provider should decrease the rating of panel members who have incurred an infraction, and remove those fraudulent panelists from the panel after a set number of infractions. Block fraudulent panelists without warning. Your panel provider should delete those panelists who act in an egregious manner from the panel, and block them from opening a subsequent account.
• Remove suspicious email addresses, domains and IPs. Your panel provider should permanently block a panelist suspected to use multiple email addresses, domains or IPs from accessing the panel.
• Detect speedsters. Your panel provider should have checks in place to filter out responses provided by speedsters – panelists who complete a survey faster than the established minimum time threshold.
• Keep bots and scripts at bay. Your panel provider should have technology solutions in place to prevent bots, scripts or other programs from creating or editing panelist accounts. This technology should also be used during account registration, account login and before a panelist submits changes to the personal information stored in their account.