When Survey Answers Become Fraud Signals

Survey fraud prevention already relies on multiple layers of evidence, from device and network signals to in-survey quality checks and redemption review. Yet experienced bad actors can still get through those controls, especially when they know what traditional attention checks are looking for.

Frequent survey takers can recognize familiar checks, infer which screener answers are more likely to qualify, and provide polished responses while misrepresenting who or where they are. Frank Kelly, Market Research Practice Lead at Virtual Incentives, sees an important distinction here: conventional in-survey checks often expose inattention more readily than deliberate fraud. A careless legitimate participant may fail an obvious attention check that a practiced fraudster knows how to pass.

Virtual Incentives analyzed data from an ESOMAR project with C Space and found that certain profile claims, reward preferences, and willingness to complete identity-verification steps appeared far more often among respondents who failed existing pre-checks than among those who passed. If those patterns hold up in further testing, ordinary survey responses could help researchers identify a smaller group for manual review.

The findings are exploratory. “Failed pre-check” is a screening classification rather than independent confirmation of fraud in every case, and the groups were not randomly assigned. Recruitment differences or other composition effects may have contributed to some of the patterns. Even with those limitations, the size of the differences between the screening groups suggests significant potential for using certain survey and profile responses as referral signals for additional fraud checks or screening.

Where Traditional Attention Checks Fall Short

Attention checks still serve an important purpose because they can identify respondents who are moving too quickly, failing to read carefully, or providing responses that do not meet basic quality standards. Deliberate fraud creates a different problem. People who take large numbers of surveys may become highly familiar with reverse scales, repeated questions, instructional prompts, and other common techniques, making those checks easier to recognize and avoid.

Response-based signals approach the problem from another direction. Researchers can look for ordinary answers that appear disproportionately often among people already associated with elevated risk. The question still serves a legitimate research purpose, while the answer may also reveal how someone is presenting themselves, what they value, or which forms of verification they would prefer to avoid.

“One of the strongest indicators of fraud may be how respondents answer the questions themselves.”
— Frank Kelly, Market Research Practice Lead, Virtual Incentives

Frank’s working theory is that some of those choices may be shaped by what a bad actor is trying to optimize. A profile that appears more likely to qualify for hard-to-fill studies may be more attractive, while certain reward choices or reluctance to complete verification steps may fit the circumstances of someone participating fraudulently. The ESOMAR study did not test those motives, so they remain hypotheses. What it did show were unusually large differences in how the two screening groups answered.

The Pattern Extended Across the Questionnaire

The analysis pooled 213 unique completed responses in each group: one classified as passing fraud pre-checks at registration and one classified as failing them. The underlying questionnaire was a roughly 10-minute survey about participants’ experiences taking surveys and their reward preferences; it was not designed as a fraud-detection instrument. Across 70 tested survey-response comparisons, 38 differed at the conventional 95% confidence threshold, and 25 remained statistically significant after adjustment for multiple comparisons. The breadth of those differences suggested that the pattern extended beyond one or two isolated questions.

One of the clearest examples involved age. Among respondents in the failed pre-check group, 71.4% had a recorded age of 25–34, compared with 9.9% of those who passed. The age difference alone should not be treated as a risk signal, particularly because recruitment sources or sample composition may have influenced the result. More meaningful was the inconsistency within some respondents’ own information: 17.8% of the failed group reported an age band that did not align with their recorded birth year, compared with 1.4% of the passed group.

An even larger divide appeared in reward preference, with an Apple gift card selected as the preferred next reward by 51.6% of the failed group and just 1.4% of the passed group. Differences also emerged around verification. Respondents in the failed group were less likely to say they would share a mobile number, provide a government-issued ID image, or submit a selfie for identity verification.

None of those answers carries much meaning in isolation. Legitimate participants can prefer an Apple gift card, have valid reasons for declining to share personal information, or belong to a demographic group that happened to be overrepresented among failed respondents in this particular sample. The next question is whether several of these signals occurring together can identify a smaller group for additional review. Further research could also help identify a short set of ordinary questions that researchers could add to studies and test as additional fraud-risk signals.

The Strongest Separation Appeared When Signals Were Combined

Frank’s follow-up analysis, based on 204 flagged respondents, tested several combinations of the most pronounced response patterns. An Apple gift card preference alone appeared among 108 flagged respondents and three passed respondents, capturing 52.9% of the flagged group in the follow-up analysis.

Pairing Apple gift card preference with age 25–34 narrowed the match to 97 respondents, all of whom belonged to the failed group. The same reward preference combined with no stated willingness to provide a selfie matched 98 failed respondents and one passed respondent, while pairing it with no willingness to share a mobile number matched 75 failed respondents and no passed respondents.

The broadest practical rule combined Apple gift card preference with either age 25–34 or no mobile-number willingness. It matched 102 failed respondents, representing half of the flagged group in the analysis, without matching anyone in the passed group. Adding more conditions generally reduced coverage without improving the observed separation.

The results remain exploratory because the same sample was used to identify and evaluate the rules, and the “flagged” label reflects failed pre-check classification rather than independently confirmed fraud. Zero observed false positives in one dataset does not mean a rule will produce zero false positives elsewhere.

For now, the clearest use case is as a referral signal: a pattern that prompts additional review while leaving the final decision to a broader set of evidence. Its value will depend on whether the same relationships appear in new respondents and whether they predict independently reviewed fraud outcomes.

Use the Signal to Focus Review

Researchers already make judgment calls about which completes deserve confidence. During reconciliation, they may review failed attention checks, implausible answers, duplicate open ends, unusual timing, participation history, or other patterns before deciding which responses belong in the final dataset.

Response-based risk signals could help prioritize that work. Completes whose answers resemble patterns previously associated with elevated risk could be reviewed alongside device, account, and open-ended-response evidence.

The same approach may be useful in panel environments, where patterns can be evaluated over time instead of treated as one-study verdicts. Repeated signals across multiple interactions could justify additional review or selective identity verification, while a single unusual answer may carry very little weight.

Researchers still need to keep response quality, identity confidence, and fraud risk separate. A genuine participant can provide careless answers, a fraudulent participant can provide thoughtful ones, and someone with legitimate privacy concerns can decline identity verification. Any operational rule should therefore be judged by how much new information it adds, how many confirmed fraud cases it helps identify, and how many legitimate participants it sends for unnecessary review.

Connect Survey Evidence With Redemption Outcomes

A more complete validation process extends beyond the survey itself. When a credible fraud finding or elevated-risk outcome appears at redemption, a stable study-specific identifier can link it back to the corresponding survey record. Researchers can then compare those records with other completes and see which response patterns recur across studies.

Outcome labels need enough precision to avoid circular conclusions. Confirmed fraud, unresolved risk, identity-verification failure, abandonment, and no detected flag should remain distinct so that prospective testing measures the actual performance of the signal.

This feedback loop may be especially useful in higher-value B2B or healthcare research, where an individual incentive can be substantial. If several survey-side indicators point toward elevated risk, they could inform whether additional checks are warranted before the reward is released without turning any one answer into an automatic block.

Fraud Prevention Has to Keep Moving

“Fraud prevention is a constantly evolving process. What works today will not work forever.”
— Frank Kelly, Market Research Practice Lead, Virtual Incentives

The specific response patterns in this study may change over time. Bad actors adapt, qualification strategies evolve, reward preferences shift, and controls that work well today can become easier to recognize tomorrow. The lasting value of this analysis lies in adding another source of evidence to the fraud-prevention process and continually testing which signals deserve attention.

Redemption provides another point of visibility because it can reveal information that was not available during fieldwork. Connecting those later outcomes back to survey-side signals can help research teams learn which indicators meaningfully improve confidence and where stronger checks are warranted.

The ESOMAR/C Space analysis is an early example of what this approach could look like. Those patterns still need validation on new respondents, but the findings suggest that researchers may be overlooking a useful source of evidence already present in their studies: the way people answer ordinary questions.

For an industry facing increasingly sophisticated fraud, that is worth testing.