Student Seminar Series: Christopher Antoun, Comparing Errors from Noncoverage and Nonresponse in a Mobile Web Survey

Purpose of the study: The quality of mobile-only Web surveys is up for discussion: some argue that surveys requiring a smartphone to participate should be avoided; others tout their usefulness for health and travel research. We help inform this discussion by evaluating two nonobservation errors in a smartphone survey. Design/methodology/approach: 1384 members of the Longitudinal Internet Studies for the Social Sciences (LISS) panel were asked about smartphone use. To estimate coverage errors, we compared those with smartphones (74%) to the full sample. Next, the eligible panel members were invited to participate in a mobile-only Web survey. To estimate nonresponse errors, we compared those who responded on their smartphone (68.5%) to the full invited sample. Preliminary findings: We find relatively large biases for some variables, suggesting that smartphone surveys should still be avoided for general population research. Comparing the two nonobservations errors, we found that the biases due to noncoverage tended to be larger in absolute magnitude than the biases due to nonresponse, which was unexpected given the relatively high coverage rates. For some variables the coverage errors offset (or canceled out) the bias associated with nonresponse; for others, the error sources moved in the same direction, compounding the bias. Originality/value: This study provides an update on earlier estimates by Fuchs and Busse (2009) of coverage errors for mobile-only surveys. We utilized the large number of demographic and behavioral measures related to health and technology that were available on almost all LISS panel members, allowing us to know the characteristics of those with and without smartphones, and those who did or did not participate in the survey. Research limitations/implications: This research is a first step towards understanding the effect of mobile-only Web surveys on data quality using total survey error framework. Future research should consider other error sources, question domains, and survey contexts. Practical implications: These findings help practitioners answer the important question of whether mobile-only Web surveys are becoming a viable alternative to traditional PC-based methods for their research topics.