Computerized Device Equivalence: A Comparison of Surveys Completed Using A Smartphone, Tablet, Desktop Computer, and Paper-and-Pencil

There is a limited body of experimental research examining the comparability of completing self-report surveys using different computerized devices. Additionally, available literature has not used complete or optimal procedures for determining device equivalence. The current study examined the comparability of surveys completed using paper-and-pencil and three popular devices: smartphone, tablet, and desktop computer. Participants consisted of 211 college students randomly assigned to conditions who completed measures of personality, social desirability, and computer self-efficacy. Results showed evidence of qualitative equivalence (internal consistency and subscale intercorrelations) across conditions. For quantitative and auxiliary equivalence, both equivalence testing and Bayesian analyses were conducted. Equivalence testing indicated quantitative (mean score) equivalence, as well as comparability for one aspect of auxiliary equivalence (missing data). Other aspects of auxiliary equivalence (completion time and comfort completing questionnaires) suggested potentially meaningful differences. Bayesian analyses typically replicated these results, with some notable exceptions regarding auxiliary equivalence.