Standards for Internet-based experimenting

Abstract

This article summarizes expertise gleaned from the first years of Internet-based experimental research and
presents recommendations on: (1) ideal circumstances for conducting a study on the Internet; (2) what precautions have to be undertaken in Web experimental design; (3) which techniques have proven useful in Web experimenting; (4) which frequent errors and misconceptions need to be avoided; and (5) what should be reported. Procedures and solutions for typical challenges in Web experimenting are discussed. Topics covered include randomization, recruitment of samples, generalizability, dropout, experimental control, identity checks, multiple submissions, configuration errors, control of motivational
confounding, and pre-testing. Several techniques are explained, including “warm-up,” “high hurdle,” password
methods, “multiple site entry,” randomization, and the use of incentives. The article concludes by proposing sixteen standards for Internet-based experimenting.