Multi-modes for Detecting Experimental Measurement Error

Tom Robinson, DPhil student, and Ray Duch, Director of Centre for Experimental Social Sciences, have co-authored a new article on how to experimental measurement error. The paper was written alongside Denise Laroze (Universidad de Santiago de Chile) and Pablo Beramendi (Duke University).


Experiments should be designed to facilitate the detection of experimental measurement error. To this end, the authors advocate the implementation of identical experimental protocols employing diverse experimental modes. They suggest iterative nonparametric estimation techniques for assessing the magnitude of heterogeneous treatment effects across these modes. They propose two diagnostic strategies—measurement metrics embedded in experiments, and measurement experiments—that help assess whether any observed heterogeneity reflects experimental measurement error. To illustrate their argument, first we conduct and analyze results from four identical interactive experiments: in the lab; online with subjects from the CESS lab subject pool; online with an online subject pool; and online with MTurk workers. Second, they implement a measurement experiment in India with CESS Online subjects and MTurk workers.

Professorial Fellow Raymond Duch is Director of Centre for Experimental Social Sciences, Fellow of Nuffield CollegeMr Tom Robinson is a DPhil student at DPIR