random assignment doesn't work (as perfectly as experimenters assume)

Abstract

In social research, estimates of the effect of one variable on another can be distorted by individual variation associated with the dependent variable. In experimental designs, random assignment of subjects to conditions helps address this challenge. However, random assignment does not always produce equivalent groups, and when distributions of an extraneous variable differ across conditions the apparent effect of a treatment may differ substantially from the true effect. On this basis, a simulation was conducted to examine (a) the extent that random assignment of subjects to conditions affects the distribution of pretest scores in a posttest-only, control group experiment, and (b) the extent to which these various distributions of pretest scores influence the observed effect of the experimental treatment on the dependent variable. Results indicate that random assignment often fails to create equivalent groups, and that these failures lead to substantial discrepancies between observed and actual effects of a treatment.