Preprints are early versions of research articles that have not been peer reviewed. They should not be regarded as conclusive and should not be reported in news media as established information.
Jaciw 2023 ADVANCE Sometimes estimates from QEDs are less biased than from RCTs.pdf (899.08 kB)
Download file

Hold the Bets! Do Quasi- and True Experimental Evaluations Yield Equally Valid Impact Results When Effect Generalization is the Goal?

Download (899.08 kB)
posted on 2023-04-24, 09:49 authored by Andrew JaciwAndrew Jaciw


Randomized experiments (RCTs) rule out bias from confounded selection of participants into conditions by design. Quasi-experiments (QEs) are often considered second-best because they do not share this benefit. However, when results from RCTs are used to generalize causal impacts, the benefit from unconfounded selection into conditions may be offset by confounded selection into locations. In this work we show that this tradeoff can lead to situations where estimates from QEs are less-biased from selection than are estimates from uncompromised RCTs. We establish the conditions theoretically, demonstrate the idea empirically, and discuss the implications of the results.


No external sources of funding


Declaration of conflicts of interest

No conflict of interest

Corresponding author email

Lead author country

  • United States

Lead author job role

  • Practitioner/Professional

Lead author institution

Empirical Education Inc.

Human Participants

  • Yes

Ethics statement

Empirical analysis is a secondary analysis of a publicly available dataset

Terms agreed

  • Yes, I agree to Advance terms


Log in to write your comment here...

Usage metrics

    Advance: Social Sciences & Humanities