Skip to main content
Research article - Peer-reviewed, 2020

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Christie, Alec P.; Abecasis, David; Adjeroud, Mehdi; Alonso, Juan C.; Amano, Tatsuya; Anton, Alvaro; Baldigo, Barry P.; Barrientos, Rafael; Bicknell, Jake E.; Buhl, Deborah A.; Cebrian, Just; Ceia, Ricardo S.; Cibils-Martina, Luciana; Clarke, Sarah; Claudet, Joachim; Craig, Michael D.; Davoult, Dominique; De Backer, Annelies; Donovan, Mary K.; Eddy, Tyler D.;
Show more authors


Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs. Randomised controlled experiments are the gold standard for scientific inference, but environmental and social scientists often rely on different study designs. Here the authors analyse the use of six common study designs in the fields of biodiversity conservation and social intervention, and quantify the biases in their estimates.

Published in

Nature Communications
2020, volume: 11, number: 1, article number: 6377

Authors' information

Christie, Alec P.
University of Cambridge
Abecasis, David
Universidade do Algarve
Adjeroud, Mehdi
Universite Perpignan Via Domitia
Alonso, Juan C.
CSIC - Museo Nacional de Ciencias Naturales (MNCN)
Amano, Tatsuya
University of Queensland
Anton, Alvaro
University of Basque Country
Baldigo, Barry P.
United States Geological Survey
Barrientos, Rafael
Complutense University of Madrid
Bicknell, Jake E.
University of Kent
Buhl, Deborah A.
United States Geological Survey
Cebrian, Just
Mississippi State University
Ceia, Ricardo S.
Universidade de Coimbra
Cibils-Martina, Luciana
Consejo Nacional de Investigaciones Cientificas y Tecnicas (CONICET)
Clarke, Sarah
Marine Inst
Claudet, Joachim
CNRS - Institute of Ecology and Environment (INEE)
Craig, Michael D.
University of Western Australia
Davoult, Dominique
Sorbonne Universite
De Backer, Annelies
Institute for Agricultural and Fisheries Research (ILVO)
Donovan, Mary K.
University of California Santa Barbara
Eddy, Tyler D.
University of South Carolina Columbia
Show more authors

Sustainable Development Goals

SDG15 Life on land

UKÄ Subject classification

Other Natural Sciences not elsewhere specified
Social Sciences Interdisciplinary

Publication Identifiers


URI (permanent link to this page)