Skip to main content
Research article - Peer-reviewed, 2021

Validation of stock assessment methods: is it me or my model talking?

Kell, Laurence T.; Sharma, Rishi; Kitakado, Toshihide; Winker, Henning; Mosqueira, Iago; Cardinale, Massimiliano; Fu, Dan

Abstract

The adoption of the Precautionary Approach requires providing advice that is robust to uncertainty. Therefore, when conducting stock assessment alternative, model structures and data sets are commonly considered. The primary diagnostics used to compare models are to examine residuals patterns to check goodness-of-fit and to conduct retrospective analysis to check the stability of estimates. However, residual patterns can be removed by adding more parameters than justified by the data, and retrospective patterns removed by ignoring the data. Therefore, neither alone can be used for validation, which requires assessing whether it is plausible that a system identical to the model generated the data. Therefore, we use hindcasting to estimate prediction skill, a measure of the accuracy of a predicted value unknown by the model relative to its observed value, to explore model misspecification and data conflicts. We compare alternative model structures based on integrated statistical and Bayesian state-space biomass dynamic models using, as an example, Indian Ocean yellowfin tuna. Validation is not a binary process (i.e. pass or fail) but a continuum; therefore, we discuss the use of prediction skill to identify alternative hypotheses, weight ensemble models and agree on reference sets of operating models when conducting Management Strategy Evaluation.

Keywords

diagnostics; hindcast; prediction skill; retrospective analysis; stock assessment; validation

Published in

ICES Journal of Marine Science
2021, volume: 78, number: 6, pages: 2244-2255
Publisher: OXFORD UNIV PRESS

Authors' information

Kell, Laurence T.
Imperial College London
Sharma, Rishi
Food and Agriculture Organization of the United Nations (FAO)
Kitakado, Toshihide
Tokyo University of Marine Science and Technology
Winker, Henning
European Commission Joint Research Centre
Mosqueira, Iago
Wageningen University and Research
Swedish University of Agricultural Sciences, Department of Aquatic Resources
Fu, Dan
Indian Ocean Tuna Commission

UKÄ Subject classification

Fish and Aquacultural Science

Publication Identifiers

DOI: https://doi.org/10.1093/icesjms/fsab104

URI (permanent link to this page)

https://res.slu.se/id/publ/114247