Inter-observer reliability in forest conservation value assessments

Abstract

Identifying and safeguarding forests of high conservation value is central to sustainable forest management. Qualitative and semi-quantitative surveys of forest conservation indicators often form the evidence base for management decisions. However, it remains unclear how consistently different surveyors assess such indicators using these methods. In this study, we evaluated inter-observer reliability (IOR) among triplets of professional biologists conducting independent surveys in 14 boreal, conifer-dominated forest stands in south-central Sweden. Surveyors recorded 50 qualitative indicators (presence-absence) and 20 semi-quantitative indices (counts and ordinal scores). We hypothesized that semi-quantitative assessments would yield higher IOR, as they are based on structured counts and ordinal scales applied within defined plots, which may reduce subjectivity. Contrary to this expectation, several qualitative indicators - based on presence-absence observations at stand scale - showed equal or even higher IOR. For example, the overall IOR for the qualitative composite score was good (intra-class correlation coefficients; ICC = 0.84), while many semi-quantitative indicators reached only moderate levels (ICC = 0.50-0.70). Indicators related to downed deadwood exhibited moderate to substantial IOR across both methods, while indicators involving standing structures, such as high nature value (HNV) trees and tree microhabitats, showed lower IOR. Our findings highlight that indicator-specific characteristics (e.g., subjectivity, rarity), rather than assessment method alone, influence reliability. Excluding low-reliability structural (e.g., tree microhabitats) indicators from qualitative protocols slightly improved overall agreement. We recommend integrating IOR analyses to refine survey protocols, guide surveyor training, and improve consistency in forest conservation value assessments. Even small-scale IOR evaluations - such as those involving three independent surveyors - can yield valuable insights into observer bias within relatively homogeneous groups of professional surveyors. Future research should expand such analyses to a wider range of ecosystems, indicator types, and surveyor backgrounds to strengthen the robustness and credibility of qualitative and semi-quantitative forest conservation value assessments.

Keywords

Boreal forest; Forest and site indicators; Observer bias; Qualitative method; Sampling errors; Semi-quantitative method

Published in

Forest Ecology and Management
2025, volume: 595, article number: 123006
Publisher: ELSEVIER

SLU Authors

• Jönsson, Mari

  • Department of Ecology, Swedish University of Agricultural Sciences
    
  • SLU Swedish Species Information Centre, Swedish University of Agricultural Sciences
    

• Hekkala, Anne-Maarit

  • Department of Wildlife, Fish and Environmental Studies, Swedish University of Agricultural Sciences
    

• Clemmensen, Karina

  • Department of Forest Mycology and Plant Pathology, Swedish University of Agricultural Sciences
    

• Kyaschenko, Julia

  • Department of Ecology, Swedish University of Agricultural Sciences
    

• Kärvemo, Simon

  • Department of Ecology, Swedish University of Agricultural Sciences
    

• Mielke, Louis

  • Department of Forest Mycology and Plant Pathology, Swedish University of Agricultural Sciences
    

• Sjögren, Jörgen

  • Department of Wildlife, Fish and Environmental Studies, Swedish University of Agricultural Sciences
    

• Strengbom, Joachim

  • Department of Ecology, Swedish University of Agricultural Sciences
    

UKÄ Subject classification

Forest Science

Publication identifier

• DOI: https://doi.org/10.1016/j.foreco.2025.123006

Permanent link to this page (URI)

https://res.slu.se/id/publ/143139