Accounting for automated identification errors in acoustic surveys

Abstract

1. Assessing the state and trend of biodiversity in the face of anthropogenic threats requires large-scale and long-time monitoring, for which new recording methods offer interesting possibilities. Reduced costs and a huge increase in storage capacity of acoustic recorders has resulted in an exponential use of Passive Acoustic Monitoring (PAM) on a wide range of animal groups in recent years, in particular for bats for which PAM constitutes an efficient tool. PAM for bats has led to a rapid growth in the quantity of acoustic data, making manual identification increasingly time-consuming. Therefore, software detecting sound events, extracting numerous features, and automatically identifying species have been developed. However, automated identification generates identification errors, which could influence analyses which looks at the ecological response of species. In this study we propose a cautious method to account for errors in acoustic identifications without excessive manual checking of recordings.
2. We propose to check a representative sample of the outputs of a software commonly used in acoustic surveys (Tadarida), to model the identification success probability of 10 species and 2 species groups as a function of the confidence score provided for each automated identification. Using this relationship, we then investigated the effect of setting different False Positive Tolerances (FPTs), from a 50% to 10% false positive rate, above which data are discarded, by repeating a largescale analysis of bat response to environmental variables and checking for consistency in the results.
3. Considering estimates, standard errors and significance of species response to environmental variables, the main changes occurred between the naive (i.e. raw data) and robust analyses (i.e. using FPTs). Responses were highly stable between FPTs.
4. We conclude it was essential to, at least, remove data above 50% FPT to minimize false positives. We recommend systematically checking the consistency of responses for at least two contrasting FPTs (e.g. 50% and 10%), in order to ensure robustness, and only going on to conclusive interpretation when these are consistent. This study provides a huge saving of time for manual checking, which will facilitate the improvement of large-scale monitoring, and ultimately our understanding of ecological responses.