BirdNET and Perch miss most labelled calls in an Amazon rainforest test

Two widely used bird-sound classifiers missed most of the expert-labelled calls in a Brazilian Amazon test. Researchers evaluated BirdNET 2.3 and Perch 2.0 against 14,705 vocalisations and found average call-level recall of 0.08 and 0.11. The finding puts a number on a major challenge for automated rainforest monitoring: recording a rich soundscape is easier than assigning reliable species labels to it.[1]

Recall measures the share of real target calls retrieved as correct detections. The low scores therefore describe missed events under the study’s conditions, rather than the fraction of individual birds or species absent from an inventory. Performance varied substantially among species, and the models performed better when predictions were combined across recording sites.[1]

Why the tropics change the rules

Global sound recognizers are trained on recordings drawn from unevenly sampled regions, species and habitats. Large archives of temperate birds, familiar singing species and clean recordings do not guarantee strong coverage of the Amazon's remarkable diversity. Dense vegetation, overlapping calls, insects, distance and recording quality complicate an already demanding task. Species that sound similar to human specialists can be harder still for a model whose examples of those species are scarce.[1]

The study found performance significantly worse for passerines than non-passerines and a general deterioration for higher-frequency vocalizations. Classification improved with signal-to-noise ratio and the amount of available species training audio. This is the familiar 'data availability' pattern with an ecological twist: the very regions where continuous automated monitoring could be valuable may have the least balanced and representative training material. A global model can be a global product without being globally reliable.[1][2]

Precision–recall area under the curve was also low: 0.07 for BirdNET and 0.12 for Perch. Together with the recall results, that argues against treating the raw outputs as dependable species records. Threshold choice changes the balance between missed calls and false detections, but it does not remove the need to establish how the classifier behaves in the habitat being surveyed.[1]

A failure at one task, evidence at another

The same paper offers a more interesting conclusion than 'AI doesn't work'. When researchers aggregated classifier detections across recording sites, the outputs retained informative patterns about how bird communities differed. The models tracked some aspects of manually labelled species richness at low confidence thresholds and preserved broad community structure. Among commonly detected species, 64% met minimum performance thresholds for at least one of the study's proposed monitoring applications after site aggregation. These qualifications describe specific analytical uses, not permission to promote low-confidence outputs to verified sightings.[1]

An acoustic index can be helpful without claiming that every label identifies a real bird. If the relative mix of predicted sounds changes systematically across sites, it may indicate that the wider soundscape is changing. But detecting turnover is different from identifying which threatened species disappeared or arrived. Conservation decisions about a particular rare animal need a different standard, normally including targeted field validation and biological context.[1][2]

The useful distinction is between a species record and a community signal. The first claims that a particular animal was detected; the second tracks a pattern across many uncertain observations. This study found more support for the second use. Keeping those jobs separate allows conservation teams to use the information the models recover without turning their missing calls into missing wildlife.[1]

Report an error or suggest a correction