Understanding and optimising Pipeline performance

Improving recording quality

It is crucial to capture the best quality field recordings possible. The better the quality of recordings, the better the Acoustic Pipeline will perform, but also the better your chances of being able to manually assign unknown recordings to species. This is particularly important for some of the more cryptic species which have similar vocalisations.

Placing your recorders or bat detectors correctly

Recording quality is greatly influenced by the placement of your microphone, recorder or bat detector. Poor placement can result in “acoustic mush” that the Pipeline will be less adept at processing. This is particularly critical in the ultrasonic, e.g. if you are interested in identifying bats. 

Recording at a bat roost

pipistrelle_bat_creative_nature.png

If recording at a bat roost, there are some additional considerations when placing your bat detector or microphone.

• Position the bat detector 10–​15 m away from the emergence itself. Directly at emergence, calls are typically elevated in frequency and are not typical of the calls of free-flying bats, which have been used to train the Pipeline algorithms. 
• Avoid recording inside a bat roost if possible. Inside the roost, echolocation calls occur alongside a broad range of social calls that are not encountered outside a roost and are often atypically short in duration, differing significantly from the calls which have been used to train the Pipeline algorithms. Recordings can also be distorted due to the small space.

Microphone, recorder and bat detector settings

We recommend that bat detector settings are used that define the maximum wav file length to be 5 seconds. This is the optimum recording length for the Pipeline algorithms to identify multiple species. This also means that comparisons of activity based on the number of recordings are more standardised within and between different projects.

Longer recordings can be uploaded and will be processed by the pipeline, but long recordings are split automatically by the desktop app into 5-second sections before they are processed.

If you do choose to capture and upload longer recordings:

• In the desktop app, you can save a copy of the wav files as they will be processed, including split wav files. The consequence of this is that results are also returned for each 5-second wav file section. For example, if a 15-second recording is uploaded, the recording will be split into three, and results returned for each 5-second file. Auditing could then be carried out at the 5-second section level, or the pipeline results could be summarised to collapse the results by the ORIGINAL FILE NAME which is included as a column in the results.
• There will be a greater proportion of results flagged as low confidence, and more wav files where no species are identified (No_ID in the species column). The last 5-second section of a long wav file often contains weak calls which occur at the end of a call sequence. These weak calls are more difficult to assign to species with confidence, or do not contain any identifiable signals. 

Pipeline performance and whole night recording 

The Ultrasonic classifiers (for bats, bush-crickets, small mammals) in the Pipeline have been designed to process whole nights of recording, and take the wider context of what is recorded across the night and site into account when considering an identification.

Please note that the performance will be reduced if single / a small number of selected .wav files of interest are uploaded, particularly where this relates to scarce species, or for bush-crickets.

Pipeline performance in different geographical areas

The performance of the pipeline for a given species will vary geographically, depending on the suite of potential species present in the region.

As an example, Pipeline performance when identifying Leisler’s Bat Nyctalus leisleri is likely to be better in Ireland than in southern Britain. This is because there are two potential confusion species alongside Leisler’s Bat in southern Britain: Noctule Nyctalus noctula and Serotine Eptesicus serotinus. You should bear this in mind when auditing your recordings. 

Full spectrum recordings and zero-crossing

We do not recommend converting full-spectrum recordings to zero-crossing after processing to audit the recordings/results.

Converting recordings will result in a significantly lower number of identifiable calls being assigned manually to species. It may also introduce biases in your data, which could influence your interpretation. 

As an example, bat recording is carried out at two locations with similar levels of Barbastelle Barbastella barbastellus activity. However, at one location, there is also high bush-cricket activity. The Pipeline is efficient at finding weak barbastelle calls in noisy recordings like this, so the activity of Barbastelle according to the Pipeline should be similar.

However, if you convert your full-spectrum recordings to zero-crossing to then check the identifications, the Barbastelle calls may be lost where there are noisy bush-crickets.

Pipeline performance: a case study

The following case study provides quantitative information on the performance of the Acoustic Pipeline.

• We collected recordings as part of a Defra-funded project about bats in English farmland, from six study areas: Dartmoor, Yorkshire Dales, fens in Cambridgeshire and Norfolk, Essex, Sussex and Warwickshire.
• The Acoustic Pipeline carried out an automatic analysis, which was followed by extensive manual verification of the identifications returned by the Pipeline. 
• An experienced bat acoustics analyst viewed and played spectrograms in Sonobat to confirm identifications. 

Notes on rare and cryptic species

• We exclude rare species captured in fewer than 20 recordings, as the small sample limits interpretation.
• The Pipeline assigns Brandt’s Bat Myotis brandtii and Whiskered Bat Myotis mystacinus to species level, but we present results for these species combined because no robust criteria for manually confirming the identification is currently available.
• We selected a random sample of 1,000 recordings for Common Pipistrelle Pipistrellus pipistrellus and Soprano Pipistrelle Pipistrellus pygmaeus for checking. We carried out manual verification for all other species recordings (84,055 in total).
• The only exceptional result was a large number of Whiskered or Brandt’s Bat recordings from northern England.

The confusion matrix below compares the number of recordings that were assigned by the Pipeline to each species with the identifications generated by the manual verification. View full size >

acoustic_pipeline_error_matrix.jpg