Biologically Informed Multi-Task Learning for Call-Type Classification in Zebra Finches Using Functional Semantic Hierarchies

Abstract

Zebra finches (Taeniopygia guttata) possess a diverse vocal repertoire that plays an important role in their social communication, yet automated classification of their vocalizations remains largely unexplored. This study presents a biologically informed multi-task learning framework that jointly classifies zebra finch vocalizations at two hierarchical levels: the call-type level, comprising eleven individual call types, and the semantic level, comprising six broader categories derived from an established perceptual taxonomy. Frozen embeddings from the BirdAVES pre-trained audio transformer are combined with a shared encoder trained under a composite objective integrating call-type cross-entropy, cross-entropy over semantic targets, and Supervised Contrastive Loss. Four experimental configurations are evaluated, comprising two single-task baselines, one for call-type and one for semantic classification, and two multi-task variants differing in their semantic loss weighting strategy. Across the two multi-task configurations, the best call-type performance is achieved by the dynamic weighting variant, reaching 87.11% accuracy and a macro F1-score of 82.76%, while the best semantic performance is achieved by the scheduled weighting variant, reaching 90.55% accuracy and a macro F1-score of 90.74%. Both multi-task configurations outperform their respective single-task baselines. Embedding space analysis reveals an approximately three-fold increase in silhouette score under multi-task learning, suggesting that biologically informed supervision may improve the geometric structure of learned acoustic representations in a manner consistent with the functional organization of the zebra finch vocal repertoire.