Acoustic recognition of tilapia feeding intensity via LoRA-fine-tuned multimodal large language models

Abstract

Real-time assessment of fish feeding intensity is essential for demand-based feeding in intensive aquaculture, yet existing methods rely on external spectrogram preprocessing and large annotated datasets, limiting practical deployment. This study proposes an end-to-end approach that reformulates feeding intensity recognition as an audio instruction-following task using the multimodal large language model Qwen2.5-Omni-3B. Raw audio clips and natural language prompts are taken as direct inputs, and low-rank adaptation (LoRA) is applied for parameter-efficient fine-tuning under data-scarce conditions. Experiments on 4186 tilapia audio recordings show that the method achieves 98.21% accuracy with 200 training samples on the same test set used by a CNN baseline, and 96.96% on an independent test set of 2000 samples. Ablation studies indicate that applying LoRA across all linear layers with rank 32 and scaling factor 256 yields a favorable performance–efficiency trade-off. The results suggest that coupling pre-trained multimodal models with parameter-efficient fine-tuning is a promising approach for reducing dependence on annotated data in underwater acoustic recognition tasks, offering a scalable solution for intelligent feeding systems in aquaculture.