A zero-shot LLM framework for analyzing user preference on product attributes: a case study of smartwatch reviews

Abstract

The rise of zero-shot learning through Large Language Models (LLMs) opens new opportunities for analyzing consumer feedback without relying on annotated data. This study proposes a zero-shot framework to uncover user preferences on product attributes, using 12,900 Amazon smartwatch reviews as a case study. The framework integrates semantic keyword extraction, LLM-based attribute classification, sentiment analysis, and Importance–Performance Analysis (IPA) to convert unstructured textual reviews into structured consumer preference insights. Validation against human annotations demonstrates strong sentiment classification performance, achieving an accuracy of 0.88 and a macro-averaged F1-score of 0.86, confirming the reliability of the zero-shot approach. The results reveal that Design & Comfort, Connectivity, and Health Monitoring Features are the most important and positively perceived smartwatch attributes, while Power & Battery Management remains a key area of consumer dissatisfaction. Brand-level analysis further shows that major brands such as Apple, Samsung, and Garmin dominate consumer attention but exhibit only moderate differentiation in sentiment performance, indicating opportunities for strategic improvement. Methodologically, the study demonstrates that zero-shot LLM inference can reliably extract and organize consumer preference structures without requiring labeled data. The findings contribute to consumer analytics and computational linguistics by demonstrating how zero-shot frameworks enable scalable, adaptive preference modeling, providing actionable insights for product development and competitive positioning.