Tech Mining with LLMs: Unveiling Trends from Corporate Websites

Abstract

Abstract Technology mining enables the systematic extraction and analysis of trends regarding scientific and technological developments from diverse Science, Technology, and Innovation (ST&I) data sources. Traditionally, indicators such as patents, surveys, and R&D statistics have been widely used to monitor technological change and innovation. This study proposes a novel methodology leveraging semantic analysis of corporate website content using large language models (LLMs) to facilitate real-time monitoring of technological developments. Specifically, we focus on Spanish manufacturing firms that have adopted artificial intelligence (AI). A pipeline was developed to automatically extract and classify bigrams and unigrams from corporate websites into technological and non-technological terms, enabling a temporal analysis of technologies for the years 2023 and 2025. The results indicate an increase in AI-related mentions, as well as notable patterns of co-occurrence with Internet of Things (IoT) and Industry 4.0 technologies. Our study demonstrates the utility of the combination of corporate websites and LLMs as scalable, dynamic, and cost-effective sources for technology mining, contributing methodologically through advanced text analytics and empirically by providing granular, web-derived insights into technological dynamics within the manufacturing sector.