LLM-Mediated Findability of Research Datasets

Abstract

This study evaluates the LLM-mediated findability of research datasets. The sample uses 100 datasets randomly selected from GESIS. Experiments were run using web-search-based models from OpenAI and Gemini. Six different query strategies combine metadata such as topic, geography, time, title, and abstract-derived natural-language research needs. Results show that query design substantially affects retrieval. One variant uses queries with dataset titles to provide the known-item baseline. In contrast, other metadata were used in generic metadata queries, including topic and geographic coverage, with temporal information to emulate user behavior and information needs. Among realistic discovery strategies, the natural-language approach (V6) performs best, achieving 26.4% Strict Source retrieval with OpenAI and 32.2% with Gemini. The findings highlight the importance of prompt design, metadata quality and specificity, web visibility, and identifier consistency for AI-mediated dataset discovery.