Multi-Modal LSFS - Theoretical Foundations for Extending LLM-Based Semantic File Systems with Unified Visual and Textual Indexing

Abstract

The rapid evolution of Large Language Model (LLM) based agents has necessitated a fundamental paradigm shift in Operating System design, moving from hierarchical, inode-based file systems to semantic, intent-driven storage architectures. While the foundational LLM-based Semantic File System (LSFS) successfully demonstrated the efficacy of natural language file operations, its utility was constrained by a uni-modal (text-only) embedding strategy. This research presents a comprehensive architectural extension to LSFS, introducing native multi-modal support grounded in high-dimensional vector space theory. By implementing a modular embedding interface that leverages Contrastive Language-Image Pre-training (CLIP) architectures and refactoring the vector storage backend to utilize Hierarchical Navigable Small World (HNSW) indexing, we enable the system to map visual and textual data into a unified semantic manifold. Furthermore, we introduce a “Dual-Path” retrieval mechanism that combines dense vector similarity with LLM-generated semantic metadata, significantly improving retrieval precision for complex user queries through Reciprocal Rank Fusion.