A Study on LLM-based Method for deriving Critical Safety Items to Enhance Submarine Operational Safety

Abstract

Submarines pose a high risk of catastrophic incidents that may result in significant loss of life and severe degradation of strategic capabilities. In this context, the systematic management of Critical Safety Items (CSIs) is a fundamental requirement throughout the design, construction, and operational phases of submarines. However, conventional CSI identification approaches have largely relied on expert experience and qualitative judgment, which may limit objectivity, consistency, and traceability when large volumes of heterogeneous technical documents are considered. This study proposes a large language model (LLM)-based methodology to address these limitations and enhance the rigor of the CSI derivation process. The proposed approach leverages diverse and authoritative reference sources, including the NATO Naval Submarine Code, SUBSAFE, NAVSEAINST 9078.1/2, MIL-STD-882E, domestic submarine design documents, and quality assurance data. Based on these inputs, CSIs are systematically and objectively derived through an LLM-driven analytical process. Within the proposed framework, submarine expert-driven prompt engineering and Retrieval-Augmented Generation (RAG) techniques are employed to enhance domain specificity, evidence traceability, and analytical reliability. Considering the current limitations of generative AI technologies, the reliability of the LLM-based analysis is reinforced through expert-led selection of critical safety functions and cross-validation of the derived results. The proposed approach reduces potential omissions in experience-dependent methodologies and enables the systematic and evidence-traceable identification of CSIs, thereby supporting a more robust and objective submarine safety management process.