Rule‐Based Versus Generative Extraction of Psychological Symptoms From Forensic Medical Certificates: A Feasibility Study in the French ORFeAD Network

Abstract

ABSTRACT Forensic medical certificates for victims of interpersonal violence describe psychological symptoms in unstructured prose, rarely explicit and requiring inference. To assess the feasibility of two extraction pipelines—rule‐based and a locally served generative large language model—in a French multicentric forensic network, and to identify variables usable without human verification. 110 randomly selected 2021 certificates from 12 ORFeAD units were processed by both pipelines: rule‐based (segmentation, lexicons, regular expressions, negation detection) and generative (Llama 3 8B, 4‐bit, Ollama, near‐default settings). 35 binary variables, 23 psychological or subjective, were coded by two forensic physicians with third‐party arbitration. Sensitivity, specificity and accuracy were computed per variable; 85% reliability defined usability. No inferential testing was performed. Specificity was high for most variables; sensitivity was heterogeneous and low for low‐prevalence symptoms. 24 of 35 variables met the threshold under at least one pipeline, 10 under both, 11 under neither. Processing required 0.1–0.15 CPU‐seconds per document versus 20–60 GPU‐seconds. Out of the box, the local model did not clearly outperform the rule‐based pipeline, at three orders of magnitude greater energy cost and requiring dedicated GPUs. Per‐variable qualification is a prerequisite for research use; neither pipeline supports individual‐level decisions.