Comprehensive AI workflow for radiological report generation from radiographs

Abstract

The manual generation of radiology reports is time-consuming and error-prone, particularly for less experienced radiologists, leading to inefficiencies and delays in clinical workflows as case volumes increase. Therefore, automated radiology report generation can serve as valuable tool to support hospitals and improve efficiency. This paper presents an automated two-stage approach for radiology report generation, consisting of keyword prediction from X-ray images followed by text-to-text report generation. The publicly available IU X-ray dataset, comprising 3,851 patients with paired PA and lateral chest X-rays and corresponding reports, was divided into training (80%), validation (10%), and test (10%) sets. Three multilabel classifiers based on the ConvNeXT deep learning architecture were developed to predict keyword clusters from paired X-ray images. Unlike prior keyword-based pipelines, the proposed method introduces a frequency-aware clustering strategy that partitions keywords into logarithmic frequency groups ([10,100), [100,1000), and [1000,10000)) with dedicated classifiers trained for each cluster. This design reduces label sparsity and improves the modeling of low frequency findings. Keyword extraction from ground-truth reports was performed using the KeyBERT model, and these keywords were used as target labels for training. To address class imbalance, BCE-based focal loss was employed in combination with transfer learning. In the second stage, the T5 (Text-to-Text Transfer Transformer) model was trained to generate radiology reports from the predicted keywords for each patient. Multiple metrics were computed for the pipeline evaluation including BLEU, ROUGE-L, METEOR, and CIDEr. The evaluation of our method demonstrated good performance compared to SOTA approaches validated on the same dataset. On the test set, it achieved BLEU-1 = 0.50 (≈11% relative improvement over R2Gen), BLEU-2 = 0.33, BLEU-3 = 0.22, BLEU-4 = 0.16, ROUGE-L = 0.41, METEOR = 0.21, and CIDEr = 0.37, indicating strong agreement between generated and reference reports. The proposed two-stage framework integrates visual and textual information, improving both accuracy and interpretability of generated radiology reports using intermediate keyword prediction. This design enhances transparency by linking image features to clinically relevant terms prior to report generation. Future work will focus on extending this approach to 3D imaging modalities and validating its performance on clinical datasets.