InstructDS: An Instruction-Tuned language model with smoothed contrastive confidence decoding for generating patient discharge instructions

Abstract

Documenting patient discharge instructions is vital for continuity of care, but it also increases provider documentation burden. Large language models now offer an opportunity to support the generation of personalised discharge instructions. This paper introduces InstructDS, an instruction-tuned large language model based on Mistral-NeMo-12B-Instruct, designed to generate patient discharge instructions. We formatted the MIMIC-IV corpus into input–output pairs, each enhanced with detailed, domain-specific instructions. To refine output quality without additional fine-tuning, we developed a decoding-time strategy, Smoothed Contrastive Confidence Decoding (SCCD), which adjusts token selection based on the confidence ratio between the top two candidate tokens. InstructDS achieved a BERTScore of 87.63% and a METEOR score of 42.66% on the MIMIC-IV test set, improving over the pre-trained Mistral-NeMo baseline and the fine-tuned model without SCCD. Wilcoxon signed-rank tests on BERTScore showed that the differences were statistically significant ( $$p < 0.01$$ ). SCCD further improved lexical precision across all n-gram levels, with relative gains of 22.40% in BLEU-1, 17.05% in BLEU-2, 11.02% in BLEU-3, and 6.93% in BLEU-4. Using GPT-4 as an LLM-based judge, InstructDS reached 100% relevance, 98.5% patient-friendly tone, and 95% instruction faithfulness. Hallucinations were observed in 4% of cases, highlighting the continued need for clinical oversight. Instruction-tuned large language models combined with confidence-aware decoding can generate clear, personalised discharge instructions and may support clinical documentation workflows, while clinician oversight remains essential for safe deployment.