Inter-reader agreement in multimodality imaging of carotid plaque-RADS: considerations for modality selection

Abstract

Abstract Objectives Plaque-RADS was recently developed to standardize reporting and risk stratification of carotid plaques. This study aimed to evaluate inter-reader agreement of Plaque-RADS across US, CTA, and MRI, and to assess the impact of reader experience. Materials and methods In 250 patients with carotid US, CTA, and MRI, nine readers (three per modality, stratified by experience) independently assessed plaque features and Plaque-RADS scores. Inter-reader agreement was analyzed using κ statistics and intraclass correlation coefficients (ICC). Learning curves were generated by comparing reader performance against MRI expert reference across five sequential case subsets. Results Among 962 plaques and 25 normal arteries, inter-modality agreement for overall Plaque-RADS was moderate to substantial (κ, 0.578–0.758). Agreement was almost perfect for RADS 1-2 (κ, 0.814–0.883) but lower for RADS 3-4 (κ, 0.427–0.709). US, CTA, and MRI demonstrated substantial to almost perfect agreement in assessing plaque presence and maximum wall thickness (κ/ICC, 0.697–0.878). CTA and MRI experts demonstrated substantial to almost perfect agreement in ulcer detection (κ, 0.880) and intraluminal thrombus (κ, 0.787), but moderate agreement in intraplaque hemorrhage (κ, 0.578). Regarding reader experience, all intermediate and expert readers across US, CTA, and MRI achieved substantial to almost perfect agreement in overall and different categories of RADS scoring (κ/ICC, 0.624–0.954). Learning curve analysis confirmed that all readers improved with training, with the most pronounced gains observed in US beginners. Conclusion Plaque-RADS is reliable for low-risk plaques across modalities, whereas accurate categorization of high‑risk plaques requires experienced readers and advanced imaging. These findings may help inform stratified imaging decisions. Key Points Question Does inter‑reader agreement for the Carotid Plaque‑RADS score differ across ultrasound, CTA, and MRI, and how does reader experience affect consistency? Findings Plaque‑RADS showed excellent agreement for low‑risk plaques across all modalities. High‑risk plaques required advanced imaging and experienced readers; structured training improved performance, especially for ultrasound. Critical Relevance Statement This study systematically compares Plaque‑RADS agreement across ultrasound, CTA, and MRI. The findings support a stratified imaging strategy (US for low‑risk assessment, MRI for high‑risk plaque characterization) pending prospective validation.