Development and validation of a theoretical test for ultrasound-guided biopsy competence in the head and neck

Abstract

PURPOSE: This prospective study is designed to test the validity of a multiple-choice question (MCQ) test in assessing theoretical knowledge of ultrasound-guided fine-needle aspiration biosy and core-needle biopsy of the head and neck. METHODS: Seven international experts reached consensus on the MCQ test using a modified Delphi process. After consensus, the MCQ was tested on novices (n=59) and experienced specialists (n=17). Item analysis was performed to evaluate difficulty and discriminatory ability, Cronbach's α was calculated to assess internal consistency reliability, and group differences were compared using ANOVA and t-tests. The contrasting groups' standard setting method was applied to set a pass/fail standard and thereafter, participants in a head and neck ultrasound course (n=27) took the test to assess the consequences. RESULTS: Two Delphi rounds yielded a 23-item MCQ test that reflected essential knowledge for ultrasound-guided biopsies. Two poorly performing items were removed following item analysis, resulting in a 21-item test with good reliability, Cronbach's α=0.82. Experienced clinicians scored significantly higher (mean 20.0±0.9) than both course participants (17.3±2.8) and novices (13.5±3.6, p<0.001). A pass/fail score of 18/21 (≈86%) was established as the cut-off. At this standard, 100% of experts passed the test while only 13.6% of novices and 63% of course participants achieved a passing score. CONCLUSION: An MCQ test of theoretical knowledge for ultrasound-guided head and neck biopsies was developed with evidence of validity across all five sources in Messick's framework. The test demonstrated strong content validity through expert consensus, good internal structure, significant discrimination between experience levels, and a defensible pass/fail standard. At present, the instrument should be interpreted as a theoretical knowledge assessment that may complement, but not replace, practical skills-based assessment.