Valid and Reliable Assessment of Upper Respiratory TractSpecimen Collection Skills during the COVID-19 Pandemic

Proper specimen collection is the most important step to ensure accurate testing forthe coronavirus disease 2019 (COVID-19) and other infectious diseases. Assessment of healthcareworkers’ upper respiratory tract specimen collection skills is needed to ensure samples of high-qualityclinical specimens for COVID-19 testing. This study explored the validity evidence for a theoreticalMCQ-test and checklists developed for nasopharyngeal (NPS) and oropharyngeal (OPS) specimencollection skills assessment. We found good inter-item reliability (Cronbach’s alpha = 0.76) for theitems of the MCQ-test and high inter-rater reliability using the checklist for the assessment of OPSand NPS skills on 0.86 and 0.87, respectively. The MCQ scores were significantly different betweenexperts (mean 98%) and novices (mean 66%),p< 0.001, and a pass/fail score of 91% was established.We found a significant discrimination between checklist scores of experts (mean 95% score for OPSand 89% for NPS) and novices (mean 50% score for OPS and 36% for NPS),p< 0.001, and a pass/failscore was established of 76% for OPS and 61% for NPS. Further, the results also demonstratedthat a group of non-healthcare educated workers can perform upper respiratory tract specimencollection comparably to experts after a short and focused simulation-based training session. Thisstudy, therefore, provides validity evidence for the use of a theoretical and practical test for upperrespiratory specimens’ collection skills that can be used for competency-based training of the workersin the COVID-19 test centers