Interobserver Agreement of Lung Ultrasound Findings of COVID-19

Andre Kumar, Yingjie Weng, Sally Graglia, Sukyung Chung, Youyou Duanmu, Farhan Lalani, Kavita Gandhi, Viveta Lobo, Trevor Jensen, Jeffrey Nahn, John Kugler

Research output: Contribution to journalArticlepeer-review

13 Scopus citations


Background: Lung ultrasound (LUS) has received considerable interest in the clinical evaluation of patients with COVID-19. Previously described LUS manifestations for COVID-19 include B-lines, consolidations, and pleural thickening. The interrater reliability (IRR) of these findings for COVID-19 is unknown. Methods: This study was conducted between March and June 2020. Nine physicians (hospitalists: n = 4; emergency medicine: n = 5) from 3 medical centers independently evaluated n = 20 LUS scans (n = 180 independent observations) collected from patients with COVID-19, diagnosed via RT-PCR. These studies were randomly selected from an image database consisting of COVID-19 patients evaluated in the emergency department with portable ultrasound devices. Physicians were blinded to any patient information or previous LUS interpretation. Kappa values (κ) were used to calculate IRR. Results: There was substantial IRR on the following items: normal LUS scan (κ = 0.79 [95% CI: 0.72–0.87]), presence of B-lines (κ = 0.79 [95% CI: 0.72–0.87]), ≥3 B-lines observed (κ = 0.72 [95% CI: 0.64–0.79]). Moderate IRR was observed for the presence of any consolidation (κ = 0.57 [95% CI: 0.50–0.64]), subpleural consolidation (κ = 0.49 [95% CI: 0.42–0.56]), and presence of effusion (κ = 0.49 [95% CI: 0.41–0.56]). Fair IRR was observed for pleural thickening (κ = 0.23 [95% CI: 0.15–0.30]). Discussion: Many LUS manifestations for COVID-19 appear to have moderate to substantial IRR across providers from multiple specialties utilizing differing portable devices. The most reliable LUS findings with COVID-19 may include the presence/count of B-lines or determining if a scan is normal. Clinical protocols for LUS with COVID-19 may require additional observers for the confirmation of less reliable findings such as consolidations.

Original languageEnglish (US)
Pages (from-to)2369-2376
Number of pages8
JournalJournal of Ultrasound in Medicine
Issue number11
StatePublished - Nov 2021
Externally publishedYes


  • COVID-19
  • interobserver agreement
  • interrater
  • lung
  • reliability
  • SARS-CoV-2
  • ultrasound

ASJC Scopus subject areas

  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Interobserver Agreement of Lung Ultrasound Findings of COVID-19'. Together they form a unique fingerprint.

Cite this