Comparison of performance between artificial intelligence and radiologists in detecting abnormalities on chest X-rays
Authors:
Jakub Dandár 1; Tomáš Jindra 2; Daniel Kvak 1, 3
Authors‘ workplace:
Carebot, s. r. o., Praha
1; Nemocnice Tábor, a. s.
2; Masarykova univerzita v Brně
3
Published in:
Čas. Lék. čes. 2025; 164: 125-140
Category:
Original Article
Overview
Artificial intelligence (AI) has been increasingly applied in radiology, where it offers the potential to improve the accuracy and efficiency of diagnosis, particularly in the evaluation of conventional imaging modalities such as chest X-rays. This study analyzes the performance of commercial software using machine learning and, respectively, artificial intelligence approaches (Carebot AI CXR; Carebot s.r.o.) in detecting abnormalities in chest radiographs compared with independent evaluations by 3 radiologists of different levels of experience. The study was conducted in collaboration with Hospital Tabor, which provided a dataset of 207 anonymised radiographs, out of which 196 were assessed as relevant. The sensitivity and specificity of AI were compared with human assessment in 5 categories of abnormalities: atelectasis (ATE), consolidation (CON), cardiac shadow enlargement (CMG), pleural effusion (EFF) and pulmonary lesions (LES).
Carebot AI CXR software achieved high sensitivity in all evaluated categories (e.g., ATE: 0.909, CMG: 0.889, EFF: 0.951), and its performance was consistent across all findings. In contrast, AI specificity was lower in some categories (e.g., EFF: 0.792, CON: 0.895), while radiologists achieved performance values approaching 1.000 in most cases (e.g., RAD 1 and RAD 2 EFF: 1.000). AI demonstrated consistently higher sensitivity than less experienced radiologists (e.g., RAD 1 ATE: 0.087, CMG: 0.327) and in some cases than more experienced assessors, but at a modest decrease in specificity.
The study also includes case reports, including false-positive and false-negative findings, which contribute to a deeper understanding of AI performance in clinical practice. The results suggest that AI can effectively complement the work of radiologists, especially for less experienced doctors, and improve the sensitivity of diagnosis on chest radiographs.
Keywords:
artificial intelligence, radiology, chest X-ray, abnormality detection, multi-reader study
Sources
- Barentsz J, Takahashi S, Oyen W et al. Commonly used imaging techniques for diagnosis and staging. J Clin Oncol 2006; 24: 3234–3244.
- Schaefer-Prokop C, Neitzel U, Venema HW et al. Digital chest radiography: an update on modern technology, dose containment and control of image quality. Eur Radiol 2008; 18: 1818–1830.
- Radiology facing a global shortage. Radiological Society of North America, 2022. Dostupné na: www.rsna.org/news/2022/may/global-radiologist-shortage
- Kvak D, Chromcová A, Ovesná P et al. Detecting pulmonary lesions in low-prevalence real-world settings using deep learning. In: Proceedings International Conference on Medical Imaging and Computer-Aided Diagnosis 2023 (MICAD 2023). Springer Nature, Singapore, 2023: 3–20.
- Yamashita R, Nishio M, Do RKG et al. Convolutional neural networks: an overview and application in radiology. Insights Imaging 2018; 9: 611–629.
- Redmon J. You only look once: unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016. Las Vegas, NV, USA, 2016. Conference Publishing Services, IEE Computer Society, Los Alamitos, Washington, Tokyo, 2016: 779–788.
- Nařízení Evropského parlamentu a Rady (EU) 2016/679 ze dne 27. dubna 2016 o ochraně fyzických osob v souvislosti se zpracováním osobních údajů a o volném pohybu těchto údajů a o zrušení směrnice 95/46/ES (obecné nařízení o ochraně osobních údajů). In: Úřední věstník L 119. 4. 5. 2016, s. 1–88.
Labels
Addictology Allergology and clinical immunology Angiology Audiology Clinical biochemistry Dermatology & STDs Paediatric gastroenterology Paediatric surgery Paediatric cardiology Paediatric neurology Paediatric ENT Paediatric psychiatry Paediatric rheumatology Diabetology Pharmacy Vascular surgery Pain management Dental HygienistArticle was published in
Journal of Czech Physicians
Most read in this issue
- Shooting at the Faculty of Arts of Charles University and activation of the trauma plan at the First Surgical Clinic of the General Teaching Hospital
- Amiodarone-induced thyreopaties
- Clinical obesity – finally a change of paradigm? From BMI to chronic disease
- Comparison of performance between artificial intelligence and radiologists in detecting abnormalities on chest X-rays