Evaluating Generalizability of Deep Learning Models Using Indian-COVID-19 CT Dataset

DOI

10.2991/978-94-6463-164-7_3How to use a DOI?

Keywords

Convolutional Neural Network; Computed Tomography Scans; COVID-19; Deep Learning

Abstract

Computer tomography (CT) have been routinely used for the diagnosis of lung diseases and recently, during the pandemic, for detecting the infectivity and severity of COVID-19 disease. One of the major concerns in using machine learning (ML) approaches for automatic processing of CT scan images in clinical setting is that these methods are trained on limited and biased subsets of publicly available COVID-19 data. This has raised concerns regarding the generalizability of these models on external datasets, not seen by the model during training. To address some of these issues, in this work CT scan images from confirmed COVID-19 data obtained from one of the largest public repositories, COVIDx CT 2A were used for training and internal validation of machine learning models. For the external validation we generated Indian-COVID-19 CT dataset, an open-source repository containing 3D CT volumes and 12096 chest CT images from 288 COVID-19 patients from India. Comparative performance evaluation of four state-of-the-art machine learning models, viz., a lightweight convolutional neural network (CNN), and three other CNN based deep learning (DL) models such as VGG-16, ResNet-50 and Inception-v3 in classifying CT images into three classes, viz., normal, non-covid pneumonia, and COVID-19 is carried out on these two datasets. Our analysis showed that the performance of all the models is comparable on the hold-out COVIDx CT 2A test set with 90%–99% accuracies (96% for CNN), while on the external Indian-COVID-19 CT dataset a drop in the performance is observed for all the models (8%–19%). The traditional machine learning model, CNN performed the best on the external dataset (accuracy 88%) in comparison to the deep learning models, indicating that a lightweight CNN is better generalizable on unseen data. The data and code are made available at https://github.com/aleesuss/c19.

Copyright

© 2023 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - S. Suba
AU  - Nita Parekh
AU  - Ramesh Loganathan
AU  - Vikram Pudi
AU  - Chinnababu Sunkavalli
PY  - 2023
DA  - 2023/06/05
TI  - Evaluating Generalizability of Deep Learning Models Using Indian-COVID-19 CT Dataset
BT  - Proceedings of the Joint 3rd International Conference on Bioinformatics and Data Science (ICBDS 2022)
PB  - Atlantis Press
SP  - 15
EP  - 28
SN  - 2468-5739
UR  - https://doi.org/10.2991/978-94-6463-164-7_3
DO  - 10.2991/978-94-6463-164-7_3
ID  - Suba2023
ER  -