Visual recognition and understanding of Gastrointestinal diseases

The use of Deep learning methods for medical diagnostic imaging can reduce delays in diagnosis and give improved accuracy rates over other analysis techniques. Learning from different areas of an image, or looking for similar patterns in new images, allows for the development of potential classification or clustering models. Techniques like these provide state of the art solutions to detecting tissue anomalies. In this project, we explore state of the art deep learning architectures used for the visual recognition of natural images and assess their applicability in medical image analysis of digitized human gastrointestinal biopsy slides. Pathology has played an essential role in diagnosing gastrointestinal disorders. However, errors can occur due to complex systems, time constraints and variable inputs. This can be further complicated when the biopsy images share histological features. Computational methods have the potential to address these challenges. In light of this, developing assistive computational methods can help mitigate said errors. The goal of applying computational methods in identifying diseases is for developing fast, reproducible and reasonably accurate methods that can be easily standardized.

Interpretation of Deep Learning Results

The interpretation of clinical biopsy images for disease diagnoses can be challenging when clinicians are faced with distinguishing between distinct but related conditions. Recently, increasing attention has been paid to methods in artificial intelligence that help clinicians to translate big data into accurate and quantitative diagnostics. In contrast, deep learning or a convolutional neural network (CNN) is a form of artificial intelligence that includes machine learning techniques designed to process data and interpret it (eg, by detecting and segmenting multiple pixel intensities within a single image and labeling features at a pixel-by-pixel level).

Computational image analysis to address staining color bias in duodenal biopsies

Hematoxylin and Eosin (H&E) stains are the most widely used stains in medical diagnosis. Digital images used in computational pathology may contain significant variations in H&E staining due to a number of factors. These factors include differences in tissue preparation, raw materials, manufacturing protocol, and digital scanners. The variations of stain color in groups of images can create a bias in deep learning models such as Convolutional Neural Networks. Color correction techniques are applied to H&E stained images to attempt to remove the potential biases from staining variations. In this project, we compare the performance of three different color normalization techniques when applied to a deep learning classification model. The techniques tested are structure-preserving color normalization, a non-linear mapping approach to stain normalization, and color balancing. The classification dataset consists of duodenal biopsy images originating from three different tissue states: Celiac Disease, Environmental Enteropathy, and histologically Normal. After pre-processing images with each color correction technique, the images are classified using an equivalent training set and deep learning architecture.

Representation of pre and post color normalization biopsy images (move slider horizontally to visualize pre and post)

Grant Support


Computational Characterization of Environmental Enteropathy (EE) K23 DK117061-01A1 – Mentored Patient-Oriented Research Career Development Award, National Institute of Health (NIH)
Deep Learning for Automated Identification and Classification of Gastrointestinal Landmarks in Video Capsule Endoscopies Engineering-in-Medicine seed grant, Center for Engineering in Medicine, UVa
Modeling Crohn’s Disease using Machine Learning: Image Analysis and Multiomics Litwin IBD Pioneers Program, Crohn's & Colitis Foundation
EEDBI Consortium Morphometry Project, Bill & Melinda Gates Foundation

Ivy Foundation

Ivy Foundation COVID-19 Translational Research Fund, Ivy Foundation Epidemiologic Modeling, Public Health, Surveillance and Sewershed Monitoring to Predict Surges in the COVID-19 Pandemic


Computational Modeling of Intestinal Mucosa: Image Analysis and Multiomics Translational Health Research Institute of Virginia (THRIV) Scholars Award, University of Virginia, Career Development Award
Computational Imaging to Predict Intestinal Mucosal Alterations in Children in Virginia Engineering-in-Medicine seed grant, Center for Engineering in Medicine, University of Virginia