Deep learning has transformed medical image analysis by delivering clinically meaningful results on challengingproblems like prostate cancer detection and lung cancer screening. FDA approval of whole-slide digital pathologyimaging (WSIs) for primary diagnosis is further increasing interest, adoption, and investment in artificial intelli-gence (AI) technology for pathology. Learning from large medical images using patient-level labels (PLLs) hasbecome an active computational pathology research area. PLLs such as pathology diagnosis or clinical outcomesare generated through healthcare operations and are often readily available. In contrast to learning paradigmsthat depend on the expert annotation of images (e.g., delineating tumor regions) and are therefore time-intensiveand limited to smaller cohorts, training directly from WSIs using PLLs will allow the development of realistictraining datasets containing tens-of-thousands of subjects that can produce models with clinically-meaningful ac-curacy. GPU accelerators have played a significant role in advancing deep learning methods for computationalpathology tools. Machine Learning Frameworks (MLFs), e.g., Pytorch and TensorFlow, provide researchers withabstractions to quickly develop models that utilize GPUs. The evolution of GPUs and MLFs has been driven bythe analysis of small images, and so applying these tools directly to WSIs or other large medical images likevolumetric magnetic resonance or computed tomography is challenging. Adapting medical imaging problems tothe small image paradigm leads to many compromises resulting in suboptimal performance, increased imple-mentation effort, and increased software/design complexity (e.g., patch based techniques or multiple instancelearning). As a result, the development of scalable ML models from PLLs by directly processing WSI imagesthrough a deep learning pipeline is infeasible today on GPUs. Recent efforts that use unified GPU memory orstreaming approaches to overcome GPU memory limits and attempt to perform end-to-end training at WSI scalehave demonstrated superior performance to annotation or MIL. However, these approaches are either slow (dueto suboptimal data movement strategies), complex to adapt/use, or highly specific to a given network architecture(limiting the ability to develop and explore new architectures). More general-purpose, efficient, and user-friendlyframeworks are needed to allow the development of WSI scale deep learning. This project will develop a robust software framework to facilitate seamless development and use of scalableML models, without the imposition of any limits on the sizes of handled images, unhindered by the limited memorycapacity in GPUs. The proposed SSTEP (Seamless Scalable Tensor-Expression Execution via Partitioning) soft-ware framework will allow scalable and portable neural network models that directly process full high-resolutionimages of arbitrary size for training or inference, on any (multi) GPU platform. SSTEP will allow the developmentof novel deep learning paradigms that are purpose-built for medical applications, and will enable developers torapidly create and evaluate these tools using familiar MLFs - PyTorch or TensorFlow.
Public Health Relevance Statement: PUBLIC HEALTH RELEVANCE The proposed software will help developers overcome the limitations of current computing hardware to design more accurate deep learning models for use in clinical diagnostics. These models will be able to analyze very large digitized images of glass slides to aid pathologists in tasks like cancer detection. The ability to analyze these images in their entirety will accelerate the development of accurate AI based diagnostics.
Project Terms:
