NaroNet: Discovery of tumor microenvironment elements from highly multiplexed images
Keywords: 
Tumor microenvironment
Weakly supervised learning
Deep learning
Spatial biology
Multiplex imaging
Imaging mass cytometry
Cellular neighborhoods
Interpretable machine learning
Self supervised learning
Issue Date: 
2022
Publisher: 
Elsevier
ISSN: 
1361-8415
Note: 
This is an open access article under the CC BY license
Citation: 
Jiménez-Sanchez, D. (Daniel); Ariz, M. (Mikel); Chang, H. (Hang); et al. "NaroNet: Discovery of tumor microenvironment elements from highly multiplexed images". Medical Image Analysis. (78), 2022, 102384
Abstract
Understanding the spatial interactions between the elements of the tumor microenvironment -i.e. tumor cells. fibroblasts, immune cells- and how these interactions relate to the diagnosis or prognosis of a tu- mor is one of the goals of computational pathology. We present NaroNet, a deep learning framework that models the multi-scale tumor microenvironment from multiplex-stained cancer tissue images and pro- vides patient-level interpretable predictions using a seamless end-to-end learning pipeline. Trained only with multiplex-stained tissue images and their corresponding patient-level clinical labels, NaroNet un- supervisedly learns which cell phenotypes, cell neighborhoods, and neighborhood interactions have the highest influence to predict the correct label. To this end, NaroNet incorporates several novel and state-of- the-art deep learning techniques, such as patch-level contrastive learning, multi-level graph embeddings, a novel max-sum pooling operation, or a metric that quantifies the relevance that each microenvironment element has in the individual predictions. We validate NaroNet using synthetic data simulating multiplex- immunostained images where a patient label is artificially associated to the -adjustable- probabilistic inci- dence of different microenvironment elements. We then apply our model to two sets of images of human cancer tissues: 336 seven-color multiplex-immunostained images from 12 high-grade endometrial cancer patients; and 382 35-plex mass cytometry images from 215 breast cancer patients. In both synthetic and real datasets, NaroNet provides outstanding predictions of relevant clinical information while associating those predictions to the presence of specific microenvironment elements.

Files in This Item:
Thumbnail
File
1-s2.0-S1361841522000366-main.pdf
Description
Size
4.29 MB
Format
Adobe PDF


Items in Dadun are protected by copyright, with all rights reserved, unless otherwise indicated.