Jingwen Ren, Zifan Zhu, Xin Bai, Jiyoun Seo, Tsung-Yu Lu and Chien-Sheng Yang
Instructor: Joseph Lim
Early Cancer Detection:
Prediction of Clinical Diagnosis Using HD-CTC
Circulating tumor cells (CTCs) are cells that detach from primary or secondary tumors and enter the circulatory system. As such, they show promise as a noninvasive liquid biopsy technique. CTCs have been validated as prognostic and predictive biomarkers in many different cancers including breast, lung, prostate, and colorectal cancer.
To detect CTCs, Dr. Peter Kuhn group has previously reported that the “High Definition” Circulating Tumor Cell (HD-CTC) platform, which takes advantage of well-established cell markers (DAPI, CK and CD45) and rapid, automated fluorescence microscopy.
Our goal is to build a model that can classify candidate cells as CTCs or normal cells by using their characteristics from the HD-CTC images. This will improve the speed of the early detection of cancer by using only the fluorescence staining images of CTCs, without further genomic analysis.
Cost-effective and Non-invasive Early Cancer Detection
Biopsies have been an integral part of cancer care and early detection of cancer for decades [6]. In undiagnosed patients, oncologists can examine suspicious observations and determine if the patients are benign or malignant.
Though highly informative, traditional tumor biopsies have several drawbacks. Due to their inherently invasive nature, tumor biopsies can induce risks to the patient, such as bleeding and infection. Additionally, studies have shown that biopsies are not always representative of the entire tumor. Because many tumors are heterogeneous, different sites within the tumor can have different levels of gene expression and metastatic potential, and a tissue biopsy may not be indicative of all tumor phenotypes [7]. Furthermore, some tumors are not easily accessible, and performing a biopsy may not be possible.
Here, we are trying to introduce and focus on liquid biopsy, which is often performed by using a simple blood draw and the innovative solution to cancer detection. While they are not yet ready to entirely replace their competition, traditional solid tissue biopsies, we can see its strong possibility and advantages on ease and convenience.
Physician's Hope -
Circulating Tumor Cells (CTC)
CTCs are described as cells shed by a primary tumor into vasculature and they keep circulating in the blood stream of cancer patients [8]. CTCs are known to be circulating in the body fluids before they metastasize to various parts of the body even in primary stages of the disease [6]. Previous studies have shown that CTCs could offer prediction on prognosis and overall survival of cancer patients. Karin et al.[8] showed that CTC screening provides a highly sensitive biomarker for the early detection of cancer.
However, they are not easily identified, as they are present in very small numbers; account for only 1 or fewer in every 10,000-1,000,000 peripheral blood mononuclear cells [9]. Due to their rarity, it is challenging to accurately classify them out of other cells in the blood. An additional procedure is required to verify their genetic profiles since CTCs usually display genetic alteration while other normal blood cells have no alteration.
The Messages from Tumor Cells -
Copy Number Variation (CNV)
Genomic analyses of CTCs is also known that it can offer highly reproducible CNV patterns from cancer patients which can be leveraged to a prediction on prognosis and diagnosis. Ni et al. [9] surveyed the CTC CNVs
from a small-cell lung cancer (SCLC) patient during sequential chemotherapy treatment and observed that the evolution of CNV was consistent along the therapeutic stage.
However, because of the certain rarity on detecting CTCs, there is a challenge on their detection to distinguish them from the other rare cells. Therefore the genomic analysis on CTCs can be applied not only for an investigation on their genetic dynamics but also for a classification clue showing the certain alterations. For this project, a single-cell whole-genome amplification (WGA) method was applied to capture the genomic alterations, especially the CNVs on CTCs. Our goal is to build a model to predict whether the candidate rare cell is CTC having certain genomic alterations or not.
Our Goal -
HD-CTC Image Classification
Our goal is to build a model that can classify the single cells from the liquid biopsy whether they are cancer cells or normal cells by using only the fluorescence staining images of them. Even though the cancer cells and normal cells might have distinct differences in their morphological characteristics, there are certain populations of the cells that are hard to distinguish them by the human eye. As we discussed above, that was the main reason for the further genomic analysis to see the genomic alterations on each cell. However, we believe that our model might learn the differences that were not able to be caught by the human eye and might be available to distinguish the cancer cells from the normal cells by using only the images.
Datasets
There are a total of 3,987 single cell images: 2815 cells from breast cancer patients and 1172 cells from prostate cancer patients. Each cell has an image with three staining channels and a copy number variation (CNV) statistic. The three channels are: 4’,6-diamidino-2-phenylindole (DAPI) to capture cell nucleus, cytokeratin-phycoerythrin (CK) to capture epithelial characteristic which should be positive for CTCs, and CD45-allophycocyanin (CD45) antibodies which should be negative for CTCs. The 3,987 CNV statistic is computed from the dispersion of number of the aligned reads per genomic position across the whole genome. An empirical threshold is then used to label the cells as either cancerous or normal.
Convolutional Neural Network
Traditional convolutional neural network (CNN) is applied to CTC image based cancer diagnosis. Our CNN model is comprised of: (1) image representation via convolution and pooling, (2) regularization techniques such as batch normalization and dropout, (3) automatic cancer detection via softmax classifier.
For each particular cancer type (breast or prostate) as well as the combined dataset for both cancer types, we input the training images to our CNN model, train a binary classification model, and test on the independent testing set. The model training and testing are repeated for 3 times for each dataset to consider the stochastic behavior of model training. The mean accuracy of 3 different rounds of a certain dataset is summarized in the following table. The prediction accuracy for different cancer types are found to be similar. Therefore, we will simply pool breast and prostate cancer datasets together and test on the combined dataset in the "Data Augmentation"section.
Capsule Network
One issue being addressed for traditional CNN network is that the model is insensitive to position of features. For example in the case of human face recognition [Figure below, left], if a kernel is trained to learn the feature of an eye, it will output strong signals if there is an eye at that position. With this mode of operation, it is very likely that a picture with randomly-distributed facial features [Figure below, right] will be predicted as a real human face. However, it is not the real case. To overcome this issue, CNN capsule network is designed to capture position-specific features and their higher-level interactions [5]. The rationale behind choosing this model is that our input data is preprocessed such that the cell of interest is centered in each image. It is therefore desirable to train a model that focuses on the features of center cells, which aligns with the spirit of CNN capsule network.
An example of CNN capsule network in our implementation comprises two layers of 16 3x3 kernels, an average pooling layer with 2x2 window, two layers of 64 3x3 kernels and finally a 2x16 capsule layer.
The model shows similar test accuracies for breast cancer (B), prostate cancer (P) and combined (C) datasets.
Residual Network
We also evaluated the feasibility of using a deeper network for our problem in hope of better accuracy. We choose residual networks proposed in [3], which has been demonstrated to achieve very high accuracy on the CIFAR dataset. In the deep neural networks, there is a limit to the number of layers increased that improves accuracy since increasing number of layers will result in vanishing gradients. Furthermore, if we keep increasing the number of layers, the accuracy will start to saturate at one point and eventually degrade (see the image below). It is known as the degradation problem in practice.
To overcome vanishing gradient problem and degradation problem, [3] introduces residual blocks (see the image below) which allow training of deeper networks. The main idea of residual blocks is to skip the training of few layers using skip-connections or residual connections. By such skip connections, residual networks essentially learn the difference between inputs and outputs (residual) instead of simply learning the outputs.
ResNet20 is applied to CTC image-based cancer diagnosis. Classification accuracies of 0.76, 0.69 and 0.77 can be achieved for breast, prostate and combined datasets, respectively. Severe over-fitting was observed in our training since the dataset size is limited. Therefore, the power of residual networks is not as comparable to as described for CIFAR datasets (91.25% accuracy by ResNet20).
Does the Models Outperform a Human Expert?
Since our CTC image datasets are relatively rare in bioinformatic research, no previous baseline is available for this problem. We seek to first establish the human performance for this problem. Luckily, one of our team member is from Dr. Kuhn's group and can be considered as a human expert in this research area. Using the same test sets, we found that human performance is sightly lower than our prediction models. This finding also indicates that our model has the potential to be integrated within a high-throughput analysis pipeline.
MODEL INTERPRETABILITY -
ANALYSIS OF ACTIVATION MAP
To understand the source of performance difference between humans and machines, we analyze the filters to identify underlying patterns recognized by the models yet missed by a human expert. First, we visualize the weights of filters in the first layer since they have the same number of channels as the original images. However, we cannot identify any specific patterns from these images.
Learned weights from the first layers of filters (held in hand)
Then we analyze the activation map of each filter to identify what input image can maximize the output for that filter. By running gradient ascent with respect to inputs, we found that the first two layers tend to extract colors in the image while the deeper layers tend to extract specific patterns in the image regardless the size of the model.
In particular, we found that the model can extract nucleus-like patterns [Figure below, 1st row] which is a necessary feature for cancer cells. The model also distinguishes between non-colocalized and colocalized signals of nuclei and receptors, which suggests the presence of a cancer cell for the latter one [Figure below, 2nd and 3rd rows]. Among all the reasons that make a human expert fail to classify an image, the most common one is the fluctuation of the absolute intensity from each channel and the relative intensity between channels, which corresponds to the hue of the image. We found that the model also captures this by learning to extract stripes with different hues [Figure below, 4th row].
Nucleus only
Non-colocalized
Colocalized
Hue
1000 Images Project !? -
Overcoming Dataset Size Issue Through Data Augmentation
Due to small dataset size, various data augmentation techniques like GAN, VAE and SSGAN are applied. In GAN and VAE, we generate thousands of images that mimic the original ones and then use old models to perform cancer detection based on both original and generated images. In SSGAN, however, the image generation and cancer detection are synthesized in one model. With SSGAN, combined accuracy increases to 0.81.
Visualization of the original images and the generated images
We show original images and the generated images from VAE, GAN and SSGAN in below. As we can see, CNV positive original images generally look more red comparing to CNV negative original images, while CNV negative original images have more green part. The images generated from VAE seem to be able to capture the characteristics of CNV positive images, but not CNV negative images, because there are no green parts at all in CNV negative images generated by VAE. The images generated by GAN fail to capture the shape of center cells, the silhouettes of which are quite blurred. For images generated by SSGAN, we mix CNV positive generated images and CNV negative ones together in the below display. SSGAN does a better job to capture the characteristics of CNV positive and negative original images. Also the shapes and positions of center cells look more random in images generated from SSGAN than ones generated from VAE.
As a conclusion, we think SSGAN works better in generating images that mimic the original ones than VAE and GAN.
Prediction accuracies of different data augmentation methods: VAE, GAN and SSGAN
With the images generated by different data augmentation methods, we could enlarge our training set, use the same CNN model described before, and test on the same testing set. We only consider the combined dataset in this section as previous results already showed the minimal difference in terms of prediction between breast and prostate cancer. The prediction accuracy output by CNN and adding the generated images to the training set is summarized in the table below.
As shown in the table, we can draw several conclusions. First, the performance of VAE and GAN are similar to the traditional CNN model without data augmentation for the original threshold. Second, SSGAN has the best performance for both original and tight thresholds. That is probably because the images generated by SSGAN have higher quality and are more similar to the real images.
Towards a Clinically Competitive Prediction Model -
Data Filtering by Copy Number Variability
Biomedical data are usually noisy and sometimes mis-labeled. We discuss this issue in this section and aim for a more competitive and robust prediction model. Since the model only display modest prediction accuracy so far, we reasoned that the subset of cells of which the CNV statistic is close to the decision boundary do not show enough similarity with either normal cells or with cancer cells. It is therefore desirable to filter datasets based on a pair of upper and lower quantiles of CNV statistic distribution instead of using simply one threshold to label data at each side of the distribution as either cancer or normal cell.
After training on the filtered datasets with the same model architecture, we observed a significant increase in test accuracies (7-12%) for all of the previous models. It also becomes clearer that our models outperforms a human expert by 3-6% without data augmentation and 12% with data augmentation.
The Future of Early Cancer Detection
In this project, We validated that HD-CTC images are highly predictive for the presence of cancer cells. By coupling SSGAN with deep neural networks to overcome small dataset issue, our model accuracy can achieve 92% and exceed human performance by 12%. Given that HD-CTCs can be obtained from liquid biopsy without the need of solid biopsy which is less cost-efficient and more invasive, early cancer detection based on HD-CTC images has the potential to be incorporated into clinical procedures. Nonetheless, further studies on the precision and recall of cancer diagnosis based on a larger set of HD-CTC images are needed to evaluate the feasibility. It is also unknown whether HD-CTCs from other cancer types also demonstrate strong predictive power for the presence of cancer cells.
References
[1] Ried, Karin and Eng, Peter and Sali, Avni. "Screening for circulating tumour cells allows early detection of cancer and monitoring of treatment effectiveness: an observational study" Asian Pacific journal of cancer prevention: APJCP. 2017.
[2] Ni, Xiaohui and Zhuo, Minglei and Su, Zhe and Duan, Jianchun and Gao, Yan and Wang, Zhijie and Zong, Chenghang and Bai, Hua and Chapman, Alec R and Zhao, Jun and others. "Reproducible copy number variation patterns among single circulating tumor cells of lung cancer patients" Proceedings of the National Academy of Sciences. 2013.
[3] He, Kaiming, et al. "Deep residual learning for image recognition."Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[4] Salimans, Tim and Goodfellow, Ian and Zaremba, Wojciech and Cheung, Vicki and Radford, Alec and Chen, Xi. "Improved techniques for training gans." Advances in neural information processing systems. 2016.
[5] Sabour, Sara and Frosst, Nicholas and Hinton, Geoffrey E. "Dynamic Routing Between Capsules." Advances in Neural Information Processing Systems 30. 2017.
[6] Barriere G, Fici P, Gallerani G, et al. Circulating tumor cells and epithelial, mesenchymal and stemness markers: characterization of cell subpopulations. Ann Transl Med. 2014.
[7] Cristofanilli M, Budd GT, Ellis MJ, et al. Circulating tumor cells, disease progression, and survival in metastatic breast cancer. N Engl J Med. 2004;351:781–91.
[8] Ried, Karin, Peter Eng, and Avni Sali. "Screening for circulating tumour cells allows early detection of cancer and monitoring of treatment effectiveness: an observational study." Asian Pacific journal of cancer prevention: APJCP 18.8 (2017): 2275.
[9] Ni, Xiaohui, et al. "Reproducible copy number variation patterns among single circulating tumor cells of lung cancer patients." Proceedings of the National Academy of Sciences 110.52 (2013): 21083-21088.