Last year, we introduced an AI algorithm that predicts proliferation of breast cancer, the first attempt to automate a specific pathologic task from end to end. Today, we introduce an update to our pathology AI, which now supports automated detection and stage assessment of breast cancer metastases in lymph nodes.


When cancer is first diagnosed, the first and most important step is staging of the cancer, i.e. assessment of how far advanced the cancer is. The TNM staging system, the most commonly used system, categorizes the cancer by local growth/invasion (T-stage), spread to regional lymph nodes (N-stage), and presence of metastasis (M-stage). Invasion to lymph nodes, highly predictive of recurrence (hence, recommended for aggressive treatment), is evaluated by pathologists (pN-stage) via detection of tumor lesions in lymph node histology slides from surgically resected tissue.

This diagnostic procedure is prone to misinterpretation and would normally require extensive time by pathologists due to the sheer volume of data that needs thorough review, approximately 200,000 × 100,000 pixels on the highest resolution level. Furthermore, multiple slides need to be reviewed to properly determine the pN-stage.

스크린샷 2017-09-02 오후 2.36.52
Automated pN-stage prediction framework

We applied our deep learning technology to develop a highly accurate pN-stage prediction algorithm, which integrates the detection and classification of metastases in multiple lymph node histological slides into one clinical outcome: the pN-stage. Our algorithm has the potential to significantly elevate the efficiency and diagnostic accuracy of pathologists for one of the most critical diagnostic process of breast cancer.

We used lymph node histological images from the Camelyon Challenge 17 dataset to build an algorithm that predicts pN-stage, i.e. whether the breast cancer has spread to the lymph nodes or not. This involves analysis of not only individual slides, but multiple slides in aggregate from patients to predict the overall pN-stage of each patient.


In our efforts in developing automatic pN-stage prediction system on whole slide images, many technical challenges were encountered, as described below:

Tissue region extraction: Accurate tissue region extraction algorithms can save computation time and reduce false positives from noisy background area. However, we observed that metastatic regions are commonly located in the edge of the tissue regions, which meant careful tissue region extraction methods were needed. We experimented various methods and maximized metastatic regions’ sensitivity from extracted tissue regions. Moreover, we filtered annotated metastatic regions that included extraneous non-tumor areas such as that of fat tissue to obtain less noisy training patches.

Class imbalance and slide variation: Various factors such as different slide scanners, different scanning settings, difference tissue staining conditions, etc, attributed to a large slide-level variability. In addition, the areas corresponding to tumor regions often covered only a minor proportion of the total slide area, contributing to a large patch-level imbalance. To solve these problems, we carefully designed patch sampling procedures to balance slide-level and patch-level properties, as well as extensive color augmentation.

Multi-center variation: The main datasets used, the Camelyon16 dataset and the Camelyon17 dataset were collected from different medical centers, susceptible to unexplained bias and confounding if used without proper adjustment. We used the two datasets assuming difference in data distribution: we trained the initial CNN model with a union of Camelyon16 and Camelyon17 datasets to maximize regularization effects, and then fine-tuned on the Camelyon17 train set only to fit a target data distribution.

Post-processing: In clinical practice, pathologists determine categories of lymph node metastasis by evaluating the size of metastatic lesions. To build an automatic lymph node metastasis classification system, we explored various approaches to find effective features extracted from tumor probability heatmap on whole slide images. By investigating effective features from approximately 1,500 variations, we found useful features including maximum length of metastases, maximum tumor probability score, tumor/non-tumor ratio, etc.


confusion matrix
Slide-level confusion matrix on Camelyon17 test set

Our algorithm shows the highest performance to find pN-stages from Camelyon17 dataset. The prediction score (in Kohen’s Kappa) has reached 92% in its current version, which represents a highly competitive performance. (Link)

Furthermore, the performance level of our algorithm not only exceeds that of currently leading approaches but also significantly reduced false-negative results. This is remarkable from a clinical perspective, as false-negative results are most critical, likely to affect patient survival due to consequent delay in diagnosis and appropriate timely treatment.

We have presented you with an overall description of our latest state-of-the-art AI algorithm in pathology. Although our efforts so far have been met with success, we believe this is just the beginning of this new era of computational pathology. Once the technological developments are clinically validated, there will be new powerful applications that will transform the clinical world, bringing unprecedented value to patients worldwide.

Posted by:paengs

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s