Meeting Report: The International Workshop on Harmonization and Standardization of Digital Pathology Image, Held on April 4, 2019 in Tokyo
Hiroshi Yoshida, Hideo Yokota, Rajendra Pal Singh, Tomoharu Kiyuna, Masahiro Yamaguchi, Susumu Kikuchi, Yukako Yagi, Atsushi Ochiai · Pathobiology · 2019
The International Workshop on Harmonization and Standardization of Digital Pathology Image (DPI) was held on April 4, 2019, at the National Cancer Center (NCC), Tokyo, Japan. Experts on artificial intelligence (AI)-based DPI analysis from the USA and Japan discussed problems relating to the implementation of AI-aided applications into real-world pathology practice. The participants included representatives of the Japan Agency of Medical Research and Development (AMED) and the Japanese Society of Pathology. During the meeting, standardization in color of DPI and DPI scanners was repeatedly demanded by various experts. This meeting report provides the summaries of presentations by the experts and introduces the consensus of this workshop.In the opening remarks, Dr. Atsushi Ochiai (NCC), the chairman, introduced his recent work on an AI-aided pathological diagnosis system based on gastrointestinal biopsy specimens [1, 2] and raised two critical issues to be solved for improving the reproducibility of DPI analysis. First, differences in staining procedures would result in different colors of the same specimen. Second, differences in DPI scanners would result in different digital images of the same slide. He demonstrated that these differences could reduce the accuracy and reproducibility of the AI-aided applications. The chairman opened this workshop by asking: “What kind of standardization should be achieved for implementing AI-aided systems into real-world pathology practice?”Dr. Hideo Yokota (RIKEN) delivered a presentation entitled “Development of AI Systems to Assist Image Diagnosis.” With the progress of machine learning technology, region extraction and object recognition for images have become possible. In particular, deep learning enables the analysis of several images with the same criteria. Furthermore, this method has succeeded in recognizing an object by determining the object features that a human is unaware of. Currently, there is a demand for the development of a diagnosis assistance system for medical images using this technology. In medical image analysis, “machine learning techniques for a small number of images” and “image quality variation due to shooting conditions” are problems to be addressed. We believe that these problems can be solved by the “construction of an image database,” “standardization of imaging methods,” and “standardization of sample processing procedures.”Prof. Rajendra Singh (Mt. Sinai School of Medicine) delivered a presentation, entitled “The Promise of AI and Digital Pathology – Hype or Real?” The key to building true clinically relevant models and algorithms that can predict patient outcomes, management, or prognosis is having access to a large amount of patient data. Gaining access to high-quality big data sources, especially open access, will require shared data governance, accuracy, and dependability. Open-access platforms with deidentification or anonymization will need to obey these principles to support such deliverables. Web-based platforms will enable collaborative annotations to be made on the data, which will also need to be verified, to produce viable models for real clinical practice.Dr. Tomoharu Kiyuna (NEC Corporation) delivered a presentation entitled “On the Stability of an AI-Based Cancer Detection System and Its Influencing Factors.” He reviewed an AI-based cancer detection process and pointed out that little attention has been paid to the stability of AI-based histological diagnosis. The stability of the AI output, i.e., fluctuation of output values and reproducibility, can be affected by several factors, such as the specimen staining quality and imaging process characteristics (focusing, light source, etc.). The instability arises not only from the input image but also from the analysis algorithm. As long as the AI decision process is based on some “threshold” of the values obtained by evaluating the “cancerous-ness” of the histopathological image, it is inevitable that the final decision fluctuates near the decision boundary.Prof. Masahiro Yamaguchi (Tokyo Institute of Technology) delivered a talk on “Standardization of Color in Pathology Image Analysis: Its Importance and Challenges.” Since color variation is caused by the staining and scanning processes, the color variation issues must be addressed in digital pathology. Previously, we developed a whole-slide imaging (WSI) image analysis system that utilized machine learning, in which the color correction module played an important role. Nevertheless, there was an argument that color variation can be learned by AI. If the purpose of AI is to only make a decision based on the visual observation that is currently made by pathologists, then AI might deal with the color variation. However, for AI to contribute to the progress of pathology and the well-being of patients, it is necessary to measure histopathologic features and analyze the disease characteristics based on quantitative measurement results. Without color standardization, the measurement results vary depending on single-slide scanners. This implies that quantitative measurement is not possible. We conducted preliminary experiments for evaluating the effectiveness of color standardization and showed that color correction enables device-independent feature measurement, although the issue of image sharpness will need to be addressed.Dr. Yukako Yagi (Memorial Sloan Kettering Cancer Center) talked about “Practical Standardization in Digital and Computational Pathology.” Color constancy is still an issue in the WSI system [3]. Different scanner systems, even the same product, produced the colors of a histological slide differently, which could influence the AI system diagnosis and results. This was due to the inconsistent color and quality of WSIs, the cause for which is unknown. Therefore, a color correction scheme was implemented using a color calibration slide made of ordinary glass with color patches mounted on it (Fig. 1) [4]. The inherent spectral colors of these patches along with their scanned colors were used to derive a color correction matrix, the coefficients of which were used to convert the pixels’ colors to their target colors. The results demonstrated previously that the slide color variations in the images produced by different WSI scanners were effectively normalized. If this scheme becomes publicly available, it is possible to standardize all WSIs from different scanners and different institutions, which will improve the quality of the AI system.Dr. Susumu Kikuchi (Olympus Corporation) delivered a presentation entitled “Technical Directions to Standardize Digital Pathology Images.” Image quality variations caused by unreliable digital scanning affect the performance of pathological AI applications, because the AI software supervised by pathologists using normal quality images cannot make precise decisions for such degraded images. Particularly, focusing and color representation are important factors for digital image scanning to maintain AI performance. Therefore, standardization of the DPI quality is necessary. The software for image quality evaluation and criteria for evaluation scores acceptable for AI applications can be provided as the standard, which will be used to eliminate degraded images. If these standards are installed into digital image scanners, the image quality can be controlled when scanning. In the next stage, hopefully, the quality of the slides will also be standardized. The digital pathology world can be established based on these image quality managements and developed through open-minded collaborations.Dr. Makoto Suematsu, President of AMED, on behalf of his organization, provided an update on AMED’s activities, challenges, and accomplishments in global data sharing in medical research. He emphasized the importance and power of global data sharing and stated that development of an efficient system for data sharing and protection of personal information should be considered appropriately.In conclusion, this report provides valuable insight into preanalytical issues such as color variation and imaging process characteristics which prevent the implementation of AI-aided applications into real-world pathology practice. Harmonization and standardization of pathological images in these points should be immediately implemented in collaboration with various stakeholders in the Reiwa, the name of Japan’s new era, meaning “beautiful harmony.”We would like to thank all the participants of this international workshop.The authors have no conflicts of interest to declare.The conference was funded by the National Cancer Center Research Development Fund (No. 29-A-5) and supported by AMED.All the authors participated in the workshop, prepared this meeting report, and approved the final version of the manuscript.