Lining Up the Pieces: Coregistration of Digital Pathology Images Is an Underused Technique With Great Promise in Advancing Computational Pathology
Jeffrey L. Fine · American Journal of Clinical Pathology · 2023
Digital pathology has been accelerating, with advances in computational pathology that move beyond traditional H&E and immunohistochemistry (IHC) images.1,2 Digital gross imaging has long been in routine use3 but seems overdue for disruption, especially with novel imaging techniques such as those in the article by Zhang et al4 in this issue of AJCP. Although the terminology seems complicated, the underlying technology is very closely related to existing digital photography and is not exotic; high dynamic range (HDR) photography is a common feature in current cell phones, and polarization is a traditional photography technique. Such images could be quickly and inexpensively actionable for many laboratories. Zhang et al4 present an image analysis application that is intended to help pathologists to make accurate and computer-assisted judgments about post neoadjuvant breast cancer cases. These are labor-intensive specimens, typically requiring a complex interplay of gross measurement, thorough microscopic tissue sampling, careful microscopic examination, and correlation with radiographic and surgical impressions. There are multiple arbitrary decisions both grossly and microscopically, including where and how much tissue to sample for histologic evaluation. A computer-assisted mechanism might be helpful in reducing interobserver variability and for accelerating workflow. This article presents such a system, and in addition to helping with sign-out, it also brings gross pathology back into the fold, in an era when many pathologists are relatively isolated from it. Zhang et al4 present compelling images that are coregistered, or directly stacked on top of one another. The resulting image stacks could also be considered a multidimensional data set, one that better connects the dots from radiology to gross pathology to microscopic pathology. Pathologists already engage in coregistration activity whenever they look at IHC stains, when they manually or mentally line up the stains with the H&E in order to find the area of interest. Breast pathologists who look at serial H&E levels also perform a form of manual coregistration, but in that instance they are doing it to create a three-dimensional mental picture, especially when atypical or papillary lesions are in the differential diagnosis. Coregistration is easiest when the images closely resemble one another, such as adjacent tissue sections or levels, but it is not strictly necessary as pathologists already understand about how tissue varies from slide to slide. The authors’ approach minimized variability, because the images were all captured on the same tissue piece at nearly the same time, which minimized differences in tissue shape. Furthermore, it is likely that when such tissue is submitted for histologic analysis, the resulting whole slide images (WSIs) will retain strong low-magnification resemblance to the gross images. This would facilitate coregistration of gross and microscopic images together as a single multidimensional digital data set. This may sound imposing, but pathologists may already do this in conventional microscope diagnosis, especially in breast resections that are serially sliced and submitted in a systematic fashion (ie, “serial sequential sampling”).5 Coregistration has long been discussed in digital pathology circles, and currently most modern digital pathology platforms provide some support for it, although it is usually not automatic and can be imperfect in practice.6 Typically, the pathologist carefully lines up WSIs that are then fixed into position for synced manual viewing (eg, moving one WSI will move all of the synced WSIs, facilitating looking at the “same” areas simultaneously). Automated coregistration is potentially a powerful tool and could lead to “virtual multiplexing”; one could imagine advanced spatial analytics that leveraged these techniques. It might be very powerful to be able to include gross pathology images in the “stack,” images such as those presented in this article,4 especially if automatic coregistration became more available. It is this author’s experience that many or most gross digital images are of little practical use beyond documentation or for demonstrating nonsubtle appearances. There is much image quality variability; as Zhang et al4 present, a controlled environment or specimen box is a superior approach that should improve consistency and also make the images more functional than with other gross imaging approaches. Furthermore, if engaging in additional effort to improve gross images, additional technical improvements (eg, polarized lenses or HDR photography) should also be incorporated. These approaches should make gross images much more relatable to the microscopic pathology, especially if the gross images can be directly coregistered with the resulting tissue pieces. Such an approach is also modular in the event that novel imaging techniques become available at a later time; these can be added into the image stacks and leverage existing digital workflows. This work shows parallels to efforts from the in vivo microscopy/ex vivo microscopy (EVM) community, which has worked to augment or replace traditional microscopy by imaging tissue directly without microscope slides.7 Most EVM work focuses on providing rapid intraoperative diagnoses or microscopy-guided tissue sampling for grossing, activities that indeed blur the line between gross and microscopic diagnosis. The report by Zhang et al4 is related to EVM, but it is also different because it is based on conventional specimen imaging, albeit with technical refinements—digital gross photographs and specimen radiographs, together with transmitted light photography. These are not exotic techniques, and while specimen radiography is potentially expensive, many pathology departments use it routinely, especially for breast resection specimens. Given this lower barrier to entry, it is possible that combining HDR dual-modal white light imaging (DWI) and specimen radiography may blaze trails that EVM can also use, again in a modular fashion. Significantly, the study by Zhang et al4 is also a potential bridge to intelligent software tools, such as the computer-assisted diagnosis for pathologists (pCAD) conceptual framework.8 One large elephant in the pathology informatics room is the issue of where this digital pathology work will take place. The currently evolving model resembles the radiology approach, with separate WSI platforms and LIS software, but it is likely that there will be a patchwork of different software modules at least in the beginning. The value of a high-level concept such as pCAD is that it provides a roadmap that helps pathology move to a more cohesive and better-integrated approach as computational pathology technology matures. As a subspecialty breast pathologist, it is not unusual for this author to receive cases with hundreds of tissue blocks in a single day when on a breast resection–only service. Extensive tissue sampling is extremely helpful, yet it represents a serious productivity issue for pathologists, histotechnologists, and pathologists’ assistants, not to mention technical cost. As with imaged Papanicolaou tests, it is very likely that future trusted artificial intelligence systems will someday screen these cases so that pathologists would review only key fields of view, but this approach still would produce too many low-yield WSIs that still must be processed, cut, stained, scanned, analyzed, possibly reviewed, and stored. Better would be to reduce overreliance on tissue sampling using novel gross imaging techniques such as this trailblazing application with image analysis and HDR-DWI.4 Once developed, such tools could serve as infrastructure examples that facilitate other efforts related to EVM, hyperplexed biomarkers (fluorescence based or otherwise), or even spatially coregistered genomic and imaging information. In summary, Zhang et al4 present an innovative take on what has been an underdeveloped area of digital pathology: gross specimen imaging. With enthusiastic imagination, we can see that this work could be extended to better integrate gross and radiographic specimen images into digital pathology workflow and thereby vastly improve rank-and-file pathologist performance, especially as new technologies emerge.