Transforming Semantic Segmentation into Instance Segmentation with a Guided U-Net
Roman Lavrynenko, Nataliya Ryabova · 2023
Traditional approaches to instance segmentation typically rely on the utilization of object detectors in the initial phase. In contrast, our approach leverages a semantic segmentation model as the foundational stage for instance segmentation. We introduce a novel “Guided U-Net” to extract instance segmentation masks from the output of the semantic segmentation phase. Inspired by educational strategies, we incorporate a “guide” mechanism akin to a teacher directing a student’s attention to specific regions of an image and prompting inquiries about the boundaries of the indicated instance.The Guided U-Net is trained using a triad of inputs: the original image, the semantic segmentation output from the primary stage, and the guide—a matrix containing a single white dot placed randomly within an instance. Through a comparison between the predicted mask and the corresponding ground truth mask, a loss value is computed. Remarkably, our approach capitalizes on a straightforward loss function, in contrast to contemporary models that rely on intricate loss functions requiring weight tuning. The proposed method shows promise, particularly in scenarios where a well-trained semantic segmentation model is already available and the need for instance-level detail arises.