Semantic-Aware Structure Preserving Image Filtering Techniques
Kunal Pradhan · 2025
Image filtering techniques play a crucial role in enhancing the quality of images.Digital images acquired by sensors often suffer from geometric, radiometric, andsensor-related distortions and noises. These imperfections can degrade image qual-ity substantially. In turn, the effectiveness of image processing and analysis meth-ods is closely tied to the quality of input images. Consequently, image filtering hasbecome an essential and indispensable step in many image processing and imageanalysis applications.There exist two approaches: spatial and frequency domain image filtering. Spa-tial domain filtering is inherently intuitive as it directly operates on image pixels,whereas frequency domain filtering works on signals derived from digital images.The spatial domain offers a practical advantage in image processing by enablingdirect pixel-based operations, which preserve structures and enhance specific re-gions without the artifacts that can arise from frequency domain transformations.This approach is computationally simpler, supporting real-time applications thatrequire fast processing. Additionally, it allows for flexible integration of semanticinformation, such as edges and textures, which are critical for tasks like objectdetection, segmentation, and classification, where maintaining spatial details isessential. Overall, the spatial domain is ideal for applications where clarity, pre-cision, and efficiency are prioritized. Spatial domain image filtering techniqueshave evolved from basic linear filters to advanced non-linear ones, including edge-preserving filters, adaptive filters, region-based structure-preserving filters, andmorphological filters. Over the past two decades, there has been a shift fromgradient-based edge-preserving filtering to adaptive window-based edge preservaion and region statistics-based structure preservation. These structure-preservingfiltering techniques aim not only to preserve individual edges but also to retainmeaningful structures while eliminating insignificant ones.The gradual developments of pixel-based filtering methods primarily rely onregion-based directional gradients (horizontal and vertical) and statistical mea-sures, such as total variation ( TV ), relative total variance ( RTV ), and regioncovariance, as descriptors of structures and textures. These methods often struggleto effectively preserve the edges of objects and perform poor smoothing for vary-ing scale textures/noises. Also, achieving optimal results with these approachesrequires meticulous parameter tuning, involving the selection of appropriate neigh-bor window, region window, or kernel functions.In recent times, the concept of semantic-aware structure-preserving filteringhas emerged. Defining semantically meaningful structures within an image is oneof the most challenging aspects of developing such filtering techniques. To addressthese challenges, in this thesis, we have proposed and developed a few robustsemantic-aware structure-preserving filtering techniques. The basic approach ofthese techniques begins by generating a semantic-aware edge map that leveragesthe semantic information from the input image. Using this edge map, an edge-aware adaptive filtering technique is then developed.As a first contribution of this thesis, a novel semantic-aware adaptive medianmorpho-filtering technique is proposed. First, it generates an edge-map by usingsemantic information extracted from the global and local morphological gradienthistograms obtained from the input image. Then, using the generated edge-mapa novel adaptive median morpho-filtering technique is proposed by defining a dy-namic window that avoids overlapping of textural and structural contents of theimage. Although the proposed technique provides satisfactory results for widevarieties of input images, the size of the window taken for considering seman-tic information is a critical parameter of this technique. The filtering result issignificantly dependent on this parameter.In order to reduce the limitation of the above filtering technique, in the second contribution, a completely different approach is proposed to generate the semanticedge-map of the input image. We exploit Jensen-Shannon Divergence ( JSD ) toincorporate semantic information into edge-map generation. Here, we present twoapproaches for generating semantic edge-maps using JSD. In the first approach,JSD is directly used to distinguish between structural edge pixels and non-edgepixels. Where as in the second approach, a set of novel features are proposed torepresent the pixels of the input image. These features are defined by exploitingJensen-Shannon ( JS ) divergence in such a way that it incorporates semanticinformation of the pixel for determining whether it is a structural edge pixel ornot. Then, projecting all the pixels into the feature space, a semantic-aware edge-map of the input image is generated by applying k-means clustering. Once theedge-map is obtained, the edge-aware adaptive recursive median filter proposedin the first contribution is used to generate the filtered image. Compared to theexisting state-of-the-art methods, the proposed method outperforms for a widevariety of images with the minimal fine tuning of its parameters within a fewdiscrete options.The third contribution of this thesis introduces a parameter-efficient filteringtechnique that builds on the semantic information developed in the first and sec-ond contributions. This novel approach generates a semantic edge map of theinput image by leveraging the morphological gradient distribution from the firstcontribution and the Jensen-Shannon divergence introduced in the second. Oncethe edge map is constructed, the edge-aware adaptive recursive median filter, alsopresented in the first contribution, is applied to produce the filtered image. No-tably, this technique requires only one parameter, manually defined within fourdiscrete options, making it efficient and straightforward to use. The effectivenessof the proposed method has been validated across a wide range of input images.While image filtering techniques are widely employed in various computergraphics applications, they are rarely leveraged to incorporate spatial informationfor image classification tasks. The fourth key contribution of this thesis addressesthis gap by demonstrating the effectiveness of our proposed filtering techniques in enhancing hyperspectral image (HSI) classification through spatial informationincorporation. Our filtering techniques are designed to preserve object structureseffectively while reducing noise and texture, thus enabling the filtered images toretain valuable spatial information.In this contribution, we first apply one of our proposed filtering techniquesto generate multiple filtered versions of the HSI, creating a filtered profile. Thisprofile provides spectral-spatial features for each pixel, which are then used asinput to a classifier for HSI classification. The effectiveness of our approach isvalidated by comparing it to several leading spectral-spatial HSI classificationmethods.Finally, we draw conclusions and discuss potential future advancements toextend this work further.Keywords: Image filtering, edge-aware filtering, structure-preservingfiltering, texture filtering, structure texture decomposition, semantic-aware image filtering, semantic image filtering