Hierarchical Image-Region Labeling via Structured Learning
Julian McAuley, Teófilo Emidio de Campos, Gabriela Csurka, Florent Perronnin · 2009
We present a graphical model that encodes hierarchical constraints for classifying image regions at multiple scales. We show that inference can be performed efficiently and exactly, rendering it amenable to structured learning. Our model is parametrised using the outputs of a series of first-order classifiers, meaning that it learns which classifiers are useful at different scales, as well as the relationships between classifiers across scales. Example results Correct labeling, using bounding-boxes from VOC2007 (1 − ∆ = 1): Baseline, using no second-order features (1 − ∆ = 0.566): Our model The ‘nodes ’ of our graphical model correspond to overlapping image regions: Second-order features without learning (1 − ∆ = 0.551): Learning of all features (1 − ∆ = 0.770): Colour-code for labels: Edges are formed by connecting nodes at different scales: we connect two nodes precisely when the corresponding image regions overlap at adjacent scales, so that our graphical model forms a quad-tree. First-order (node) features Our image features are based on those from [2], in which image-level, region-level, and patch-level classifiers are proposed. We use all classifiers at all scales (Pr,label is the probability that the region r is labeled label): Φ nodes (r, label) = (0,..., P 1 r,label,..., 0)... (0,..., P features for first classifier n r,label,..., 0) features for nth classifier Thus we learn which classifiers are useful at which scales. Hierarchical constraints We want to ban inconsistent assignments at different scales: aeroplane