Towards the Semantic Interpretation of Arbitrary Traffic Signs: Semantic Parsing for Action-oriented Signs
Sasha Strelnikoff, Jiejun Xu, Alireza Esna Ashari · 2023
The safe navigation of autonomous vehicles (AVs) is dependent on accurate and up-to-date information, typically assuming access to pre-generated, high-definition map data, which limits the utility of AVs in novel environments. To overcome this, traffic signs have been considered as a rich and reliable source of real-time data to aid AV operations, but existing research has focused primarily on the classification of standard traffic signs and sign text recognition. In this paper, we explore the problem of parsing the semantics of traffic signs in a manner which is useful for downstream control and navigation systems. In order to narrow our scope, we focus on action-oriented traffic signs. We start by introducing a flexible shallow parsing representation for action-oriented signs and subsequently outline a method for automated representation construction. This method combines rule-based parsing with neural-based parsing and incorporates domain-specific regularizing priors in order to handle semi-structured sign text and improve parsing accuracy. The proposed method is evaluated against a popular baseline and an ablation of our method. Our results demonstrate the potential of our approach to produce a rich intermediate representation of action-oriented sign text for AV systems to react appropriately to novel action-oriented signs.