OpenVNA: A Framework for Analyzing the Behavior of Multimodal Language Understanding System under Noisy Scenarios
Ziqi Yuan, Baozheng Zhang, Hua Xu, Zhiyun Liang, Kai Gao · 2024
We present OpenVNA, an open-source framework designed for analyzing the behavior of multimodal language understanding systems under noisy conditions.OpenVNA serves as an intuitive toolkit tailored for researchers, facilitating convenience batch-level robustness evaluation and on-the-fly instance-level demonstration.It primarily features a benchmark Python library for assessing global model robustness, offering high flexibility and extensibility, thereby enabling customization with user-defined noise types and models.Additionally, a GUI-based interface has been developed to intuitively analyze local model behavior.In this paper, we delineate the design principles and utilization of the created library and GUI-based web platform.