Human Vs. Machine Eye for Chart Interpretation
Prerna Mishra, Urmila N. Shrawankar · 2022 IEEE Region 10 Symposium (TENSYMP) · 2022
Recent advances in machine learning have enabled the creation of models that can read, parse, and to a certain extent, reason about charts. These models are yet to reach up to human interpretation, but given the fast progressive technologies, these models can analyze, interpret and visualize chart images to some extent. Many people are acquainted with data visualization paradigms; still, estimating a users' capability to read complex charts or graphs is not easy. For designing an interpretation and visualization system for the non-specialist user, it becomes necessary to separate the potential efficiency of the system and the comprehension capability of the user. Complex data interpretation requires extraction, analysis, and manipulation of the data repeatedly, with the influence of multiple perceptions leveraging multiple interpretation tools. Moreover, interpreting charts requires the extraction of textual and graphical encodings from chart images. With the complex layout, the retrieval system fails to interpret data relationships. The paper discusses human and machine perception for understanding charts, and the societal need for chart interpretation. Visually impaired users rely more on summaries to understand images. However, neither images are described by the assistive readers, nor chart images are accompanied by any proper contextual messages. To overcome this issue, the paper also proposes a style-invariant automated encoding system that uses inter and intra-relationship of chart elements to interpret data encodings from various types of chart images.