Machine Translation of Comics with Visual Reconstruction for Linguistic Accessibility
Thamer Horbylon Nascimento, Leonardo Souza Silva, Camila Horbylon, Diego Siqueira, Juliana Paula Félix, Renan Vinícius Aranha, Fabrizzio Alphonsus Alves de Melo Nunes Soares · 2025
This paper presents a computational approach for the machine translation of comic book texts, aiming to promote linguistic accessibility, particularly in emerging countries. The proposed system performs end-to-end processing: it detects text regions, extracts content using Optical Character Recognition (OCR), translates the text into the target language, and reinserts the translated content into the original image, preserving its visual structure. Built with open-source libraries, the system is lightweight and suitable for low-resource computational environments. The methodology was validated through an experiment involving 1,000 pages of comics in four languages—English, French, Spanish, and Japanese. Results demonstrated high accuracy in the OCR and translation stages for Latin-based languages, and satisfactory performance for Japanese, despite its right-to-left reading layout and ideographic characters. The tool also showed potential as an assistive reading solution, with applications in educational and inclusive contexts. This work contributes to the development of accessible technologies aligned with the United Nations Sustainable Development Goals (SDGs), particularly in promoting quality education and reducing inequalities.