Assessing fidelity in synthetic datasets: A multi-criteria combination methodology
Alexandra Duminil, Sio-Song Ieng, Dominique Gruyer · 2024
With the development of driving simulators, graphics engines and synthetic-to-real domain adaptation algorithms, synthetic datasets become increasingly more photo-realistic. The advancement of such dataset is crucial for advanced driving systems, particularly for training learning-based methods and validation. An important consideration is around the fidelity of synthetic datasets, particularly regarding their suitability for deep learning applications such as object detection or segmentation. However, quantifying fidelity poses a significant challenges. To address this gap, we propose a set of fidelity scores to quantify the level of fidelity of RGB images from these datasets. Through in-depth examination, we aim to reveal information about the texture patterns and high-frequency components that contribute to the objective perception of data realism in road scenes. Furthermore, a multi-criteria combination using belief theory is performed to merge these scores and give a global score involving the level of fidelity, the level of uncertainty on this decision, and the level of conflict between the scores.