Optimization of Scene and Material Parameters for the Generation of Synthetic Training Datasets for Machine Learning-Based Object Segmentation
Malte Nagel, Kolja Hedrich, Nils Melchert, Lennart Hinz, Eduard Reithmeier · Computers · 2025
Synthetic training data is often essential for neural-network-based segmentation when real datasets are difficult or impossible to obtain. Conventional synthetic data generation relies on manually selecting scene and material parameters. This can lead to poor performance because the optimal parameters are often non-intuitive and depend heavily on the specific use case and on the objects to be segmented. This study proposes a novel, automated optimization pipeline to improve the quality of synthetic datasets for specific object segmentation tasks. Synthetic datasets are generated by varying material and scene parameters with the BlenderProc framework. These parameters are optimized with the Optuna framework to maximize the average precision achieved by models trained on this data and validated using a small real dataset. After initial single-parameter studies and subsequent multidimensional optimization, optimal scene and material parameters are identified for each object. The results demonstrate the potential of this optimization pipeline to produce synthetic training datasets that enhance neural network performance for specific segmentation tasks, offering insights into the critical role of scene design and material selection in synthetic data generation.