In-car damage dirt and stain estimation with RGB images

Abstract

Shared autonomous vehicles (SAV) numbers are going to increase over the next years. The absence of human driver will create a new paradigm for in-car safety. This paper addresses the problem, presenting a monitoring system capable of estimating the state of the car interior, namely the presence of damage, dirt and stains. We propose the use of Semantic Segmentation methods to perform appropriate pixel-wise classification of certain textures found in the car's cabin as defect classes. Two methods, U-Net and DeepLabV3+, were trained and tested for different hiper-parameter and ablation scenarios, using RGB images. To be able to test and validate these approaches an In-car dataset was created, comprised by 1861 samples from 78 cars, and than splitted in 1303 train, 186 validation and 372 test RGB images. DeepLabV3+ showed promissing results, achieving an average accuracy for good, damage, stain and dirt of 77.17%, 58.60%, 65.81% and 68.82%, respectively.