Spatio-Temporal Noise Filtering using Convolutional Neural Networks with a Realistic Noise Model under Low-Light Conditions
Tim Borglund, Tor Nilsson · Lund University Publications Student Papers (Lund University) · 2019
Convolutional neural networks have in recent years been successfully employed for various image processing tasks, such as filtering noise. There are however relatively few published attempts for processing video in this way. Image processing methods on single images can be applied frame by frame, but often fail to consider continuity and flow between frames. In this master's thesis we constructed several fully convolutional neural network models, trained to filter noise spatially as well as temporally. We present the differences between these models and compare the performance of each of them with a noise filter from a state-of-the-art camera, as well as with a solely spatial filter. Our data was created by adding noise to clean videos according to a noise model which realistically simulates noise from camera sensors under low-light conditions. On a frame by frame basis, our best model outperforms the state-of-the-art camera in most situations. Despite still having minor struggles with continuity in video, clear improvement can also be seen in comparison with only spatial noise filtering.