05 dez 2023
14:00 Doctoral defense Room 53 of IC2
Digital Video Stabilization: Methods, Datasets and Evaluation
Marcos Roberto e Souza
Advisor / Teacher
Helio Pedrini
Brief summary
Digital video stabilization aims to improve the visual quality of videos with flickering, smoothing the camera trajectory. Although numerous research papers have addressed this challenge, there is a lack of organization and analysis in the literature. In this work, we present an extensive literature review, including methods and evaluations, categorized according to proposed taxonomies. We provide a formal problem definition, identify key challenges, and discuss future directions. Furthermore, we performed a meta-analysis of the results reported in several studies, from which we realized, for example, that recent approaches such as DWS achieve lower quality compared to methods from the classical approach. This organizational effort resulted in two surveys. We also cover the evaluation of video stabilization, emphasizing its importance in understanding stability from the perspective of human perception and the need for appropriate datasets. First, we conduct a brief study on the correlation of current video stability measures, introducing new kinematics-based alternatives. We achieve up to 78% correlation with human perception, compared to around 65% for the best metric in the literature. We also highlight the importance of evaluating each stage of stabilization. For the two-dimensional motion estimation step, we proposed an evaluation approach based on camera motion fields. Our experimental results demonstrate the reliability of our metrics and their potential for evaluating different scenarios. Finally, we address the challenges of DWS methods, which use deep learning to predict stabilized frames without explicitly calculating unsteady camera motion. In this sense, we proposed NAFT, a semi-online DWS method that integrates a neighbor-aware response update mechanism into a RAFT-based model. Our approach learns how to stabilize from data without explicitly including the definition of stability in the training loss functions. Additionally, we introduce SynthStab, a synthetic dataset that facilitates surveillance through camera motion and is used to train NAFT. Our experimental results show that NAFT closes the performance gap of DWS methods relative to state-of-the-art methods, while also reducing the number of parameters and model size to just 7% of its competitors.
Examination Board
Hélio Pedrini IC / UNICAMP
Luiz Maurílio da Silva Maciel ICE/UFJF
André Eugênio Lazzaretti DAELN/UTFPR
Alexandre Mello Ferreira EEP
Esther Luna Colombini IC / UNICAMP
Andre Santanche IC / UNICAMP
Erickson Rangel do Nascimento ICEx/UFMG
Ronaldo Cristiano Prati CMCC / UFABC