07August2025
09:00 Doctoral defense room 85 of IC2
Topic on
Self-Supervised Learning Applied to Spatio-Temporal Data
Student
Leandro Stival
Advisor / Teacher
Helio Pedrini - Co-advisor: Ricardo da Silva Torres
Brief summary
Spatiotemporal data (STD) are present in many areas, such as neuroscience, social sciences, criminology, earth sciences and video processing. This diversity of areas is reflected in the large amount of data available, which requires appropriate tools for analysis. The development and use of machine learning methods have gained attention as a promising direction to support data analysis. However, in most cases, these data do not have labels to support the development of supervised techniques, which poses a challenge for using the available data in practical applications involving prediction tasks. This thesis explores methodologies that use unlabeled spatiotemporal data to train robust machine learning models, improving the propagation of information over time and building meaningful feature representations. In this sense, two distinct application domains are explored: deep learning for video colorization (DLVC) and tasks involving multispectral remote sensing images (MSRSI). In the context of DLVC, the central challenge is to achieve faithful color reconstruction while ensuring temporal consistency between video frames. Thus, we propose methodologies to produce feature representations that more effectively encode spatial and temporal dependencies. To this end, contributions in this area include: a comprehensive survey of open problems and trends in DLVC, the production and evaluation of existing colorization and propagation strategies, the development of new network architectures focused on feature fusion, and the introduction of training protocols and computer vision architectures. The proposed methods demonstrate colorization performance that exceeds existing benchmarks. In the context of remote sensing, the central challenge is to extract effective representations of MSRSI, which integrate semantic, spatial and temporal information in a single space. This thesis addresses how the semantic richness of multispectral images can enhance the feature space through Self-Supervised Learning (SSL), analyzing semantic and textural patterns integrated into the training of deep models. These patterns have been validated in remote sensing tasks such as land cover classification, semantic segmentation and change detection. Considering temporal information, such aspects were extensively investigated in MSRSI through time series of vegetation indices at pixel level. Validation of the multimodality of the trained models was performed through a series of tasks, including classification and prediction of time series of per-pixel vegetation indices. In summary, the contributions of this thesis deepen the knowledge of SSL for STD, DLVC, and MSRSI.
Examination Board
Headlines:
| Hélio Pedrini | IC / UNICAMP |
| David Menotti Gomes | DInf / UFPR |
| Edimilson Batista dos Santos | DCOMP / UFSJ |
| Marcelo da Silva Reis | IC / UNICAMP |
| Marco Antonio Garcia de Carvalho | FT / UNICAMP |
Substitutes:
| André Santanchè | IC / UNICAMP |
| Moacir Antonelli Ponti | ICMC / USP |
| Ronaldo Cristiano Prati | CMCC / UFABC |