19 February 2020
13:30 Doctoral defense Auditorium 1
Theme
Pattern Recognition in Facial Expressions: Algorithms and Applications Candidate
Student
Jadisha Yarif Ramírez Cornejo
Advisor / Teacher
Helio Pedrini
Brief summary
The recognition of emotions has become a relevant topic of research by the scientific community, since it plays an essential role in the continuous improvement of human-computer interaction systems. In addition, it can be applied in several areas, such as medicine, entertainment, surveillance, biometrics, education, social networks and affective computing. Emotions can be expressed through one or more stimuli. The most common forms are facial expressions and prosody of speech. However, for the development of emotional systems based on facial expressions, there are some open challenges, such as the inclusion of data that reflect more spontaneous emotions and real scenarios, for example, facial occlusions and head positions. In the specific case of static facial expressions, they do not have dynamic time information. There is also research on multimodal alternatives more similar to the way human-human interactions work. In this doctoral thesis, we propose different approaches for the development of emotion recognition systems based on facial expressions, as well as their applicability in solving other similar problems. In Chapter 1, we propose an emotion recognition methodology for occluded facial expressions, based on the Transform Histogram Census (CENTRIST). Occluded facial expressions are reconstructed using Robust Principal Component Analysis. The extraction of characteristics from facial expressions is performed by CENTRIST, as well as by Local Binary Standards (LBP), by Local Gradient Coding (LGC) and by an extension of LGC. The space of characteristics generated is reduced by applying Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). The nearest K-neighbor (K-NN) and Support Vector Machine (SVM) algorithms are used for classification. The method achieved competitive hit rates for occluded and non-occluded facial expressions. In Chapter 2, we introduced dynamic recognition of facial expressions based on Visual Rhythms (VR) and Images of the History of Movement (MHI), so that a fusion of both descriptors encodes the appearance, shape and movement information of the videos. For extracting the characteristics, the Weber Local Descriptor (WLD), CENTRIST, Oriented Gradient Histogram (HOG) and the Gray Level Co-occurrence Matrix (GLCM) are used. The approach presents a new proposal for the dynamic recognition of facial expressions and an analysis of the relevance of facial parts. In Chapter 3, an effective method is presented for the recognition of audiovisual emotions based on speech and facial expressions. The methodology involves a hybrid neural network to extract visual and audio characteristics from the videos. For audio extraction, a Convolutional Neural Network (CNN) based on Mel's log-spectrogram is used, while a CNN built on the Census Transform is used for the extraction of visual characteristics. The audiovisual attributes are reduced by PCA and LDA, then classified by K-NN, SVM, Logistic Regression (LR) and Gaussian Na \ "ive Bayes (GNB). The approach achieved competitive recognition rates, especially in spontaneous data. In Chapter 4, the problem of detecting Down's syndrome from photographs is investigated. An extended geometric descriptor is proposed to extract facial features. Experiments carried out on a public database show the effectiveness of the developed methodology. In Chapter 5, a methodology for the recognition of genetic syndromes in photographs is presented. The method aims to extract facial attributes using characteristics of a deep neural network and anthropometric measurements.
Examination Board
Headlines:
| Hélio Pedrini | IC / UNICAMP |
| Sarajane Marques Peres | EACH / USP |
| Edimilson Batista dos Santos | DCC / UFSJ |
| Fabricio Aparecido Breve | DEMAC / UNESP |
| Marco Antonio Garcia de Carvalho | FT / UNICAMP |
Substitutes:
| André Santanchè | IC / UNICAMP |
| Alexandre Mello Ferreira | IC / UNICAMP |
| Moacir Antonelli Ponti | ICMC / USP |