What skills will be developed in the course?

The Extension Course in Natural Language Processing aims to train professionals for the current job market, exploring from classical techniques to attentional models and Transformers, exploring several practical applications.

INF-1000

Extension course

Email: nlp@ic.unicamp.br

Phone: (19) 3521-5883 / 3521-5861

Presented by:

Overview

Format

The Extension Course in Natural Language Processing consists of 4 modules that cover the main concepts of the area, with a total workload of 40 hours, 32 hours of classes and 8 hours of supervised activities.

Digital Certificate

Successful students will be entitled to the certificate of completion of the Extension Course in Natural Language Processing, issued by the Unicamp Extension School.

Faculty

The faculty of the Extension Course in Natural Language Processing is composed of professors and researchers from Unicamp, with great experience in the area, all with a PhD.



  • Modules

    • INTRODUCTION TO NATURAL LANGUAGE PROCESSING
    Fundamentals: brief history, definitions, types of approaches, textual representations, types of textual pattern analysis. Text pre-processing: standardization, tokenization, normalization, filtering, word relevance, morphological tagging. Text mapping: feature extraction, word bag, vectorization, term frequency, document inverse frequency. Language models: probabilistic, Markov, unigrams, bigrams, n-grams, evaluation.

    ATTENTIONAL MODELS
    Introduction to attention and attentional mechanisms: hard, soft and self-attention. Classic attention-based deep learning architectures: CNNs, recurring and generative. End-to-end attentional models: NT, GAT. Applications and benefits of attention-based models.

    TRANSFORMERS
    Motivation and architectural overview. Encoder (positional encoders, multi-head attention, feed-forward layer). Decoder (masked multi-head attention, linear layer and softmax). Transformer variations (BERT, T5, GPT3, Efficient Transformer, etc.).

    APPLICATIONS
    Mono/Multilingual Transformers for PLN (Text Generation, Translation). Transformers in Images (image classification). Transformers in other media (audio, sensors, etc.). Challenges and opportunities (explicability, size of models, train vs. refine).



  • Details

    • Prerequisite: Basic knowledge of programming.
    Target Audience: Computer professionals, trained in Computing or related areas (Engineering or Exact).
    Course type: Extension course.
    Required Material: As it is a practical course, all students must use their notebooks in class.
    Course coordinator: Prof. doctor Zanoni Dias.
    Offering: Exclusively in the "in company" model (closed to companies). Request a quote by Email.