29 August 2024
16:00 Master's Defense IC3 Auditorium
Theme
Evaluation of Ethics Tools in Language Models Developed in Portuguese
Student
Jhéssica Victória Santos da Silva
Advisor / Teacher
Hélio Pedrini - Co-supervisor: Sandra Eliza Fontes de Avila
Brief summary
In the area of Artificial Intelligence (AI), language models have gained important space due to the great advances made in various fields of knowledge and the recent popularization of systems capable of simulating realistic conversations with human beings through the generation of texts. Due to their impacts on society, it is essential that these models are developed and implemented responsibly, with attention to their negative impacts and possible damages. In recent years, there has been an increase in the publication of AI Ethics Tools (AIETs). These AIETs aim to help developers, companies, governments and other stakeholders establish trust, transparency and accountability with their technologies. AIETs aim to bring accepted values to guide the design, development and use stages of AI. However, many AIETs lack good documentation, examples of use and evidence of their effectiveness. In this work, we present a comprehensive methodology for selecting and evaluating AIETs in language models. Our approach involved carrying out an extensive literature search on AIETs. We applied inclusion and exclusion criteria to filter all AIETs found, resulting in eight AIETs. To test our method, we interviewed developers of language models developed for the Portuguese language, where we applied four different AIETs – Model Cards, ALTAI, FactSheets and Harms Modeling – to the interviewed language models. The assessment considered developers' perceptions of the use and quality of AIETs in helping to identify ethical considerations about language models, and whether the considerations raised correspond to the literature on the ethical impacts of language models. The results show developers' preferences for each AIET used and suggest developers' approval of the use of AIETs. However, we note that the ease of use of AIETs is an important factor that influences developers' preferences.
Examination Board
Headlines:
| Hélio Pedrini | IC / UNICAMP |
| Mariza Ferro | IC / UFF |
| Marisol Marini | FFLCH/USP |
Substitutes:
| Tatiane Nogueira Rios | IC/UFBA |
| Leonardo Tomazeli Duarte | FCA/UNICAMP |