06 August 2024
14:00 Master's Defense Room 85 of IC2
Theme
Strategies for Reducing Communication Cost in Federated Learning: An Information Theory Perspective and Model Compression
Student
Rómulo Walter Condori Bustincio
Advisor / Teacher
Luiz Fernando Bittencourt - Co-supervisor: Allan Mariano de Souza
Brief summary
In the rapid advancement of cloud computing, numerous challenges have emerged, including communication costs and the overhead of transferring information from one host to another, known as communication overhead. To mitigate these problems and reduce communication costs, edge computing was developed, distributing processing to end or edge devices. With the advancement of hardware, the development of large-scale machine learning began, combining the power of computational resources on terminal devices with the need to train machine learning models using sensitive or private data. This convergence has led to the development of federated learning, a robust strategy for training machine learning models in a distributed manner, focusing on data protection. In this context, the problem of communication overhead emerges as a critical problem that needs to be addressed to improve and sustain efficient models and infrastructures. This master's thesis focuses on addressing the communication overhead in federated learning, developing a taxonomy to identify the approaches currently used in the literature to solve this problem. In the second part, it proposes a client selection algorithm aiming to reduce communication costs based on suitability metrics to choose devices or clients for training. Lastly, it introduces a sparsity pruning algorithm to reduce the number of parameters or weights in a neural network to be shared with the server, focusing on eliminating weights that may be considered irrelevant. All these contributions aim to systematically address the problem described.
Examination Board
Headlines:
| Luiz Fernando Bittencourt | IC / UNICAMP |
| Júlio Cezar Estrella | ICMC / USP |
| Edson Borin | IC / UNICAMP |
Substitutes:
| Edmundo Roberto Mauro Madeira | IC / UNICAMP |
| Rodolfo da Silva Villaca | CT/UFES |
| Vinícius Fernandes Soares Mota | CT/UFES |