Federated Learning of Neural Network Models with Heterogeneous Structures

Kundjanasith Thonglek, Keichi Takahashi, Kohei Ichikawa, Hajimu Iida, Chawanat Nakasan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Citations (Scopus)

Abstract

Federated learning trains a model on a centralized server using datasets distributed over a large number of edge devices. Applying federated learning ensures data privacy because it does not transfer local data from edge devices to the server. Existing federated learning algorithms assume that all deployed models share the same structure. However, it is often infeasible to distribute the same model to every edge device because of hardware limitations such as computing performance and storage space. This paper proposes a novel federated learning algorithm to aggregate information from multiple heterogeneous models. The proposed method uses weighted average ensemble to combine the outputs from each model. The weight for the ensemble is optimized using black box optimization methods. We evaluated the proposed method using diverse models and datasets and found that it can achieve comparable performance to conventional training using centralized datasets. Furthermore, we compared six different optimization methods to tune the weights for the weighted average ensemble and found that tree parzen estimator achieves the highest accuracy among the alternatives.

Original languageEnglish
Title of host publicationProceedings - 19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020
EditorsM. Arif Wani, Feng Luo, Xiaolin Li, Dejing Dou, Francesco Bonchi
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages735-740
Number of pages6
ISBN (Electronic)9781728184708
DOIs
Publication statusPublished - 2020 Dec
Event19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020 - Virtual, Miami, United States
Duration: 2020 Dec 142020 Dec 17

Publication series

NameProceedings - 19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020

Conference

Conference19th IEEE International Conference on Machine Learning and Applications, ICMLA 2020
Country/TerritoryUnited States
CityVirtual, Miami
Period20/12/1420/12/17

Keywords

  • Decentralized dataset
  • Deep Neural Network
  • Ensemble learning
  • Federated learning

Fingerprint

Dive into the research topics of 'Federated Learning of Neural Network Models with Heterogeneous Structures'. Together they form a unique fingerprint.

Cite this