Multi-backbone Ensembling for Performance Improvement in Federated Learning Setup

Multi-Backbone Ensembling (MBE) approach is proposed to enhance performance of deep neural networks (DNNs) in Federated Learning (FL) setup by leveraging parallelized lightweight architectures, using the LCNet architecture as a case study. The “width over depth” approach is investigated, in which multiple backbones are combined to increase model capacity. Experiments on CIFAR-100, conducted in an FL setting simulated by the Flower framework, evaluate the impact of key parameters, including batch size, local epochs, and the number of backbones. Key findings show that MBE architectures outperform single-backbone models in validation accuracy for higher values of the number of backbones (N_MB) with values in {1, 2, 4, 8}. Additionally, smaller batch sizes (N_b), evaluated in the range {32, 512, 8192}, lead to faster convergence and improved accuracy. Higher numbers of local epochs (N_le), tested in the range {1, 10, 100}, also contribute to accuracy gains. Overall, the results highlight MBE’s potential in resource-constrained Edge Intelligence (EI) environments, offering a scalable solution to balance accuracy and efficiency in FL deployments.