Optimizing Federated Learning Performance in Heterogeneous Distributed Systems from Computational Resource and Data Distribution Perspectives

Mohammad Hossein Shafiabadi; Rasool  Daryaei Aghkand

Authors

Mohammad Hossein Shafiabadi Department of Computer engineering, Isl.C., Islamic Azad University, Islamshahr, Iran https://orcid.org/0000-0002-6218-7996
Rasool Daryaei Aghkand Department of Computer engineering, Isl.C., Islamic Azad University, Islamshahr, Iran Author https://orcid.org/0009-0008-5965-577X

Keywords:

Federated learning, Computational heterogeneity, Non-IID data distribution, Adaptive resource allocation, Energy-efficient machine learning

Abstract

This study aims to develop an adaptive resource allocation framework that simultaneously optimizes computational efficiency and data distribution heterogeneity in federated learning (FL) systems. A simulation-based experimental design evaluated FL performance across 50 heterogeneous client nodes (2-core to 32-core CPUs) under three resource allocation strategies (homogeneous, heterogeneous, adaptive) and three data skew levels (IID, moderate non-IID, severe non-IID). The framework utilized TensorFlow Federated for FL orchestration, Prometheus/Grafana for real-time resource monitoring, and a custom Python module for data skew quantification. Metrics included global accuracy, communication rounds, energy consumption, and convergence dynamics across CIFAR-10, FEMNIST, and medical imaging datasets. Adaptive allocation reduced communication rounds by 37.2% (vs. homogeneous) while maintaining 94.3% target accuracy under severe non-IID conditions (Gini >0.7), outperforming homogeneous (78.2% accuracy) and heterogeneous (85.1% accuracy) strategies. It achieved a 28.7% system-wide energy reduction by capping edge-node compute at 55% utilization (preventing underutilization) and prioritizing cloud-node bandwidth (avoiding overutilization). The framework eliminated straggler effects (3.2% vs. 18.3% in homogeneous allocation) and demonstrated a 12.7% accuracy advantage over heterogeneous allocation when data skew exceeded Gini coefficient 0.6. Statistical validation confirmed significant interaction effects between resource allocation and data skew (F = 142.7, p < 0.001). This work establishes that intelligent resource orchestration—dynamically adjusting compute quotas based on real-time node performance and data skew—resolves the interdependent challenges of computational heterogeneity and non-IID data in FL. The adaptive framework delivers multiplicative gains in accuracy (up to 13.1%), communication efficiency (37.2% fewer rounds), and energy conservation (28.7% less consumption), providing a scalable blueprint for real-world FL deployment in healthcare, finance, and IoT.

References

Abedian, M., Amindoust, A., Maddahi, R., & Jouzdani, J. (2022). A Nash equilibrium based decision-making method for performance evaluation: a case study. Journal of Ambient Intelligence and Humanized Computing, 13(12), 5563-5579. https://doi.org/10.1007/s12652-021-03188-8

Alsuwailem, A. A. S., Salem, E., & Saudagar, A. K. J. (2023). Performance of different machine learning algorithms in detecting financial fraud. Computational Economics, 62(4), 1631-1667. https://doi.org/10.1007/s10614-022-10314-x

Arabnejad, H., & Barbosa, J. G. (2014). List scheduling algorithm for heterogeneous systems by an optimistic cost table. IEEE Transactions on Parallel and Distributed Systems, 25(3), 682-694. https://doi.org/10.1109/TPDS.2013.57

Avetisyan, H., & Broneske, D. (2023). Large language models and low resource languages: An examination of Armenian NLP. Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings),

Badal, V. D., Depp, C. A., Hitchcock, P. F., Penn, D. L., Harvey, P. D., & Pinkham, A. E. (2021). Computational methods for integrative evaluation of confidence, accuracy, and reaction time in facial affect recognition in schizophrenia. Schizophrenia Research: Cognition, 25, 100196. https://doi.org/10.1016/j.scog.2021.100196

Fard, H. M., Prodan, R., & Fahringer, T. (2012). Multi-objective list scheduling of workflow applications in heterogeneous systems 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing,

Hu, D., Hu, Y., Hu, R., Tan, Z., Ni, P., Chen, Y., & Liu, J. (2025). Machine Learning-Finite Element Mesh Optimization-Based Modeling and Prediction of Excavation-Induced Shield Tunnel Ground Settlement. International Journal of Computational Methods. https://doi.org/10.1142/S021987622450066X

Lü, M. (2024). Application of Machine Learning Algorithms in Resource Allocation for Wireless Communications. Applied and Computational Engineering, 102(1), 37-42. https://doi.org/10.54254/2755-2721/102/20240966

Nosrati, A. (2024). Multi-objective Optimization of Machine Learning Algorithms for Reducing Computational Costs and Improving Performance in Adaptive Artificial Intelligence. Nineteenth International Conference on Innovation and Research in Engineering Sciences,

Paytakhti Askouei, S. A., Negahbani, A., Mehrgan, N., & Nahidi Amirkhiz, M. R. (2022). Islamic financing and mobilization of banking resources in Iran. Quarterly Journal of Computational Economics, 1(4).

Wang, J., Zhai, X., Wu, X., Shi, J., & Zeng, G. (2024). Applications of Deep Learning in Computational Imaging With Structured Illumination. 50. https://doi.org/10.1117/12.3017729

Xie, C., Hou, C., Sun, Y., & Sugumaran, V. (2025). Exploring Risks and Challenges in Crowdfunding Performance Using Text Analytics and Deep Learning. International Journal of Software Science and Computational Intelligence, 17(1), 1-31. https://doi.org/10.4018/ijssci.370002

Yuan, H., & Zhou, M. (2020). Profit-maximized collaborative computation offloading and resource allocation in distributed cloud and edge computing systems. IEEE Transactions on Automation Science and Engineering, 18(3), 1277-1287. https://doi.org/10.1109/TASE.2020.3000946

Zhang, M., & Pang, H. (2020). Analysis of Damping Performance of Frame Structure With Viscoelastic Dampers. Engineering Computations, 38(2), 913-928. https://doi.org/10.1108/ec-02-2020-0116