Federated Learning BI across Multi-Cloud Data Silos
Main Article Content
Abstract
The enterprise data is sensitive, huge and spans across various cloud environments and quickly becoming a critical data source for Business Intelligence (BI) solutions. But, the need for a single data integration creates issues with data protection, data privacy, compliance, data ownership, latency and inter-cloud interoperability with traditional BI solutions. In the multi-cloud world with multiple and distributed data silos, a federated learning based BI framework is introduced to attain a collaborative analysis of these data silos without physically moving the raw data on the local data silos. The proposed system relies on a federation of local data repositories easily developed in the cloud (like Storage Service), a federation of models trained, a secure aggregation of the data involved and a layer of privacy-preserving computation, offering also a single BI visualization layer. This architecture may provide a way for models of analytics to be trained locally (using only data from each cloud) and for further shadowing (more specifically, transport) of the parameters of such models / information of analytics to the central co-ordination server but in an encrypted format. The global model is summarized and sent back to the Cloud nodes involved in the summarization, and can be utilized overtime for obtaining new BI insights without any data pollution. This framework enables customers' decision making at all clouds, customer analytics, financial forecasting and supply chain intelligence, healthcare reporting and enterprise risk monitoring. A variety of serious problems with centralized analytics, such as data duplication, compliance risks, bandwidth costs and vendor solutions can be overcome by combining federated learning and BI workflow. These include aspects of architecture, processes, security aspects, challenges and opportunities for the future intelligent multi-clouds BI ecosystem. The hinted solution path indicates how federated learning can transform the current landscape of BI – from producing and sending reports to a centralized data repository towards a distributed intelligence paradigm with privacy protection and scalability for today's information-driven enterprises.
Article Details
Section
How to Cite
References
[1] H. B. McMahan et al., “Communication-efficient learning of deep networks from decentralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
[2] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith,
“Federated optimization in heterogeneous networks,” Proceedings of Machine Learning and Systems (PMLR), vol. 2, pp. 429–450, 2020.
[3] S. P. Karimireddy et al., “Scaffold: Stochastic controlled averaging for on-device federated learning,” in International Conference on Machine Learning (ICML), 2020.
[4] S. U. Stich, “Local SGD converges fast and communicates little,” in International Conference on Learning Representations (ICLR), 2019.
[5] J. Konečný, H. B. McMahan, D. Ramage, and P. Richtárik, “Federated optimization: Distributed machine learning for on-device intelligence,” arXiv preprint arXiv:1610.02527, 2016.
[6] P. Kairouz et al., “Advances and open problems in federated learning,” arXiv preprint arXiv:1912.04977, 2019.
[7] C. He et al., “Group knowledge transfer: Federated learning of large CNNs at the edge,” in Advances in Neural Information Processing Systems (NeurIPS), 2020.
[8] Q. Li, Y. Diao, Q. Chen, and B. He, “Federated learning on non-IID data silos: An experimental study,” arXiv preprint arXiv:2102.02079, 2021.
[9] J. Wang and G. Joshi, “Cooperative SGD: A unified framework for the design and analysis of local-update SGD algorithms,” Journal of Machine Learning Research, vol. 22, no. 213, pp. 1–50, 2021.
[10] L. Liu, J. Zhang, S. H. Song, and K. B. Letaief, “Client-edge-cloud hierarchical federated learning,” in IEEE International Conference on Communications (ICC), 2020.
[11] Y. Wang et al., “Federated latent Dirichlet allocation: A local differential privacy based framework,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 6283–6290, 2020.
[12] Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent Systems and Technology, vol. 10, no. 2, pp. 1–19, 2019.