ML-based edge assessment and pricing mechanisms for edge-cloud systems

Abstract

The rapid development of the Internet of Things (IoT) has transformed various aspects of modern society, impacting healthcare, transportation, industrial automation, retail and supply chain, and more. The proliferation of mobile internet and diverse applications has influenced individuals’ lifestyles. While cloud computing offers computation and storage solutions, it has limitations such as latency, energy consumption, privacy concerns, and bandwidth issues. Edge-cloud systems encompass four typical applications: client selection for interconnection, client data value assessment for resource conservation, and network data marketplace incentive mechanism for open sourcing. The efficient design of incentive mechanisms aims to achieve collaborative cooperation and mutual benefits. Large-scale simulation experiments are conducted to validate the performance of the proposed mechanisms. The main research content and innovative achievements of this thesis are as follows: The research begins by addressing the critical task of client selection in federated learning, which involves achieving a balance between model accuracy and communication efficiency. Existing methods face challenges in handling data heterogeneity, computational burdens, and treating clients as independent entities. To overcome these limitations, a novel approach called GPFL (Gradient Projection-Based Federated Learning) is proposed. GPFL is a pre-selection method that evaluates clients based on their historical contributions and incorporates UCB (Upper Confidence Bound) mechanisms to balance exploration and exploitation. This approach aims to discover optimal client combinations, enhance performance, and ensure effective client selection in federated learning scenarios. Additionally, the research explores data pricing mechanisms in network data trading markets, aiming to incentivize data owners to share their data, break down data barriers, promote data sharing and circulation, and unlock the value of data. While the Shapley value is widely used to assess individual contributions in this context, it suffers from computational complexity. Calculating the Shapley value requires evaluating the marginal contribution of each player in every possible coalition, leading to an exponential number of calculations. This becomes impractical as the number of participants increases, limiting the scalability of the Shapley value, especially in large-scale scenarios. To overcome this limitation, the research investigates an improved version of the Shapley value-based pricing method introduced in NmFLI. This holistic approach effectively balances accuracy and computational complexity, addressing the challenges associated with incentivizing federated learning. In GPFL, we propose an innovative approach to measure the data value of clients. This is achieved by comparing the gradient projection of clients’ local gradients on the global gradient. This approach assesses the clients’ ability in loss reduction and takes into account their historical contributions and potential value. Furthermore, the research investigates data incentive mechanisms in network data trading markets, particularly focusing on incentivizing federated learning within a non-monopoly market. This mechanism aims to motivate data owners to share their data, overcome data barriers, promote data sharing and circulation, and unlock the value of data. To address this challenge, a pioneering mechanism called NmFLI (Non-monopoly Federated Learning Incentive) is introduced. NmFLI utilizes a double-auction mechanism to implement incentives for federated learning, ensuring fair participation among users. The trustworthiness of clients is maintained through the utilization of the Vickery-Clarke-Groves (VCG) mechanism, establishing a reliable environment. This holistic approach effectively balances accuracy and computational complexity, addressing the challenges of incentivizing federated learning.