Blockchain Sharding Over Wireless Channels: Dynamic Resource Allocation With Sparse Reward
Zhangjun Ren, Long Xing Shi, Zhe Wang, Jun Li, Zehui Xiong · IEEE Wireless Communications Letters · 2024
Blockchain sharding over wireless channels (BSoW) is a promising technology to enhance the throughput of blockchain systems by parallelly processing the transactions over multiple shards. Nonetheless, the time-varying characteristic of wireless fading channels may result in unequal transaction processing rates across different shards, which limits the overall system delay. In this letter, we formulate the dynamic resource allocation as a Markov decision process (MDP) that jointly optimizes the bandwidth allocation and block sizes, aiming to minimize the cumulative transaction completion latency for the BSoW network. Each transaction undergoes a two-stage consensus, where it first waits in the transaction queue for member-PBFT consensus, and then waits in the block queue for final-PBFT consensus before being appended to the main chain. Therefore, the reward feedback from the environment is delayed and sparse. Due to the lack of intermediate feedback, the conventional temporal-difference based reinforcement learning algorithms such as Proximal Policy Optimization (PPO) may not be able to identify the impact of each action on the final reward, which suffers from low learning efficiency. To address this issue, we propose an Hindsight Distribution Correction Estimation (HDICE)-PPO algorithm to effectively learn the credit assignment of sparse reward over intermediate actions for the policy optimization. The simulation results show that the proposed algorithm reduces the average transaction latency by 46% compared with Hindsight Credit Assignment (HCA)-PPO algorithm, and by 72.4% compared with PPO algorithm.