Data collection is a fundamental operation in energy harvesting industrial Internet of Things networks. To this end, we consider a hybrid access point (HAP) or controller that is responsible for charging and collecting L bits from sensor devices. The problem at hand is to optimize the transmit power allocation of the HAP over multiple time frames. The main challenge is that the HAP has causal channel state information to devices. In this article, we outline a novel two-step reinforcement learning with Gibbs sampling (TSRL-Gibbs) strategy, where the first step uses Q-learning and an action space comprising transmit power allocation sampled from a multidimensional simplex. The second step applies Gibbs sampling to further refine the action space. Our results show that TSRL-Gibbs requires up to 28.5% fewer frames than competing approaches.
In this article, we consider a network with a hybrid access point (HAP) and radio frequency (RF)-energy harvesting wireless devices. The HAP is responsible for charging these devices and receiving their data. Our problem is to select a set of devices to transmit in each time slot so as to maximize a given reward over a planning horizon. In contrast to prior works, we consider the challenging case whereby the HAP has imperfect channel state information (CSI) nor information about the battery state of devices. We also consider nonlinear RF-energy conversion rates and battery leakage. We propose a cross entropy approach to identify the best set of devices to select in each time slot over random channel gains. In addition, we propose a fast Gibbs sampling approach, called Gibbs+, that incorporates a novel step to evict noncompetitive devices. We compare our solutions against random pick, round robin, original Gibbs sampling, and perfect information selection (PIS). Our results show that CE