This letter considers optimizing information freshness in a network with Radio Frequency (RF)-energy harvesting wireless devices. A Hybrid Access Point (HAP) charges these devices and instructs a subset of devices to carry out sampling and transmit their sample. We outline a Distributed Q-Learning (DQL) algorithm that allows the HAP to select devices without knowing their uplink channel state and battery state. Our results show that DQL achieves at most 48%, 57%, and 61% lower average AoI than Round Robin (RR), Random Pick (RP), and AoI-Greedy (AG), respectively. The average AoI of DQL is only around 7% higher than the optimal selection strategy.