How it works 

In traditional reinforcement learning, agents must interact with the environment to collect training data. However, this interaction can be costly, risky, or impractical for real-world applications like autonomous driving. Offline reinforcement learning circumvents this by training agents using pre-existing datasets. In the F1tenth racing context, a controller gathers race data, such as motor thrust and steering rates. This data is then used to train offline RL models utilizing RCDTP and RWDTP frameworks. These policies utilize decision trees, providing fast, explainable results without interacting with the physical environment, thus mitigating real-time failure risks. 

The RCDTP and RWDTP frameworks improve upon many existing frameworks, simplifying the approach to achieve greater agent trading and inference speeds while demonstrating performance at least on par with the established methods. RCDTP models the agent’s actions based on the state, return-to-go (the future reward sum), and timestep, conditioning each action on the trajectory’s goal. This aids in maintaining consistent long-term objectives. In contrast, RWDTP utilizes only the current state but applies a weighted return, where a weighted cumulative reward influences actions. This simplifies decision-making, especially under constrained data, making it less computationally demanding. RCDTP’s return-to-go approach makes it more suited for tasks requiring precise control over longer sequences, whereas RWDTP’s return-weighting enhances efficiency for scenarios where quick, interpretable responses are prioritized.