Robust trajectory design and guidance for far-range rendezvous using reinforcement learning with safety and observability considerations
Minduli Wijayatunga, Roberto Armellin, Harry Holt · Aerospace Science and Technology · 2025
Observability, safety, and robustness are critical for successful rendezvous and proximity operation (RPO) missions. The use of angles-only navigation (AON) for these missions is often seen as limited due to its inability to determine range, though it remains appealing for its low cost. This work utilizes the proximal policy optimization algorithm in reinforcement learning for the guidance of the far-range phase of an RPO mission, ensuring observability, safety, and minimal fuel consumption under AON. Trajectory planning for the mission is done via a nonlinear optimizer, which also considers safety and observability. The constraint satisfaction challenges during trajectory planning and guidance are alleviated through the problem formulation, which incorporates Lambert's method to guarantee that the target state is always reached. During the training of the reinforcement learning controller, a predefined set of grid points in the initial state distribution is used to evaluate the policies and select the best policy fairly. The nominal and reinforcement learning-guided trajectories are validated for observability and safety, and the guidance controller's performance is tested through Monte Carlo simulations. Results show that for a 6 h mission previously presented in the literature, in the presence of errors, the reinforcement learning controller consumes 22.31% less Δ v compared to the next-best guidance strategy explored while fully adhering to safety and observability constraints. • Developed a far-range trajectory design and robust guidance method using nonlinear optimization and reinforcement learning. • Incorporated safety and observability considerations in trajectory design under angles-only navigation constraints. • Developed a reinforcement learning guidance strategy that handles thrust uncertainties and initial state variations. • Demonstrated a 22.31% reduction in Δ v compared to the next best benchmark while maintaining safety and observability. • Achieved 2.36 times lower average Δ v compared to previous solutions for a far-range test case.