<ul data-eligibleForWebStory="true">Anatomical changes during head-and-neck cancer proton therapy can necessitate treatment replanning, but current methods are labor-intensive.A patient-specific deep reinforcement learning framework for automated replanning using a reward-shaping mechanism was proposed.The framework trained personalized agents for each patient to adjust optimization priorities for maximizing plan quality.The approach leverages anatomical similarities to adapt plans effectively throughout treatment.Two deep reinforcement learning algorithms, Deep Q-Network and Proximal Policy Optimization, were implemented.Both algorithms improved initial plan scores significantly, surpassing manual replans from a human planner.Evaluation on five head-and-neck cancer patients showed improved tumor coverage and organ-at-risk sparing.The work highlights the potential of deep reinforcement learning in adaptive proton therapy for addressing dosimetric complexities.