Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning
- View PDF HTML (experimental) Abstract:Group Relative Policy Optimization (GRPO) has become a promising approach for training large language model agents.
- However, its uniform assignment of trajectory-level advantages to all policy tokens fails to distinguish consequential decisions from less relevant ones, obscuring which intermediate decisions contributed to success.
- We introduce ProVer, a framework that targets potentially pivotal decisions for fine-grained credit assignment in agentic reinforcement learning.
Unverified
- View PDF HTML (experimental) Abstract:Group Relative Policy Optimization (GRPO) has become a promising approach for training large language model agents.
- However, its uniform assignment of trajectory-level advantages to all policy tokens fails to distinguish consequential decisions from less relevant ones, obscuring which intermediate decisions contributed to success.
- We introduce ProVer, a framework that targets potentially pivotal decisions for fine-grained credit assignment in agentic reinforcement learning.
Sources: Arxiv