ImplementCore implementation

PPO: clipped surrogate objective

Implement `run(old_prob, new_prob, advantage, epsilon)` returning the PPO clipped surrogate value L^CLIP for a single (state, action): r = new_prob / old_prob, surrogate = r * A, clipped = clip(r, 1-eps, 1+eps) * A, and L = min(surrogate, clipped).

python10 mindifficulty 2/4

From Proximal Policy Optimization Algorithms

Sample tests

run([0.5,0.5,1,0.2])→1
run([0.5,0.7,1,0.2])→1.2
run([0.5,0.3,1,0.2])→0.6

Hints

The probability ratio r(θ) compares each action's odds under the new policy against the old one.
The clip keeps the objective flat when r leaves [1-epsilon, 1+epsilon].
L = min(r*A, clip(r, 1-eps, 1+eps)*A) — the min means a too-large positive ratio gets capped, a too-small negative advantage is not exploited.
With epsilon=0.2 (paper default), the trust region is r in [0.8, 1.2].

Hints

Code runs in an isolated sandbox on the OpenSandbox judge. No model key required.