Proof of Concept / Reinforcement Learning
Q-Learning Agent for Specialty Oncology Rebate Optimization
A tabular Q-learning reinforcement learning agent that discovers the revenue-maximizing rebate strategy for a specialty oncology drug across four payer archetypes. The agent learns to balance formulary access against net price by exploring a 144-state Markov decision process — converging on differentiated contracting strategies per payer segment after 500 training episodes. Built entirely in-browser with no backend dependencies.
Disclaimer: This is a proof-of-concept demonstration using synthetic market parameters and simplified contracting dynamics. All payer characteristics, rebate thresholds, and revenue figures are illustrative. This tool does NOT represent actual contracting strategies, market access data, or pricing policies of any pharmaceutical company. Not intended for commercial decision-making.
Training RL agent — 500 episodes × 12 steps...
Exploring 144 states × 6 rebate actions with ε-greedy Q-learning
A 144 × 6 Q-table (states × actions) stores learned value estimates for every (payer, competitive tier, formulary position, contract timing) × rebate action combination. The Bellman update Q(s,a) ← Q(s,a) + α(r + γ·max Q(s',a') − Q(s,a)) iterates toward the optimal policy using learning rate α=0.15 and discount γ=0.9.
Early training uses high exploration (ε=0.5) to survey all 864 state-action pairs. Epsilon decays exponentially toward 0.05 over 500 episodes, ensuring thorough initial exploration before committing to learned optima. The decay schedule prevents premature convergence on locally optimal but globally suboptimal rebate strategies.
Formulary tier transitions are probabilistic: meeting the preferred threshold yields preferred status with 85% probability (reflecting P&T committee variability). Competitor rebate evolves as a random walk each step. Contract renewal cycles reset every 4 steps. This stochasticity forces the agent to learn robust policies rather than deterministic one-time strategies.