MATH 170 · Derivatives, descent and learning · Reinforcement learning (the Roomba from the first activity, for real)
Train a tiny agent Q-learning
The Roomba from the very first worksheet: bumps cost, finding the dock pays. It has
no map — only rewards. Train it and watch the map grow backwards from the goal as
numbers in the squares.
episodes: 0
Untrained — every square is worth nothing yet.
the update, after each move:
new value = old + ½ · ( reward + 0.9 · best-next − old )
high early = explores; low late = exploits what it knows
Rewards: each move −0.1 (battery) · bump −5 · dock +10.
Editing the room resets learning — "the room changed; the old map is stale."
Try: Run 25, watch the dock's neighbours darken first — then Run again and "let it drive."