Learning to control
things that move.
I'm working toward robot learning — reinforcement and imitation learning for manipulation, the dynamics underneath them, and the gap between simulation and the real thing.
Computational physics gave me numerical intuition, linear algebra, and simulation thinking. Those are the right three things to bring. What's missing is dynamics, optimization, and enough control theory to know why a policy behaves the way it does — so that's the order I'm learning it in.
Where I am now
Stage 1 of 5 · State space
The sequence
Five stages, in order, one at a time. RL comes last on purpose — after the dynamics, not before.
- State spaceIn progressNise, ch. 3 and 12
- Rigid-body dynamicsLynch & Park ch. 8 · Craig ch. 6–7
- Optimization and optimal controlBoyd (selected) · Tedrake on LQR and MPC
- Underactuated roboticsTedrake, MIT 6.832
- Reinforcement and imitation learningSutton & Barto (selected) · Levine, CS285
What I'll build
These start at stage three. Each one ships with a write-up of what failed and why.
Swing-up
Cart-pole and acrobot, LQR and then MPC. Small and classic, and it proves the control foundation is real.
A learned policy
A manipulation task in simulation, trained end to end.
Sim-to-real on cheap hardware
A low-cost open-source arm, teleoperated demonstrations, and imitation learning.