Curriculum Learning: You Can’t Skip the Basics

Full-game training from day one

Random deck construction from the full Silent card pool — roughly 80 cards. Random enemy selection across the full Act 1 roster defined in stages.py — 21 normal encounters (single and multi) plus 6 elites, so around 27 distinct fights. Full mechanical complexity: powers, debuffs, upgraded cards, relic interactions. Everything on, all the time.

The theory was that if the agent experiences the full distribution, it will learn a policy that generalizes. Curriculum learning, in that frame, is training wheels — it narrows the distribution, which has to limit final performance.

That was wrong. The result was that the agent learned nothing useful at all.

The agent learned nothing useful after thousands of episodes.

The state space was too large for the signal to reach. Four specific failure modes, which I will list in the order I noticed them:

1. Rare card exposure. Eighty cards in the pool, ten cards in a deck. The agent saw most cards a handful of times per hundred episodes. “Play Strike, enemy HP drops, good” is a simple lesson, but you need to see Strike hundreds of times to learn it reliably. At a handful of exposures per hundred episodes, the signal could not stabilize.
2. Enemy diversity without foundation. Fossil Stalker, Punch Construct, Sewer Clam, Vine Shambler, the Byrdonis and Phrog Parasite elites — each has distinct attack patterns, intents, and optimal responses. With random matchups, the agent couldn’t specialize enough to learn any single enemy well. It certainly couldn’t learn when to switch strategies.
3. Win rate plateaued at ~15%. Better than random, but only because the agent had picked up the most shallow lesson available: play attacks when enemies have low HP, play defense on high-damage turns. That is essentially the policy a five-minute-old human gets to. It was not going to get deeper without more signal.
4. Training signal was too noisy. Every episode was a completely different combat. The variance in reward per episode was huge, so gradient updates pointed in wildly different directions. PPO’s trust region — the per-update cap on how far the policy can shift — kept the policy changes small, but not small enough to prevent drift around a noisy mean.

After about two weeks of this with no measurable progress, I pivoted.

I broke training into 13 stages, gradually introducing complexity.

Stage 0 is almost trivial — 4 unique cards, one enemy type, single-digit HP difference per turn. The agent could figure out the basics fast: 80%+ win rate in maybe 500 episodes.

By Stage 5, the agent had a solid foundation of card-game fundamentals: block when the enemy is about to hit hard, prioritize killing weak enemies first, upgrade cards for efficiency. Each subsequent stage added a few new concepts, not an entirely new game.

Automatic stage advancement

Early stages blow through quickly — maybe an hour of training each. Hard stages get as much time as they need. And I don’t have to manually decide when to advance, which is the productivity piece that actually matters. Manual advancement steals momentum every time you pause the run; automated advancement means I can leave training overnight and come back to a model that’s several stages further along.

Stage config lives in stage.json, written by StageConfig.to_dict in stages.py. Six keys: stage, deck_size_min, deck_size_max, upgrade_chance, card_pool, enemy_pool. Deck construction is weighted random draws from the stage’s card_pool into a deck sized within [deck_size_min, deck_size_max], with each card upgraded independently at upgrade_chance. Per _STAGE_DEFS that chance jumps from 0% (stage 0) to 10% (stages 1–8) to 30% (stages 9–12) — stepped, not a smooth ramp. Rollback — falling back to a prior stage if the agent regresses — I considered and never implemented. It hasn’t been needed.