An opponent that plays Rome
rather than cheating at it.
A fogged, no-cheat AI faction for Total War: Rome Remastered, reasoning through Agentic Orchestration and acting through the same console and input surface a person uses.
Every Total War player knows the AI cheats, and knowing it takes something out of the win.
You bait it into a river crossing and it walks in anyway. You leave a city undefended for fifteen turns and nobody comes. When it does beat you, it usually had a resource bonus to do it with.
So the question here is narrow: can you build an opponent that is hard to beat without giving it anything a human does not have? Not fog of war switched off and a gold cheat running — one that sees what you see, learns from what happens, and has to earn it.
The shape of it
A local application on a Windows machine reads Rome Remastered through its own scripting and console interfaces, drives it with synthetic keyboard and mouse, and consults a language model for strategy through Agentic Orchestration running on a second box.
The game process is the host and AO is a service it consults, never the other way around. A move is never blocked waiting on a model — a timeout, an error or a malformed response all resolve to a neutral directive and play continues.
What makes it work
Time stops so the machine can think
toggle_game_update freezes the battle sim, turning a real-time fight into an effectively turn-based one. The agent thinks in seconds; the battle never notices.
The model picks the objective, not the moves
{ "objective": "bleed_and_withdraw", "acceptable_own_losses": 0.35, "required_enemy_losses": 0.90, "abort_if": { "general_dies": true } }
It never selects an action. It declares a target result, and aggression is computed from that — which also makes the intent checkable after the fact.
Nobody writes down what a good move is
No hand-tuned evaluation function. Rome computes scores, casualties and rankings itself — the agent's opinions are hypotheses, the outcome log is the judge.
The component map
Three processes on the Windows box and an engine on a second machine. The split exists so a stalled agent or a hung interface can never leave a key held down.
Process A
Game I/O. Owns the kill switch, never allowed to stall
- WGC window capture P0 done
- Ring buffer and frame selection P0 done
- SendInput actuator P2 done
- RomeShell console channel P2 done
- Log tailers P1 done
- Game state machine P1 done
- Control state machine P3 done
- Intent record writer P2 done
Process B
Agent runtime. Allowed to stall
- Belief store P1 done
- Deterministic predictors P5 not started
- View compositor P6 not started
- Reach client P6 not started
Process C
Overlay UI. Allowed to crash
- Edge glow P3 done
- Virtual keyboard P3 done
- Cursor indicator P3 done
- AO cycle window P3 done
On ada
The engine and the model, on the same LAN
- AO session overlay P6 not started
- The model P6 not started
- The corpora P7 not started
How far along it is
Preconditions and capture DONE
Window detection, WGC window capture of the game window, ring buffer, all three startup self tests.
Observation DONE
Background script emitting state and transitions, log tailers, console query channel, belief store with provenance, age and confidence.
Actuation DONE
Console wrapper with the fair play gate, SendInput actuator with dwell and hover, diff based order computation, verification tiers, intent record, vision located HUD controls and modal dismissal.
Result4 Sep 2026: drove a Julii campaign from turn 1 to turn 25, hardcoded and unattended, verified against Rome's own turn autosaves rather than the run's counters. The run ended when a battle was entered, which belongs to the battle loop phase.
Overlay DONE
Process C, four surfaces, event stream, takeover handshake, three control transitions.
Result5 Sep 2026: `comstar-overlay --self-test` passed against a live game with all five surfaces up — window styles on all five, non-activation, click through at 7 sampled points, and capture exclusion with 0 stray pixels in a 1920x1080 frame. The kill switch half was met first, and mattered more: takeover() deadlocked on a non-reentrant lock and left the lock held, so the kill switch could not run at all. With that fixed, kill releases input from all four control modes.
Campaign UI atlas IN PROGRESS
Parse the game's own string tables and join them to measured on-screen geometry, so every panel has both a meaning and a locator. Catalogue every panel, dialog, control and cursor action on the campaign map: what it is, what it means, what it is for, how it opens, how it closes, and whether it blocks.
Result5 Sep 2026: Julii turn 1 atlas pass — Building Browser opener and unlock line; live Arretium spends (Practice Range 1080, Peasants 100, Hastati 440) with treasury 5000→2300; Lists Military Forces + Family Tree hazard; FIELD CONSTRUCTION (watchtower 200 / fort 500) from Construction <6> with a named general in the field; map orders are left-click once the cursor glyph changes (sword for an army). Phase 4 accept (zero unknowns, every cursor action in atlas, held-out frames) is not closed.
Battle loop NOT STARTED
toggle_game_update freeze cycle, output_unit_positions parse, deployment phase handling, post battle harvest.
Deterministic predictors NOT STARTED
Built from Battle_and_Campaign_Formulae.md: melee and charge resolution, fatigue and morale, auto resolve estimate, siege duration, economy projection, movement reachability.
AO integration NOT STARTED
Reach session overlay, mTLS enrolment, direct_agent calls, view compositor, neutral directive fallback, cancellation on stale.
Learning NOT STARTED
After action records, the experience corpus, offline consolidation into doctrine.
Enforced in code, not by good intentions
| Rule | What it means |
|---|---|
| Fog of war | It sees what a player sees. Every belief carries an age and a confidence that decays. |
| No cheats, ever | add_money, auto_win, force_battle_victory and the rest sit in a hard-blocked set. A run that touches them is tainted and marked. |
| Declares intent | Every action is declared before it executes. No approval gate, nothing waits — but the record exists. |
| Single player only | Automating multiplayer is cheating other people, which is a different thing entirely. |
Windows will lie to you
The interesting engineering here is not the model. It is that the platform reports success for things that did not happen. SendInput blocked by UIPI returns success and does nothing, and Microsoft document that neither the return value nor GetLastError will tell you.
Which is why every action in this system is a closed loop: declare, execute, verify by observation, retry. Not because it is elegant, but because the alternative is a system that appears to work and does not.
Agentic Orchestration
Ten documents behind the build
Architecture, constraints and acceptance tests live here. The code follows them; when the two disagree, the brief wins until it is revised.
Phase 4
Parse the game's own string tables and join them to measured on-screen geometry, so every panel has both a meaning and a locator. Catalogue every panel, dialog, control and cursor action on the campaign map: what it is, what it means, what it is for, how it opens, how it closes, and whether it blocks.