GPT-6.2-Astra Is the New King of the Game Agent Coding League
After a long break, I decided to update the Game Agent Coding League. If you’re new to GACL, here’s how it works. Language models write code for agents that play games. The agents compete in a league, and their results go on a leaderboard. The important detail is that the agents make every move themselves. Their code is generated offline before the tournament, and from then on the agents compete on their own, with no need to ask the language models what move to make each turn.
On November 21, I ran a tournament across eight games with agents written by the 12 models I considered strongest. The agents played more than 200,000 matches in total.
I couldn’t try Carbon in the Gemini app because I don’t have a Google AI Ultra subscription. I was especially curious about Neon. On the Artificial Analysis Intelligence Index, Neon (xhigh) scored 61 and Carbon (high) scored 60. I included Muse Wonder in the tournament because only Muse Code subscribers can access Muse Wonder Full. You can see the 12 models I selected.
If you’re curious about the prompts I gave the models, where I generated the agents, how I calculated the scores, or any of the other details, you’ll find them on the benchmark site. Game Agent Coding League
GPT-6.2-Astra’s undisputed lead
Astra finished first in all four overall Elo rankings. The four rankings come from combining two filters. The first lets you give all games equal weight or apply different weights to different games. The second lets you use each model’s best-performing agent in each game (Max Elo) or average the ratings of its agents (Mean Elo). I expected Astra to win. What surprised me was how far ahead it was. When I weighted all eight games equally and averaged the Elo ratings of each model’s two agents, Astra finished 40.34 points ahead of GPT-6.2-Sol in second place. In that same ranking, second and fourth were just 7.89 points apart. It can be hard to see the difference between Astra and Sol in everyday use. Here I can directly compare the results of agents playing by the same rules.
Astra finished first in Battleship, Connect4, and WordMatrix. It made the top four in seven of the eight games by Max Elo. The Battleship result caught my eye the most. Astra’s best agent won 2,893 of 3,520 matches (82.2%). In Connect4, Astra’s two agents took the top two spots. They won 45 of 64 matches against Sol’s agents and drew three. Tournament results
Why did the Claude models fall behind?
Most tests online leave the desired result loosely defined. That’s especially true of visually focused YouTube comparisons with open-ended prompts like “make me this,” “design a city,” or “build a game.” There are thousands of ways to give the user a result they’ll like. I think that’s exactly where the Claude models stand out. When the output is loosely specified, Claude pays attention to fine details and spends more time and tokens, taking the quality of the result to another level.
Given similarly vague prompts, GPT models use roughly half as many tokens as Claude and leave the gaps in the prompt alone. They deliver what was requested without getting bogged down in details. It reminds me of an article I read a few weeks ago, though I can’t remember where. Developers who’ve spent years writing software prefer GPT because they want it to stick to what they asked for. People with little or no software experience who are trying to build something with AI tend to gravitate toward Claude because it fills in the gaps in their prompts and surprises them with what it comes up with.
How does that relate to this benchmark? There’s little room for creativity in the usual sense here. In GACL, creativity goes into building a smarter agent within the brief. The benchmark measures pure intelligence directly, with a bit of coding skill thrown in. Each game has a time limit, so poorly optimized code makes it harder to find a good move.
I’ve gone on long enough, so here’s the short version. If you want the model to do exactly what you asked and leave everything else alone, steer clear of Claude. If your prompts are light on detail, you’re unsure about what you’ve written, and you want the result to impress you, stick with Claude.
Gemini is so back!
When the new Gemini models came out, I heard plenty of praise for them on forums and YouTube. I was especially pleased with Neon’s result in Wizard. It beat Astra to take first place in the six-player game. In the equally weighted Max Elo ranking, it landed between Sol and Opus. Hydrogen finished sixth, comfortably ahead of Sonnet.
So who uses Sonnet?
I think Haiku is good for the price, though I left it out of this round. With Haiku at one end and Opus and Fable at the other, I’d been wondering what tasks I’d choose Sonnet for. Sonnet 5.5 beat every open-weight model and finished just behind Hydrogen. What caught my eye was the gap between Sonnet and Sol. Sol wrote better agents for these games, and both models have the same standard API pricing. Each costs $2 per million input tokens and $10 per million output tokens. Anthropic pricing · OpenAI pricing
DeepSeek was the open-weight standout
DeepSeek-V4.1-Flash-1017 finished eighth. It was just 11.39 Elo points behind Sonnet and 45.58 points ahead of Kimi K3.1 Pro. I hadn’t expected an open-weight model to get that close to Sonnet this round.
I used the DeepSeek Harness to generate the agents for the open-weight models. That could have given DeepSeek’s own model an edge. My guess is that the effect was small. How the agents were made
GPT-6.1-Terra, stick with the visuals
Terra finished 11th out of 12, behind DeepSeek and Kimi and ahead of the closed-weight Qwen4-Max. I’ll keep using Terra for visual tasks such as image annotation and OCR. I’ll rarely reach for it for anything else.
Which result caught your eye the most? If you’d like to talk about the results or spot a mistake I missed, feel free to get in touch. See you next time!