29 Versions, 23 Failures, and the Three-Line Fix That Changed Everything

I built it by hand first. Sixty-eight versions over four months. Twelve thousand matches. SEMalytics reached #1 on the Clawber.ai leaderboard.

Then I asked a different question: could an AI system do that same thing without me in the loop?

That experiment produced two public repos:

• internexio/clawberbot: the manual build. 68+ versions, every diagnostic session, the full four-month development log.
• internexio/clawberbot-auto: the autonomous system. 33 versions, zero human direction after launch, matched the manual baseline in two weeks.

This is what both processes looked like.

The Arena

Clawber.ai is a competitive programming arena. Each bot is a single JavaScript function (update(input)) called every game tick. Five bots per team. Abilities cost ammo and deal AoE damage. Melee is free but short range. The ELO ladder means at high rank you face the same two to four opponents repeatedly, which turns out to matter a great deal.

The Manual Bot

Four months. Sixty-eight versions. Roughly 12,000 ladder matches.

The process was the same every cycle: watch replays, identify the failure pattern, run a KnowledgeForge diagnostic session, implement the fix, submit, wait for enough matches to read a signal. Each cycle took one to three days. The bottleneck was always me: deciding what to look at, forming hypotheses, choosing what to change.

The development log breaks into four eras:

Pack tactics (v1–v18): Formation geometry and terrain navigation. Getting five bots to move as a unit rather than scatter. Basic ammo management. Staying out of blast radius.

Ammo system adaptation (v19–v33): The game changed its balance. Ability costs shifted. These versions were pure reactive adjustment, adapting to the new economy.

Counter-strategy (v34–v44): At ELO 2,200+, the same two opponents dominated the loss column. These versions targeted specific matchup patterns rather than general mechanics.

Surgical improvement (v44–v68): Isolated, one-variable-at-a-time changes. Weakest-enemy targeting. Publisher-follower role assignments. Health-based retreat thresholds. Each fix was designed to be measurable in isolation.

The bot peaked at ELO 2,585 and the #1 position on the leaderboard.

KnowledgeForge assisted every diagnostic cycle, but I was in every loop. The human attention requirement was the limiting factor: each version needed someone to frame the problem, read the replay, and decide what to change. Four months to build something a human could be proud of.

The Autonomous System

The target: match ELO 2,585 starting from nothing, with zero human involvement after launch.

The autonomous loop runs every 60 minutes. It fetches match history, applies a Wilson 95% confidence interval gate, and makes one of three decisions: wait for more matches, promote the current version if the win rate clears the threshold, or trigger a DIAGNOSE cycle.

The Wilson CI matters because it enables early stopping. A bot at 2W-8L doesn't need 40 matches to know it's failing. The confidence interval upper bound falls below the threshold around N=12, saving 28 wasted ladder matches per bad version.

When DIAGNOSE fires, it hands off to KnowledgeForge:

1. kf-debugger: ingests frame-level replay data (death ticks, HP and ammo trajectory over time, team spread, ability cast counts) and must reach 0.8 confidence before declaring a root cause
2. kf-strategist: evaluates fix options and selects the most surgical one; no new thresholds unless required; documents explicit trade-offs
3. kf-builder: implements the fix as a complete, self-contained bot file; under 50KB, no imports, single exported function
4. kf-critic: reviews every edge case, null input, syntax error, and correctness issue; blocks submission if critical issues found

The new bot gets submitted to the live ladder. The loop repeats.

Six Weeks of Failure

From June 14 to July 29, the loop ran 23 consecutive DIAGNOSE cycles. Win rates between 38 and 60 percent. No version broke through.

Looking back at the version log, 17 of the first 21 iterations made some form of ammo management change. v3 through v8 tuned foraging thresholds. v9 through v15 adjusted kill targeting and ammo floors. v16 through v19 recalibrated roles and ammo. v20 through v23 tried finish bonuses and formation scattering.

The problem wasn't KnowledgeForge's reasoning. It was the signal KF was given.

The match history API returns opponent names as "unknown" for historical matches. KF was seeing mixed loss samples: three different opponents, three different tactical situations, averaged together. The diagnosis was landing on the statistical middle of a noisy distribution. And KF had no record of what it had already tried; each iteration started fresh and often identified ammo management as the culprit again.

The Signal Fix

Two structural changes to the diagnostic context pipeline made the difference.

Opponent stratification. The loop now fetches individual match summaries to get real opponent names, then focuses all five replay slots on the dominant opponent (the team causing most of our losses in the gate window). KF sees the same formation, the same ability timing, the same death patterns across five replays. Clean, consistent signal instead of a mixed sample.

Failed-hypothesis log. Every prior version, what it changed, and what the gate returned, prepended to the context with an explicit instruction: "Your root cause MUST differ from every title in this table." Multi-session memory, in writing. KF could no longer rediscover ammo management as the root cause; the log told it that ground had been covered 17 times already.

With those two changes in place, KF found in one pass what 23 versions had missed.

The Breakthrough

Version 24. July 29. KF's diagnosis, verbatim:

"Basic 'attack' action was almost entirely unused: gated on an unverified assumption that it costs ammo, restricted to only the last surviving enemy, and blocked whenever ability wasn't recently cast, leaving the bot passive on offense during the extended zero-ammo stretches that dominate match duration."

Three gates had stacked on the melee attack simultaneously.

Bug 1: The code assumed melee costs ammo. That assumption was never verified against the game spec. Melee is free.

Bug 2: The attack was restricted to only the last surviving enemy, so in a full 5v5 engagement it almost never fired.

Bug 3: onAbilityCd means "ability was just cast," not "ability is unavailable." The condition was blocking attack on every normal tick.

The fix: fire attack as free supplemental damage on any tick we didn't cast an ability, targeting the weakest enemy in melee range. Three lines changed.

Win rate went from 28% to 100%.

Five Straight Promotes

The melee unlock didn't just fix one bug. It revealed four more structural bugs that had existed since the beginning, invisible because the bot rarely survived long enough to reach them.

v25: The zone-retreat hard override was nested inside an if-not-action block. Whenever melee fired, the zone retreat was skipped. Bots were fighting outside the shrinking safe zone.

v26: Critically wounded bots were chasing ammo instead of retreating.

v27: A health gate was blocking ammo-seeking even at zero ammo. This is why 17 versions of ammo-foraging threshold tuning had never worked: the parameters lived inside a branch most bots structurally never reached. Seven prior fixes had been adjusting a dead code path.

v28: The retreat threshold was static with no burst-damage awareness. Bots focused by two enemies simultaneously crossed into lethal range before the trigger fired.

Each fix promoted cleanly. Five straight promotes, ELO from roughly 2,200 to 2,584.

The Comparison

The autonomous bot reached ELO 2,584. The hand-crafted original: 2,585.

They're not the same bot. The original uses a publisher-follower role system with weakest-enemy targeting to maximize kill conversions. The autonomous bot, built from scratch by KF, runs all five bots symmetrically with fire-sync coordination targeting AoE clusters instead.

Two different architectures. Same ELO. The autonomous system got there in three days of active iteration once it had the right signal. The original took four months of human-in-the-loop sessions.

What Made It Work

Four design decisions:

Wilson CI early stopping. Act on 12 matches instead of 40. Save 28 wasted matches per bad version.

One change per version. If a version promotes, the fix worked. Multi-variable changes make causality impossible to trace.

Failed-hypothesis log. KF doesn't repeat ground already covered. Durable memory across sessions, written into the context.

Opponent stratification. Five clean replays of the same matchup. Precision diagnosis requires precision signal.

The punchline: KF's reasoning was always there. The diagnostic capability existed from version 1. What it needed was the right context: what had been tried, and a consistent signal. That's the infrastructure problem, not the reasoning problem.

The manual bot proved the same thing from the other direction. Seventeen consecutive ammo-tuning iterations happened on the human-in-the-loop side too, before the architectural shift to counter-strategy in v34. Humans and autonomous systems make the same class of mistake when the signal is noisy: they optimize the wrong variable with great precision.

What's Running Now

The autonomous loop is still active. v29 through v33 are live. The system keeps iterating.

Both repos are public:

• internexio/clawberbot: every manual bot version (v1–v68+), diagnostic session logs, development eras, the full four-month build history
• internexio/clawberbot-auto: the autonomous pipeline, architecture docs, all 33 versions, the diagnostic loop scripts, SYSTEM.md and AUTONOMOUS.md operator guides

The KnowledgeForge framework that powered both is going open source at internexio.com/knowledgeforge.