KnowledgeForge Open Source — One Week In

The origin post two weeks ago ended at a cliffhanger: 43 modules, “anxiety in Markdown,” and a promise to explain what I did about it.

This is that explanation. Plus the launch.

How it started

In 2023, I had a ChatGPT Project with a growing stack of files. Not a framework. Just files. Instructions I'd written. Patterns I'd noticed. Things I wanted the model to carry between sessions.

The problem: the files weren't linked. Same question, different session, different answer. So I started asking the model to improve how they were organized. I noticed a pattern in what actually produced consistent output. I had it generalize that pattern into something reusable.

That became KnowledgeForge. The name made sense because it was forging the knowledgebase into something that held its shape.

By 2024, when I migrated to Claude, it had stopped being a knowledge base. It had become a reasoning framework: a structured way of thinking through problems before asking the model to solve them.

By late 2025, it had 43 modules.

Forty-three.

What 43 modules means in practice

It means you've solved every problem you've ever had, in advance, in writing, including the problems you don't have yet.

Each module addressed a real failure mode. Decision classification. Confidence calibration. Adversarial verification. Knowledge accretion. Every one of them earned its way in by something that actually went wrong.

But a framework with 43 modules is not a framework. It's a documentation project. The overhead of knowing which module applies when, of understanding how they interact, of keeping the whole thing coherent across updates: it collapses the thing it was supposed to improve. The model can't hold 43 modules of instructions with consistent fidelity. Neither could I.

The solution wasn't to delete modules. It was to find what the 43 were actually doing and express it as fewer, cleaner ideas.

What it became

Nine modes. Twenty-five modules. A compiler that generates platform-specific versions from a single source.

The nine modes are the core:

Builder for creating things. Spec, implementation, architecture, RFC. Forces the question: what are the constraints?

Critic for reviewing things. Finds gaps, contradictions, unstated assumptions. Read-only. It never modifies what it reviews.

Debugger for diagnosing failures. Requires 0.8 confidence before declaring a root cause. Not a best guess. A threshold that forces the question: is there enough signal here to commit to this direction?

Strategist for decisions. Evaluates options with explicit trade-offs. Forces prioritization and never recommends doing everything.

Expert for adversarial depth. Second-order effects, blast radius, compound failures. Goes further than first-order findings.

Synthesizer for extracting patterns. Turns examples into reusable frameworks. Every pattern must come with anti-patterns.

Calibrator for AI coder configuration. Right-sizes guardrails for the project. Hobby projects don't need enterprise scaffolding.

Coordinator for multi-agent workflows. Maps dependencies first, derives coordination pattern from the dependency graph.

Navigator for genuinely ambiguous requests. Fires when the same input could route to two different modes and produce meaningfully different outputs. Asks one targeted clarifying question.

Routing is automatic. KF classifies every request before doing anything else. Verifiable answer: respond directly and stop. Evaluative judgment: route to the right mode. Novel judgment: expand reasoning and flag for human review.

The thing raw Claude skips, and that most prompting doesn't fix, is the second question. A Builder asks what constraints exist. A Critic asks what the spec assumes. A Strategist asks what you're deferring. KF makes those questions structural rather than optional.

Where the confidence threshold matters

The 0.8 threshold on the Debugger is the clearest example of what the modes actually do differently.

Without a threshold, a model will declare a root cause. It might hedge with “probably” or “likely,” but it commits to a direction. The threshold forces a different question: is there actually enough signal to say this? If not, what would change that?

The clawberbot autonomous system is the clearest proof I have. It ran 23 consecutive diagnostic cycles across six weeks without promoting a single fix. Win rates between 38 and 60 percent. The debugger kept identifying root causes, the strategist kept selecting fixes, the builder kept implementing them. None worked.

The problem wasn't the reasoning. It was the signal going into the reasoning.

The match data was mixing three different opponents into a single diagnostic context. KF was finding the statistical average of a noisy distribution and treating it as a pattern. And it had no memory of what it had already tried, so it kept arriving at ammo management as the root cause because nothing told it that 17 prior versions had already covered that ground.

Two infrastructure changes fixed it: focus all diagnostic replays on one opponent, and prepend a log of prior attempts with an explicit instruction to find a different root cause. KF found in one pass what 23 iterations hadn't.

That case study has its own post. The point here is narrower: the threshold surfaces when something is wrong with the inputs, not just the outputs. A model that confidently declares a root cause on bad signal is harder to debug than one that stops and says it needs more.

The honest benchmark

I ran KF against GAIA a few weeks ago. The result was a null.

KF and a well-written generic prompt scored identically on GAIA's tasks. The reason was specific: every task in the sample was what KF calls a “reckoning,” a direct-answer question where specialized reasoning modes don't apply. KF correctly classified them as reckonings and answered plainly. The modes never fired.

A reasoning framework can only be tested on tasks that demand reasoning of the kind it scaffolds. GAIA's difficulty axis is multi-hop retrieval and tool use, not open-ended design, diagnosis, or judgment. Those are different axes.

The honest boundary: KF doesn't help on deterministic-answer work. Where it helps is the work that has unstated assumptions, noisy signal, real trade-offs, or outputs worth reviewing before you act on them. Design decisions, debugging, architecture review, option evaluation. The full GAIA post has the methodology and the uncomfortable part about what the telemetry actually showed.

What's open

The framework is at internexio.com/knowledgeforge. GitHub: internexio/knowledgeforge.

What's in the repo:

• All 25 module specs
• The compiler that generates platform-specific versions for Claude Code, Claude Projects, and Claude Web
• Model profiles, per-model weakness and strength maps the platform bindings reference
• The routing test suite
• GAIA benchmark results and harness

Install for Claude Code:

curl -fsSL https://raw.githubusercontent.com/internexio/knowledgeforge/main/install.sh | bash

If you're working with AI tools daily, writing specs, debugging systems, reviewing plans, making architectural decisions, KF is worth a look. Not as a one-time configuration, but as a set of questions the framework forces before you act on the model's first answer.

Three years of building it in private. Now it's out. More on what's next in the coming weeks.