• 2 Posts
  • 12 Comments
Joined 7 days ago
cake
Cake day: November 9th, 2026

help-circle
  • You say Opus 5.5 “gets 100% anyway” but did you read the footnote?

    • AGENTS.md points agents to bundled Next.js documentation. Both conditions can access these docs. With AGENTS.md shows the success rate with that initial guidance.

    All models including the frontier models had the bundled docs available the whole time. Your 100% just shows that the AGENTS.md pointer wasn’t necessary for some of them. Which is, if anything, another point for CoDoc ;)

    Now here’s the part you skipped: GLM5.2 (open-weight model) also hit 100% with the AGENTS.md guidance, at 32% less cost than the Opus 5.5 you use “basically 100% of the time.” So the frontier model isn’t magic. The context is. And that context is free with CoDoc for all models.

    “It’s a very conservative estimate based on my experience.” So it was a vibe after all… And your vibe, built on:

    • “I haven’t checked these, but go and look at e.g. LLVM, Git, libpng, SDL, Qt, etc.”

    Cherry-picking five eyeballed repos as evidence for “50% of software” is just hilarious. And you’re mixing up two different things. I didn’t ask whether docs live in the same Git repo. I asked whether libraries ship complete, AI-optimized documentation inside the package artifact. So here we have someone who eyeballs five repos and calls it a “very conservative estimate.” 👏

    “Frontier models are happy to reverse engineer binaries without prompting. Reading a bit of source code is nothing to them.” Great, here we have someone advocating for letting frontier models waste compute power and frivolously consume your tokens to do what co-packaged documentation does for free. How about we put some of those data centers next to your place?

    And by the way, two things:

    1. The paper disarms your argument of models looking at source-code, section 10.5 if you care to read it. There is a reason why humans have built documentation wikis, sites and portals for years.

    2. You might want to read this: Claude Opus 5.5 Reverse Engineering Refusals

    “No I’m not dumb enough to use dynamic typing.” The point was never about typing. It was about a model hallucinating your v5 API signatures.

    Since you use Opus 5.5 basically 100% of the time, let me recall something for you. On June 12th, a single export control letter switched Fable 5 off for every customer worldwide for 17 days. Opus 5.5 is one letter away from the same fate. Besides, even if they do not share the same fate via the export control letter, the company behind the model you rely on is not profitable (it is actually very far from it). As you may know, it will soon IPO to finance their operations and service their existing debt. The irony: they have frontier models under their fingertips, yet their own models cannot solve their business model or their financing problems. You are more than welcome to put all your oranges in one basket.

    TLDR: a vibed estimate from five unchecked repos, completely dismissing the benefits of CoDoc for ecosystems with dynamically-typed languages, and a frontier model that “gets 100% anyway” which sure, it does: with the bundled docs that CoDoc ships.

    I’ll be here if you want to come back with solid evidence. If, however, you do not want to show the evidence behind your claims and you just wanna play the trust me bro game, please play it with some naive people. It ain’t gonna work with me ;)


  • I would say maybe 50% of software already does that.

    Would you care to show some evidence, or is it a vibe?

    For example 100% of Rust crates ship the docs with the code.

    True, and that’s actually a point for CoDoc, not against it. Rust already does the right thing. The question is what the other 90% of the whole software ecosystem is doing.

    AI will happily skip the docs and just figure things out from the source code, which is usually better anyway.

    Show us the evidence. Vercel’s evals say the opposite: 100% pass rate with bundled docs vs 53% without, on APIs absent from training data. It got it wrong 47% of the time. The docs weren’t skipped… they were the difference.

    Using which model? They don’t say? Huh.

    Let me save you some tokens: https://nextjs.org/evals

    Frontier models very rarely hallucinate anymore, and they’re actually really good at writing tests now

    How does that address a frontier-model hallucinating your v5 API signatures? Would you like to find this out at runtime and burn more tokens for the corrections?

    Spoiler: If you look at the Next.js evals here you will see that even GPT 6 Astra (high) benefits from bundled documentation.


  • Not surprised that your neighbors haven’t heard of them…

    What really surprises me is your claim here: “any software engineer worth their salt has” heard of Amadeus. So the bar for being a competent software engineer is now name-recognition of a specific travel-tech company? Let me pass that to HR: “have you heard of Amadeus?” is going straight into our next interviews.

    And as for your AI-generated diatribe claim, I invite you to look up the meaning of diatribe. A well articulated and thought-provoking post is not a diatribe.

    • Post and comment 100% written by a human, in case you had any doubts :)


    • CoDoc (Co-packaged Documentation) is a standard advocating for colocating AI-optimized documentation next to the code installed for each software library used in your project.

    • Understand-Anything is a Claude Code plugin that maps your existing codebase into a knowledge graph.

    They are complementary. Software libraries following CoDoc would supply additional files and directories for each dependency in your project enriching the output of the Understand Anything plugin.

    I also understand that they solve different problems. CoDoc would allow an AI agent to operate in auto-pilot mode (with less AI slop) by having all the necessary context locally for all your project’s dependencies. CoDoc enables the AI agent.

    Understand Anything maps your codebase to help you navigate and understand a complex codebase (it builds a JSON graph). Understand Anything enables the human software engineer.





  • You called it “pure drivel” and then quoted the one line that’s a file-naming convention, not the argument… The moat in CoDoc is that the documentation ships inside the package, version-matched, offline, deterministic. Can you tell me what proportion of software libraries today ship complete AI-optimized documentation within the library’s artifact? I’ll wait.

    And to your frontier-model guess, the models behind the research agents are: Opus 5.5 and Astra. So no, not slop.

    As for “open weight models are nowhere near the capabilities of frontier models”: sure, on raw benchmark scores (but they always end up catching up). Vercel’s evals on Next.js 16 APIs (absent from training data) show 100% pass rate with bundled docs vs 53% without. As stated in the post, the bottleneck for a 27B local model isn’t raw intelligence, it’s context. CoDoc fixes the context for free. A 27B on your laptop with the right context can beat a frontier model that’s hallucinating your v5 API signatures.

    Keep letting the Goliaths vacuum your code, your tokens, and your money. The fizzy orange drink is on them.

    P.S. Post and comment 100% written by a human, in case you had any doubts :)