Versioned, reproducible benchmarks of AI app builders - methodology-first and version-stamped.
76 posts
Every timing a developer has was taken from a system their own presence kept awake. A pre-registered seven-signal axis for what the first visitor after a quiet period actually experiences.
Every length limit is checked with a value the developer typed themselves, and on that value all four units agree. A pre-registered seven-signal axis for whether the value submitted is the value stored.
Every export is checked by the one reader whose settings match the writer's. A pre-registered seven-signal axis for whether a generated application's export survives arriving anywhere else.
A generated app calls a third party's API from inside a request. We propose an axis for what it does when no answer comes back, and find that retrying is a layered property no single document describes.
One user cannot exercise a shared connection pool. We propose an axis for whether a generated app survives ten people at once, and find the unbounded wait is the default at three separate layers.
Exporting the code is the easy half. We answer the practical question, using BuilderProof scores already published rather than new ones, and name the reason the obvious test gives a false pass.
A generated app almost always issues a credential at sign-in and almost never decides when it stops working. We pre-register a seven-signal rubric for session lifetime and revocation, and name the trap that hides it.
Two strings a human reads as one name can be two different byte sequences, and the default comparison in a generated app calls them different. We pre-register a seven-signal rubric for text-comparison and collation correctness, and name the trap that hides it.
A webhook endpoint that never verifies its caller behaves identically to one that does, for every request an honest provider ever sends. We pre-register a seven-signal rubric for inbound webhook verification in generated apps, and name the trap that hides it.
A proposed BuilderProof axis measuring whether the applications AI app builders generate store and compute money exactly, or only render it that way. Seven weighted signals, four postures, a ten-step protocol with a control step. Pre-registered: no builder is scored here.
A proposed BuilderProof axis measuring whether the applications AI app builders generate actually deliver security response headers, and whether the policies they do ship bind where they are delivered. Seven weighted signals, four postures, a ten-step protocol with a control step. Pre-registered: no builder is scored here.
A candidate BuilderProof benchmark axis scoring whether the scheduled and deferred work a generated app emits actually executes, executes once, and leaves evidence when it does not. Seven weighted signals, four postures, a ten-step protocol, drafted September 2026 and open for comment.
