Tag
35 posts tagged.
A proposed BuilderProof benchmark axis measuring whether the file-upload path an AI app builder emits is owner-scoped, content-verified, bounded and durable, or whether it only works because the developer testing it is the only account in the app. Six weighted signals, four posture levels, a reproducible protocol.
A proposed BuilderProof benchmark axis measuring whether the app an AI builder generates can actually deliver transactional mail to a real user, not just to the developer who built it. Six weighted signals, four posture levels, and a reproducible protocol.
Six empirical papers on AI app builders and vibe coding landed between June 17 and August 17, 2026. Only one measures the products people buy. What each actually measures, its numbers, and the limitations its own authors state.
A candidate BuilderProof benchmark axis that scores whether the code AI app builders emit stays correct when two writes to the same record overlap. Rubric, four posture levels, a reproduction protocol, and an open call for comment.
A candidate BuilderProof benchmark axis that scores whether the HTTP surface an AI app builder emits is internally coherent across every endpoint: one error shape, uniform status-code semantics, one addressing scheme, consistent collection semantics, and a machine-readable contract. Divergence-based scoring, four documented postures, a probe-sweep protocol, and an open call for comment.
A candidate BuilderProof benchmark axis that scores the crawler-facing artifacts AI app builders emit by default: per-route titles, canonicals, social cards, robots.txt, sitemap.xml, structured data, and whether route content reaches a crawler at all. Rubric, four rendering postures found in the vendor docs, a dual user-agent reproduction protocol, and an open call for comment.
A candidate BuilderProof benchmark axis that scores whether the code AI app builders emit validates untrusted input at the server boundary, or trusts whatever the client sends. Rubric, four posture levels, an adversarial-payload reproduction protocol, and an open call for comment.
A proposed BuilderProof benchmark axis measuring whether the code an AI app builder emits can actually build, run, and be self-hosted off the vendor platform, not just be exported.
Dates are the bug that ships silently: correct on the builder's clock, wrong for everyone else. A proposed, reproducible axis for scoring how AI app builders handle time zones, UTC storage, and daylight saving.
A candidate BuilderProof benchmark axis that scores the static type safety of the code AI app builders actually emit, measured from the untouched export with no runtime. Rubric, reproduction protocol, and an open call for comment.
A neutral, reproducible methodology proposal: how to measure whether AI app builders ship request throttling and brute-force protection by default, and where that protection actually lives.
State-handling completeness is a proposed BuilderProof benchmark axis (August 2026) that scores how well an AI app builder generates the non-ideal runtime states of the apps it produces: loading, empty, and error states. It is a 20-point axis across five sub-criteria, measured reproducibly by giving all five commercial builders (v0, Lovable, Replit, Base44, Bolt.new) an identical fixed prompt and then inspecting the generated app under a throttled network, an empty account, and a forced request failure.
