Tag
23 posts tagged.
The 2026 prompting guides of Lovable, Bolt and v0 all say prompt wording changes the output. So a single prompt cannot benchmark a builder. Why prompt sensitivity is a validity threat, and what a fixed, versioned prompt suite looks like.
AI app builders swap default models and scaffolding almost weekly. Read from five 2026 vendor changelogs: why an undated benchmark score decays, and the versioning protocol BuilderProof uses to keep scores reproducible.
Three rigorous, independent AI app builder benchmarks now exist: UI-Bench (design), From Prompt to Product (human end-to-end), and BuilderProof's six-axis rubric. What each measures, mapped side by side from the primary sources.
As of July 2026 the five builders in our cohort bill in five incompatible units. We propose a sixth BuilderProof axis, Cost-to-Ship, that fixes the workload and measures real dollars to a deployed app. Open for review until August 3, 2026.
A proposed community-editable BuilderProof axis scoring how well the app an AI builder generates protects sign-in and per-row data access. Five 20-point sub-axes, a fixed protocol, and a provisional documentation-based cohort table for July 2026.
First-build scores rate one generation. Most real work is the follow-up edit. We propose iteration fidelity: a five-part rubric, a repeatable protocol, and a provisional July 2026 cohort table.
We are proposing a fifth BuilderProof axis to score whether an AI app builder ships code that can leave the platform. Five sub-axes, 0 to 100, provisional cohort scores included.
Effective with the H2 2026 ranking, BuilderProof retires binary 'first-build success' as a scored axis, on the basis of documented vendor changes through H1 2026 rather than any first-party run. It becomes a precondition (must pass to be ranked) and the rank weight moves to time-to-first-functional-build.
Two Lighthouse runs on the same deployed AI-builder output rarely return the same score. This note documents the variance phenomenon, names its seven sources from Google's own documentation, and publishes the median-of-five protocol a hands-on deploy-quality harness would have to meet. Corrected August 21, 2026: the June 2026 result table it accompanied has been withdrawn, and the account of how that table was produced is withdrawn with it.
Proposing first-build stability as the fifth BuilderProof axis: the fraction of OQ-7 prompts that complete without manual intervention. Failure-mode taxonomy, measurement protocol, scoring rubric and open questions, dated June 20, 2026.
BuilderProof methodology v1.1: the published rubric, brief OQ-7, environment standards and weights used to score AI app builders on output quality, speed, deploy quality and agency suitability. The four June 2026 result sets were withdrawn on August 21, 2026 as placeholder data, so the lab currently publishes method, not scores.
