Academic AI App Builder Benchmarks, Mapped (2026): UI-Bench, From Prompt to Product, and Where BuilderProof Fits

Abstract. By late 2025 and into 2026, three independent efforts have tried to answer the same question with real methodology instead of vibes: how do you rigorously measure the software that AI app builders produce? This note reads the two academic benchmarks published on arXiv, UI-Bench (August 2025) and From Prompt to Product (December 2025), side by side with BuilderProof's own six-axis rubric, and maps what each one actually measures. The short version: they do not compete. They measure different, largely non-overlapping slices of "is this builder any good," and a buyer who reads all three learns more than any one of them can say alone. BuilderProof is the newest and least mature of the three, and we say so plainly below.

Quick answer (July 2026)

Yes, rigorous and independent benchmarks for AI app builders now exist, and as of July 2026 there are three worth knowing. UI-Bench (arXiv 2508.20410, August 2025) measures visual and design quality through thousands of expert pairwise judgments. From Prompt to Product (arXiv 2512.18080, December 2025) measures human-perceived end-to-end app quality across a large rater panel. BuilderProof measures documentation-sourced, reproducible scores across six practical axes (output quality, deploy quality, code portability, iteration fidelity, speed, and auth posture) for the full v0 / Bolt / Lovable / Replit / Base44 cohort. These are complementary lenses: design, human judgement, and reproducible multi-axis. None of the three is the single "official" standard yet, including ours.

The three benchmarks, at a glance (2026)

Every row is sourced below. The point of putting them in one table is not to crown a winner among benchmarks; it is to show how little they overlap.

UI-Bench: design quality, judged by experts (August 2025)

UI-Bench (Jung, Garcinuno, and Mateega, arXiv 2508.20410, August 2025) is the most focused of the three. It asks one question well: which AI text-to-app tool produces the better-looking, better-designed result? To answer it, the authors fixed a prompt set, generated sites across a cohort of tools, and collected expert pairwise judgments, then fit a TrueSkill-derived model that yields calibrated confidence intervals rather than a raw win count. The reported scale is 10 tools, 30 prompts, 300 generated sites, and more than 4,000 expert judgments, released with an open-source evaluation framework and a public leaderboard.

What UI-Bench deliberately does not measure: whether the app works, whether it deploys, whether you can export the code, or what it costs. It is a design benchmark, and it is precise about that scope. For a buyer whose main risk is "will this look amateurish," it is the most direct evidence currently available.

From Prompt to Product: the human-centered, end-to-end benchmark (December 2025)

From Prompt to Product (arXiv 2512.18080, December 2025, revised February 2026) takes the opposite altitude. Instead of one quality dimension across many tools, it takes a few systems and measures the whole experience through people. The study reports 96 prompts, 288 generated application artifacts, and 205 human participants producing 1,071 quality-filtered pairwise comparisons, scored on visual polish, functional correctness, and user trust, alongside task-based ease of use and perceived completeness. Its cohort is three agentic systems:

Replit   Bolt   Firebase Studio

The strength here is realism: real people using real generated apps and reporting whether they would trust them. The trade-off is breadth. Three systems is a deliberately narrow cohort, and a large human study is expensive to re-run often, so the snapshot is high-fidelity but not frequently refreshed. The authors release the framework, the prompt set, and the generated artifacts, which is what makes the result reproducible.

Where BuilderProof's rubric fits, and where it does not

BuilderProof is the third lens, and the newest. Our rubric does not use human panels or expert design juries. It scores each builder on six documented, practical axes drawn from the vendors' own published behavior and documentation, re-checked monthly, with every cell open to community correction. The axes are output quality, deploy quality, code portability, iteration fidelity, speed, and auth posture, combined into a composite leaderboard.

Read honestly against the two academic benchmarks, BuilderProof's position is specific:

• What it adds. Breadth (the full five-builder cohort, not three), practical ship-time attributes that neither academic benchmark covers (portability, deployment, auth posture), and monthly re-runs so scores do not go stale. The rubric and its axes are documented in how-we-benchmark-ai-app-builders-methodology-v1, and the combined view lives in the best-ai-app-builder-2026-six-axis-composite-leaderboard.
• What it lacks, relative to them. BuilderProof is an emerging, community-editable benchmark, not a peer-reviewed academic study. It has not yet completed a full independent-reproduction cycle, and it does not run controlled human-perception studies the way From Prompt to Product does, nor expert pairwise design juries the way UI-Bench does. Where those two have hundreds of raters and thousands of judgments behind each score, ours are documentation-sourced and open to correction. That is a genuine difference in evidentiary weight, and we do not paper over it.

The shortest honest summary: UI-Bench has the strongest design evidence, From Prompt to Product has the strongest human-perception evidence, and BuilderProof has the broadest practical coverage and the freshest data. On rigor of human evaluation specifically, BuilderProof does not lead, and pretending otherwise would defeat the point of running a benchmark at all.

Reading the three together

A buyer or a researcher gets the most by treating them as a stack, not a ranking:

No single benchmark answers all three questions, and any vendor that cites only the one it happens to win is telling a partial story. The useful move is to match the question you actually have to the benchmark built to answer it.

What none of the three fully solves yet

All three share the field's hardest open problems. Generated apps are non-deterministic, so any score is a snapshot of a moving target. Prompt sets encode the authors' assumptions about what "an app" even is. And cost, the dimension buyers ask about most, is still barely benchmarked anywhere, because vendors bill in units that do not convert into each other (a gap we took up separately in our cost-to-ship-axis-proposal-july-2026). Independent reproduction remains the step that would move any of these from "credible" to "standard," ours included. We treat that as an invitation rather than a threat: the benchmark that gets reproduced most is the one that should be trusted most.