State-handling completeness: a proposed benchmark axis for AI app builders (August 2026)

AI app builders are very good at the happy path. Ask one of them for a bookings dashboard and you get a clean table, styled buttons, and a working create form in minutes. Then a real user opens the app on a slow connection, or with an empty account, or the backing request returns a 500, and the same screen goes blank. The demo was never wrong. It just never covered the moments when there is nothing to show, something is still loading, or something broke.

Those three moments have names in interface design: the loading state, the empty state, and the error state. They are the parts of a product a user actually spends a lot of time in, and they are exactly the parts a generator trained to produce a convincing first render tends to skip. This post proposes a new BuilderProof benchmark axis to measure that gap directly, and sets out how we would score it reproducibly across all five commercial builders we track.

What is the state-handling completeness axis?

State-handling completeness scores how thoroughly the app a builder generates handles the non-ideal runtime states of its own data-bound views: while data is loading, when there is no data yet, and when a request fails. It is a property of the generated application, not of the builder's chat UI or its own dashboard.

Concretely, for every screen that reads or writes data, we ask three questions:

1. Loading: while the data is in flight, does the UI show a skeleton, spinner, or other progress affordance, or does it render nothing (or a flash of broken layout)?
2. Empty: when the query legitimately returns zero rows, does the UI show a purposeful empty state that tells the user what they are looking at and what to do next, or a blank region that is indistinguishable from a bug?
3. Error: when a request throws or returns a non-2xx response, does the UI show a contained, user-readable error with a way forward, or does the whole view crash to a white screen?

The axis rewards apps that answer "yes, gracefully" across all three, consistently, on every data-bound view rather than on one hand-polished demo screen.

Why this axis matters

The failure mode is well documented, and it is specific to generated apps. A widely shared 2026 teardown of production failures put it plainly: AI-generated code "is usually built around the happy path: the ideal sequence of actions a user takes when everything goes as expected," and the telling diagnostic question is "what happens when a request fails? (If the answer is 'nothing,' that's a problem.)" (MindStudio, April 2026).

Builders themselves surface the symptom in their own troubleshooting docs. Bolt's documentation notes that "a white screen can mean there's a runtime error that Bolt can't automatically detect" (Bolt support, 2026), and Base44's troubleshooting guide tells users they "might see a blank screen, get stuck on the loading icon, or notice that everything is running slowly or unresponsive" (Base44 docs, 2026). Practitioners describe the same thing in the field: a builder who tests generated apps like a real user reports "empty states, errors, permission loading, states all missing" as a recurring pattern.

Design research has treated these states as first-class for years. Nielsen Norman Group warns designers not to "default to totally empty states," because doing so leaves "users wondering if the system is still loading information or if errors have occurred" (NN/g, 2021), and its error-message guidance stresses that a good error must "help users recognize, diagnose, and recover" rather than merely state that something went wrong (NN/g, 2023). The web platform gives generators the tools to comply: React's error boundaries exist precisely to "display some fallback UI instead of the part that crashed" (React docs, 2026). The question this axis asks is simply whether a builder uses them by default.

How we would score it

State-handling completeness would be a 20-point axis, five sub-criteria worth up to 4 points each. Scores are assigned per generated app, from inspection of both the produced source and the running application, never from the builder's marketing claims.

The rubric is deliberately about the shipped behavior, not about whether the builder can produce these states when explicitly asked. A user who has to know the words "add an empty state and an error boundary" is a user the generator has already failed on this axis.

The reproducible measurement protocol

The point of a BuilderProof axis is that anyone can rerun it and get the same picture. State-handling completeness is measured with the same discipline as our other axes:

1. Fixed prompt. Each builder receives an identical, version-pinned brief from our fixed-prompt suite that asks for a small data-driven app (a list, a detail view, and a create form) and says nothing about loading, empty, or error handling. The omission is intentional: we are testing defaults, not instruction-following.
2. Static inspection. We read the generated source for progress components, empty-state branches, and error boundaries or their framework equivalents.
3. Runtime perturbation. We then exercise the running app under three induced conditions: a throttled network to expose loading behavior, a freshly seeded account to expose empty states, and a forced non-2xx response to expose error behavior.
4. Scoring and evidence. Each sub-criterion is scored against the rubric above, with a screenshot or code excerpt recorded as evidence, so the score is auditable rather than asserted.

Because the same prompt and the same perturbations are applied to every builder, the axis stays reproducible and versioned, and the underlying observations are published as open data. That is the property that lets state-handling completeness sit next to the other axes in a composite rather than being one reviewer's impression.

Documented posture across the five builders (2026)

Ahead of a full scored run, it is worth recording where each builder stands in its own documentation today. The table below is a snapshot of documented posture, not an empirical score: it reflects what each vendor tells builders about generating these states, sourced and year-tagged. Empirical scores require the harness run described above; this is the neutral starting line.

Read this table carefully: it says only what each builder documents, which is a weak proxy for what it ships. A builder can document nothing and still generate excellent error boundaries by default, or document everything and still skip them under a plain prompt. That gap between documented intent and generated reality is exactly what the scored axis is designed to close.

What this axis does not measure

State-handling completeness is narrow on purpose, and it is easy to confuse with axes we already publish. To keep the composite clean, we draw hard boundaries:

• It is not error-message debuggability, which grades the developer-facing error text and diagnostics you get while building. This axis grades the end user's runtime UI when a request fails.
• It is not error-recovery autonomy, which measures whether the builder recovers from its own build failures. This axis measures whether the generated app handles its own runtime failures.
• It is not observability and logging posture, which is about what the running app records for operators. A well-logged app can still crash to a blank screen for the user.
• It overlaps with, but is distinct from, accessibility posture: an error state must also be announced to assistive technology, but a11y scoring covers far more than these three states, and this axis scores their presence and usefulness for all users.

Keeping these lines sharp is what lets each axis carry independent signal. If you want the frame around all of them, see our benchmark methodology.

Open questions for contributors

BuilderProof axes are community-editable, and this one has genuine open design questions before it graduates to a scored run:

• Should loading, empty, and error carry equal weight, or should error states weigh more because their absence is the most damaging (a crash) rather than merely unpolished?
• How should partial failure be scored: an app that handles empty and loading well but crashes on error is not the same as one that skips all three.
• Where is the line between a legitimate default and instruction-following, when some builders let you set project-wide UI conventions?

If you benchmark AI app builders, or you have generated-app examples that handle these states well or badly, the methodology and its rubric are open for revision. Every source behind this proposal is linked inline above, each year-tagged, from the vendors' own documentation and from published design and engineering research.

BuilderProof editorial team. "State-handling completeness: a proposed benchmark axis for AI app builders (August 2026)". BuilderProof, August 2026. https://www.builderproof.org/benchmarks/state-handling-completeness-empty-loading-error-axis-proposal-august-2026.

@misc{builderproof-state-handling-completeness-empty-loading-error-axis-proposal-august-2026,
  title  = {{State-handling completeness: a proposed benchmark axis for AI app builders (August 2026)}},
  author = {{BuilderProof editorial team}},
  year   = {2026},
  month  = {aug},
  howpublished = {\url{https://www.builderproof.org/benchmarks/state-handling-completeness-empty-loading-error-axis-proposal-august-2026}},
  note   = {BuilderProof, builderproof.org}
}