API-Design Consistency of Emitted Routes: A Proposed Axis for Whether an AI Builder's Endpoints Agree With Each Other (August 2026)

Every axis BuilderProof has proposed so far asks a question about one artifact. Does this handler validate its input. Does this route emit a canonical tag. Does this error message name a cause. This axis is different in kind, and that difference is the interesting part. Consistency is not a property you can read off a single endpoint. It only exists between endpoints, and it only becomes visible when you line up everything the builder emitted and look at the variance. That changes what a rubric has to measure, and it changes the scoring device. We are not scoring anyone today. We are publishing a candidate axis, its rubric, its posture levels, and a reproduction protocol, and opening all of it for comment.

Quick Answer

API-design consistency of emitted routes is a proposed BuilderProof benchmark axis, drafted August 19, 2026, that scores whether the HTTP surface an AI app builder emits is internally coherent across all of its endpoints: one predictable URL and verb scheme, correct and uniform status-code semantics, a single reusable error-body shape, consistent collection semantics, and a machine-readable contract a second client could consume. It is scored as variance across the endpoint set, not as a checklist per endpoint, and it separates internal consistency from standards conformance because a surface can be perfectly consistent and standards-divergent at the same time. The cohort under consideration for this axis is the five commercial builders we track across the August axis series: v0 , Lovable , Bolt , Replit , and Base44 . This page is an axis proposal open for community edits, not a leaderboard.

Why we are proposing this axis

The generated app has exactly one consumer at birth: the frontend the same run produced. That single consumer will paper over almost any incoherence, because it was written against whatever the backend happened to do. If one route returns { error: "not found" } with a 200 and the next returns a bare text body with a 500, the generated client handles both, because the same model wrote both sides in the same breath. Nothing is broken. Nothing is measurable from the browser.

The bill arrives with the second consumer. A mobile client. A partner integration. A background job. An agent calling the app through a tool definition. Any of those has to be written against the surface as it actually is, endpoint by endpoint, with no shared assumption to lean on, and that is when incoherence converts into work. The practitioner version of this shows up all over the builder communities as a vaguer complaint about the moment the generated code stops cooperating. One r/nocode thread, "Anyone here move off Lovable / Bolt? Why?", captures the feeling precisely: "The moment I need something slightly custom or want to tweak the generated code, it feels like I'm fighting the tool instead of working with it."

That complaint is usually read as a statement about code quality in general. We think a measurable share of it is specifically about surface coherence, and that it is worth isolating. In 2026 the second consumer is also increasingly not a human. An app whose endpoints do not follow a predictable scheme and do not publish a contract is an app an agent cannot reliably drive, and that is a cost that did not exist when these tools launched.

What "API-design consistency" means here

Four boundaries keep the axis honest and keep it from swallowing its neighbours.

• It measures coherence, not correctness. A surface where every endpoint returns 200 with an ok: false body is wrong, but it is consistently wrong, and it scores well here and badly elsewhere. Splitting those two judgements is deliberate. Collapsing them is how you get a rubric that just re-measures general quality.
• It measures across endpoints, not within one. A single endpoint cannot be inconsistent with itself. The unit of observation is the pairwise relationship between endpoints, which means the axis needs a minimum endpoint count to produce a meaningful score at all.
• It measures the untouched default. Read from the emitted project of the reference build, before any human refactors a route or asks the agent to standardize the error shape.
• It is not an architecture preference. REST, RPC-over-POST, a typed SDK and a database-derived API are all legitimate. The axis asks whether the builder committed to one and applied it uniformly, not whether it picked the one we like.

What this axis explicitly does not measure: whether input is validated, whether errors are debuggable for a human reader, whether the surface is authorized correctly, or how much of it is logged. Those are separate axes we already publish or already propose.

The scoring device has to change

This is the methodological reason the axis is worth writing up rather than just adding to a backlog. Every previous rubric in this series is additive: enumerate signals, check each one, weight, sum. That works when the property lives inside the artifact.

Consistency does not survive that treatment. Consider two generated projects, each with eight endpoints. Project A returns a uniform error object from all eight. Project B returns that same object from four and a bare string from four. A checklist scored per endpoint gives A eight passes and B four, a 50 percent score, which sounds like partial credit. But the consumer experience of B is not half as good as A. It is categorically different, because B has no rule the client can be written against, and a client must now branch on shape at every call site. Partial uniformity is closer to no uniformity than the arithmetic suggests.

So this axis is scored on divergence, not on hits. For each signal, enumerate the distinct implementations present across the endpoint set and score the count, not the ratio. One shape across eight endpoints is full marks. Two shapes is a heavy penalty even at seven to one, because the client still needs the branch. The proposal below encodes that as a dominant-pattern-plus-exceptions score, and the choice to make it superlinear in the number of exceptions is the part we most expect to be argued with.

There is a second consequence. Because a two-endpoint project can only be coherent or not, and an eight-endpoint project has twenty-eight pairs, the score is not comparable across projects of different sizes unless the reference build fixes the endpoint count. That constraint belongs in the harness brief, not in the rubric, and it is one of the reasons we are publishing the rubric before running anything.

Consistency and conformance are orthogonal

The cleanest way to see why these must be two sub-scores rather than one is a real, documented case.

PostgREST , the layer that turns a Postgres schema into an HTTP API and therefore the de facto API layer under the builders that sit on Supabase , returns a documented, uniform error body with the fields code, message, details and hint, and it maps Postgres error codes onto HTTP status codes systematically: a uniqueness violation becomes 409, a foreign-key violation becomes 409, an undefined table becomes 404. It also prefixes its own errors with PGRST so a client can tell a database error from a gateway error. That is about as internally consistent as an error contract gets, and it is machine-readable and stable.

It is also not the standard. RFC 9457, Problem Details for HTTP APIs, published July 2023 on the Standards Track as the successor to RFC 7807, specifies type, status, title, detail and instance under the media type application/problem+json, and its stated rationale is exactly the failure mode this axis measures: HTTP status codes alone "cannot always convey enough information about errors to be helpful," and without a common structure every API is left to "define their own," which non-human consumers cannot act on generically.

So a builder inheriting PostgREST scores near the ceiling on internal consistency and does not score on conformance, because a generic client cannot recognize the payload by media type. A builder that emits three different hand-rolled shapes, one of which happens to be problem+json, scores badly on consistency and partially on conformance. Those are different diagnoses with different remedies, and one number hides both. Any rubric that folds them together will rank two very different surfaces identically.

The proposed rubric

Seven signals, weighted, scored from the emitted project and its running endpoints. Weights are a proposal and are the part we most want argued with.

Two notes on the weights. Error-body uniformity carries the most because it is the signal that most directly determines whether a second client can be written generically, and because it is the one that degrades fastest when routes are authored one at a time. Auth surface uniformity carries the least not because it is unimportant but because it is largely determined by the platform rather than by the generation, and because the auth and access-control posture axis already scores the substance of it.

Each signal is scored on divergence: full marks for one implementation, a steep deduction for the second distinct implementation, and a smaller marginal deduction for each further one, so the curve punishes the introduction of a branch more than the accumulation of many.

The four documented postures

Reading the five vendors' current documentation side by side, the finding is that they are not arranged along one axis from incoherent to coherent. They occupy four structurally different postures, and one of them does not emit a first-party HTTP surface at all. That last case is not a zero. It is a different measurement problem, and a rubric that scores it as a zero is simply wrong.

Posture one, no first-party HTTP surface. The generated client talks to the database through a client library, and authorization lives in row-level security rather than in an application route. Lovable's documentation describes exactly this arrangement: it "wires your app's UI to your data," and the security instruction is at the database layer, that before going live you should "make sure every table has Row Level Security policies that restrict who can read and write each row," with the warning that "Missing RLS policies are the most common way app data gets exposed." There is an API here, but the builder did not design it. It was derived from the schema, and its consistency is a property of the derivation layer rather than of the generation.

Posture two, inherited uniform API plus authored exceptions. The same platforms then add server-side functions for whatever cannot run on the client. Lovable documents that you do not write these yourself: "Describe the behavior in chat, and Lovable writes the function, deploys it to your Supabase project, and updates your app to call it." This is the posture with the most interesting measurement problem, because the app now has two surfaces with different provenance. The derived one is uniform by construction. The authored ones are each a fresh decision, and nothing forces the second function to answer like the first.

Posture three, SDK as the contract. Base44 exposes data through a typed JavaScript client rather than through a documented HTTP surface, with entity methods invoked as base44.entities.Task.create() alongside list, filter and update, instantiated from createClient in @base44/sdk. Consistency at the call-site level is near-total, because the SDK is the interface and it is uniform. But the HTTP layer beneath it is not the published contract, which means the second consumer must either adopt the SDK or reverse-engineer. That is a genuine trade-off, not a failure, and the rubric should express it as a high consistency score with a contract score that depends on how the SDK is distributed and typed.

Posture four, framework-capable and route-by-route authored. v0 generates Next.js projects, where the Route Handler convention supports GET, POST, PUT, PATCH, DELETE, HEAD and OPTIONS, and helpfully notes that if OPTIONS is not defined the framework "will automatically implement OPTIONS and set the appropriate Response Allow header depending on the other methods defined in the Route Handler." That is a free win on part of the method-safety signal. What the framework does not do, anywhere, is prescribe an error-body shape. There is no built-in envelope, no problem+json helper, no convention. Each handler decides.

And there is a detail in that documentation worth stating carefully, because it is the most concrete mechanism we can point at for why generated surfaces drift. In the same reference page, the success examples return Response.json({ ... }), while the webhook example returns its failure as a plain-text body, new Response("Webhook error: ...", { status: 400 }). Both are correct framework usage. Together they model a JSON success and a text failure in one document. We are not claiming this causes anything. We are noting that the canonical reference for the convention a code-generating model is imitating does not itself demonstrate a uniform envelope, and that if generated projects turn out to mix shapes, this is the first place to look rather than the last.

A note on the fifth builder. Replit's Agent documentation states that "Agent writes code, sets up infrastructure, and tests the result" and covers "writing code, setting up infrastructure, configuring databases," but we could not find, in the Agent documentation as of August 2026, any statement of API conventions for the routes it writes: no prescribed error shape, no addressing scheme, no contract emission. We are reporting that absence as an absence, not as a score. It does mean the expected variance between two runs is highest here, which is a testable prediction and one the protocol below is designed to check.

How to reproduce it

The protocol is a static read plus a probe sweep, and it is cheap enough to run against your own build today.

1. Generate the reference application from the standard brief, with no instruction about API design, error handling or conventions in the prompt. The brief must be one that requires at least two resources with full lifecycle operations, so the endpoint set is large enough for pairwise comparison.
2. Enumerate the emitted HTTP surface from the source: every route file, function, or handler. Record path, method, and whether it was platform-derived or authored during the run. If no first-party surface exists, record posture one and enumerate the derived surface separately, tagged as inherited.
3. For each endpoint, statically extract the success response shape and every failure response shape, with field names, and normalize them for comparison.
4. Probe each endpoint for the not-found case with a well-formed request for an absent identifier, and for the conflict case where the schema permits one, recording status and body. Do not send malformed payloads. That is the input-validation axis, and mixing the two contaminates both.
5. Send an unsupported method to every endpoint and record the status and whether an Allow header came back.
6. Send the same PUT and the same DELETE twice and record whether the second call returns the same outcome as the first.
7. Count distinct implementations per signal across the endpoint set, and score divergence per the rubric. Report the raw pattern counts alongside the score, because the counts are the reviewable evidence and the score is a summary.
8. Check for a contract artifact: an OpenAPI document, a generated client, a published schema. Record its existence and whether it was emitted or would have to be written.
9. Repeat on a second independent generation from the same brief, and report the two scores separately before averaging anything. For posture four, the spread between runs may be more informative than either number.

Steps 4 and 5 are the ones most likely to be skipped, and they are the ones that produce the divergence evidence. A surface can look uniform in the happy path and fragment completely in the failure path, which is the normal outcome when routes are written one at a time.

The sole-consumer illusion

Our previous axes each named their characteristic trap. Tests that pass without asserting are the green-check illusion. Validation that exists only in front of a human is the green-form illusion. Inheriting a capable framework and calling it an emitted artifact is the capability illusion. The trap here is the sole-consumer illusion.

It works like this. The generated app has one client, and that client was written by the same process, at the same time, with full knowledge of every quirk. Every incoherence is absorbed on the spot. The app demos perfectly. The operator, reasonably, concludes the backend is fine, because every observable signal says so.

The illusion breaks only on contact with a consumer that was not there at generation time. That consumer cannot be written once against a rule, because there is no rule; it has to be written per endpoint, and every future endpoint added to the surface extends the same tax. This is why the axis is worth having despite being invisible in normal use: it predicts a cost that arrives later, reliably, and always at the point when the app is finally doing something that matters.

The 2026 version of this is sharper. An agent consuming the app through a tool definition is the least forgiving consumer there is, because it needs the contract to reason about the surface at all, and it cannot ask the original generation what it meant.

How this relates to our existing axes

We want to be explicit that this is not a duplicate of adjacent work.

It is not the input-validation and data-integrity posture axis. That axis sends malformed payloads and asks whether the server boundary rejects them at all, scoring rejection semantics as one of six signals inside a single endpoint. This axis sends well-formed requests and asks whether the answers agree with each other across the surface. An app can validate rigorously on every endpoint and still return five different rejection shapes, scoring well there and badly here. The reverse also holds: a surface can be perfectly uniform in shape and validate nothing. The two are close enough that the protocol above explicitly forbids reusing the malformed-payload battery, because a shared probe would make the scores correlate for procedural reasons rather than real ones.

It is not the concurrent-write safety posture axis either, which is worth stating because the two share the word idempotency and almost nothing else. The idempotency sub-criterion here is the protocol one: whether GET is free of side effects and whether PUT and DELETE are repeatable in the HTTP sense. That axis asks a data-layer question instead, whether two overlapping writes to the same row both survive. A surface can be flawlessly RESTful, answer 405 with a correct Allow header, and still lose an update every time two people edit the same record.

It is also not observability, which asks what the platform records about a request after it is served, nor error-message debuggability, which asks whether a failure explains itself to a human reader. Both of those judge the content of a response. This one judges only whether the responses across a surface follow the same rules, which is a question about the set rather than about any member of it.

What we are not claiming yet

No scores today. This page proposes the axis, the rubric, the divergence-scoring device, the postures and the protocol. It does not rank anyone. The postures above are read from vendor and framework documentation current as of August 2026, and documentation is evidence of design intent, not of behaviour on a given run. Whether a specific generated project holds one error shape across eight endpoints is precisely the measurement this proposal exists to enable, and every number in the rubric is a candidate that we would rather see attacked before it produces a ranking.

We are also not claiming the neutral standard is the right target for these tools. It is entirely defensible for a builder to emit a simple uniform envelope of its own rather than problem+json. The conformance sub-score exists to record which choice was made, not to punish the pragmatic one.

Limitations and open questions

• Endpoint count confounds the score. Divergence measured across pairs is not comparable between a four-endpoint and a twelve-endpoint project. The reference brief has to pin the resource count, and even then a builder that emits fewer, fatter endpoints will score better on coherence for a structural reason rather than a design one. We do not have a clean correction for this.
• The superlinear penalty is a judgement. Making the second distinct pattern cost far more than the fifth encodes a specific claim about client cost. A reasonable person could argue the penalty should be linear, and that argument reorders the levels.
• Posture one may not be scorable on this axis at all. If the builder did not design the surface, scoring the surface tells you about the derivation layer, and two builders sitting on the same layer would receive near-identical scores that say nothing about either. The honest options are to score only the authored exceptions, to report inherited and authored sub-scores separately, or to mark the axis not applicable. We currently favour the second and are not confident.
• A typed SDK is a real contract that this rubric under-credits. If the generated client is fully typed and distributed, a second consumer in the same language is arguably better served than by an OpenAPI document. The contract signal as written is biased toward HTTP-level artifacts, and it may need to be split by consumer type.
• Per-run variance is unmeasured and possibly dominant. For agent-authored routes the score may be a property of the run rather than of the builder. Two generations is the minimum in the protocol and is probably not enough to separate those.
• Static extraction of response shapes is imperfect. Shapes assembled conditionally, or thrown from a shared helper and reshaped by a framework boundary, will be misread by a purely static pass. The probe sweep exists to catch this, and the two methods will sometimes disagree. When they do, the probe wins and the disagreement gets reported.

Comments, counter-rubrics and reproduction attempts are welcome, and the most useful thing you can send us is a divergence count from your own generated project that contradicts what the documentation above implies. That is the correction that changes a rubric before it ever produces a score.

BuilderProof Editorial Team. "API-Design Consistency of Emitted Routes: A Proposed Axis for Whether an AI Builder's Endpoints Agree With Each Other (August 2026)". BuilderProof, August 2026. https://www.builderproof.org/benchmarks/api-design-consistency-emitted-routes-axis-proposal-august-2026.

@misc{builderproof-api-design-consistency-emitted-routes-axis-proposal-august-2026,
  title  = {{API-Design Consistency of Emitted Routes: A Proposed Axis for Whether an AI Builder's Endpoints Agree With Each Other (August 2026)}},
  author = {{BuilderProof editorial team}},
  year   = {2026},
  month  = {aug},
  howpublished = {\url{https://www.builderproof.org/benchmarks/api-design-consistency-emitted-routes-axis-proposal-august-2026}},
  note   = {BuilderProof, builderproof.org}
}