Does It Answer the First Visitor? A Proposed Axis for Cold Start and First-Request Latency (September 2026)
Every timing a developer has was taken from a system their own presence kept awake. A pre-registered seven-signal axis for what the first visitor after a quiet period actually experiences.
Updated on September 17, 2026
On this page
Quick answer. Every measurement a developer takes of their own application is taken from a system their own presence has kept awake. The compute instance is warm because they just deployed to it, the database compute is awake because they just queried it, and the hosting project is active because they have been active. A generated application is not one process with one idle clock. It is at least three independently idling layers, each with its own documented threshold, and those thresholds are published in three different documents that never mention each other. This axis proposes a way to score what the first visitor after a quiet period actually experiences. It is a pre-registration, not a result: no builder is scored, named or assigned a posture on this page.
A generated application gets deployed, the URL works, and that is usually the end of the question. The link is shared, somebody opens it, and it loads.
The interesting case is the second visitor, three days later, at seven in the morning, when nobody has touched the application since. Whether it answers them in two hundred milliseconds or sixty seconds, or whether it answers them at all, is decided by configuration nobody chose, in layers nobody enumerated, on timers that are documented individually and composed nowhere.
Why this is not already covered
Four axes in this series sit close enough that the boundary has to be drawn explicitly, in both directions, or the axis is not worth proposing. Three more are near enough to name.
Against speed-to-first-paint
That axis times the builder. Its two clocks run from prompt submission to the first rendered preview frame, and from prompt submission until the brief's acceptance checks pass. Both stop at the moment you have a working application. This axis starts there. It times the thing the builder produced, answering somebody who is not you, at a moment you are not present.
Stated both ways. A builder can paint a preview in eight seconds and pass its acceptance checks faster than any competitor, scoring at the top of that axis, and emit an application whose first request after an idle weekend takes a minute, because generation speed and runtime warmth are properties of different machines owned by different companies. And a builder can be among the slowest to produce a working application while emitting one that answers its first visitor instantly, because it happened to target a runtime that does not idle.
The boundary is who is waiting. There, you are. Here, a stranger is. Full detail is in our speed-to-first-paint protocol, whose own caveat section says the brief is kept small "to isolate startup latency from generation volume", using startup to mean the builder starting up rather than the application.
There is a naming collision here worth flagging, because it will otherwise cause confusion. Our methodology note retiring first-build success introduced the abbreviation TTFB for time-to-first-functional-build, a generation-time metric. The web performance literature uses the same four letters for time to first byte, a delivery-time metric. They are unrelated quantities, and that note says so itself, distinguishing a deploy-time metric from a generation-time one. This axis is about the second sense. We will spell it out in full rather than abbreviate it.
Against deploy quality
That axis runs three audits against the production build: a Lighthouse performance score, axe-core plus a manual accessibility pass, and an SEO checklist. It is the closest neighbour, and the boundary is sharp for a reason that is a finding in its own right.
An audit of that kind navigates to the URL and measures what happens. If the instance was cold, the navigation that starts the audit is the navigation that wakes it. By the time the audit is measuring render-blocking resources and layout shift, the thing this axis is about has already finished happening, and it happened to the auditor rather than to a user. The instrument warms the subject on contact.
Stated both ways. An application can score well on every deploy-quality dimension and still make its first visitor of the day wait a minute, because bundle weight and image sizing are properties of a response that has already begun. And an application can answer instantly from cold while shipping a render-blocking bundle and failing an accessibility sweep, which is that axis failing and this one passing.
Against state-handling completeness
That axis asks whether a loading state exists. This one asks how long it is shown, and, more awkwardly, whose loading state it is.
Stated both ways. An application can render an exemplary skeleton for every pending request, scoring full marks there, and show it for forty seconds because the runtime underneath is starting from nothing. And an application can answer from cold in eighty milliseconds while rendering a blank white screen during every fetch, which is this axis passing and that one failing.
There is a sharper interaction, and it is documented. Some platforms show their own interstitial while a suspended service starts, which means the application's own loading state is not what the visitor sees. A perfect score on that axis can be invisible in exactly the case this axis measures. The boundary is drawn in our state-handling completeness proposal, whose scope note draws seven boundaries against other axes and mentions duration in none of them.
Against database connection and query cost
That axis asks whether ten people can use the application at once. It is about contention: pool sizing, connection limits, query timeouts under simultaneous load. This axis asks about the opposite condition, when nobody has used it at all.
Stated both ways. An application can be correctly pooled and survive a hundred concurrent users, and still have its database compute suspended when the hundred and first arrives after a quiet night. And an application can have a permanently warm database and fall over at four concurrent requests.
They also compose, which is worth saying explicitly. A connection pool created at module scope is a property of a warm instance. When the instance itself has been recycled, the pool is not merely empty, it does not exist yet, and the first request pays for establishing it in addition to whatever the layers below are doing.
Named without a link, to stay inside the internal-link budget
Three further axes touch this territory and are deliberately not linked. HTTP caching and revalidation correctness interacts with this axis in a way that gets its own section below. Cost-to-ship is adjacent, because the configurations that idle most aggressively are the free ones. Observability and logging posture is adjacent in the other direction, because a first request that takes a minute leaves a trace only if something is recording.
Four documented facts that make this measurable
Every figure below is quoted from a primary source and, where it is arithmetic, recomputed rather than repeated.
1. The platform's own cold-start remedy excludes the first request, and says so
Vercel's fluid compute documentation lists "Automatic cold start optimizations" among the model's advantages, and describes the mechanism: "When using Node.js version 20+, Vercel Functions use bytecode caching to reduce cold start times. This stores the compiled bytecode of JavaScript files after their first execution, eliminating the need for recompilation during subsequent cold starts."
The next sentence is the one that matters for this axis. "As a result, the first request isn't cached yet. However, subsequent requests benefit from the cached bytecode, enabling faster initialization."
Read plainly, the documented mitigation for cold starts is, by its own description, not available on the first request. That is not a criticism of the mechanism, which does exactly what it says and helps the case it describes. It is an observation that the remedy and the axis are aimed at different requests, and the request this axis is about is the one explicitly excluded.
The same page carries a second sentence with a consequence nobody usually notices: "Bytecode caching is only applied to production environments, and is not available in development or preview deployments." The environment a developer tests in is documented as having different cold-start behaviour from the environment their users hit. Whatever they measured locally was measured on a different configuration, by design.
The contrasting posture is equally well documented. Cloudflare's description of how Workers work states that the isolate model "eliminates the cold starts of the virtual machine model", that "Isolates are also designed to start very quickly. Instead of creating a virtual machine for each function, an isolate is created within an existing environment", and that "Any given isolate can start around a hundred times faster than a Node process on a container or virtual machine."
Two vendors, two architectures, two honestly stated positions. A generated application inherits one of them without the person who prompted it having chosen either.
2. Three idle clocks, three timescales, spanning a factor of about two thousand
The layers a generated application sits on do not share an idle threshold. They do not share an order of magnitude.
Neon documents the database compute: "When your database is inactive, it automatically scales to zero after 5 minutes." Reactivation is fast and stated: "Once you query the database again, it reactivates automatically within a few hundred milliseconds." The setting is not always yours to change: "For Neon Free plan users, this setting is fixed."
Render documents the compute instance: "Render spins down a Free web service that goes 15 minutes without receiving any inbound traffic." The recovery is stated too, and it is a different order of magnitude: "This process takes about one minute."
Supabase documents the project: "A Free plan project is considered inactive if it does not receive sufficient user database activity over the past week." The threshold is a week, and the consequence is categorically different from a delay, which the next fact covers.
Recomputed, those three published thresholds are five minutes, fifteen minutes, and one week. One week is ten thousand and eighty minutes. The shortest and longest idle thresholds a single generated application can be simultaneously subject to therefore differ by a factor of about two thousand.
The important word is simultaneously. These are not alternatives. An application deployed on one platform, using a database from a second, with a backend project on a third, is subject to all three at once, and each vendor's page is entirely correct about its own layer and silent about the others.
3. One documented spin-up is about thirty-three times the threshold the field calls poor
The web performance literature gives the yardstick. Time to first byte is defined on web.dev as "the time between starting navigating to a page and when the first byte of a response begins to arrive", with published thresholds of 0.8 seconds or less for good, 0.8 to 1.8 seconds for needs improvement, and greater than 1.8 seconds for poor.
The page is also careful about its own status, and the honest framing matters: "Because TTFB isn't a Core Web Vitals metric, it's not absolutely necessary that sites meet the 'good' TTFB threshold". It notes that time to first byte precedes first contentful paint and largest contentful paint, so it is a floor under the metrics that are Core Web Vitals rather than one of them.
Recomputed against the thresholds: Render's documented spin-up of about one minute is 60,000 milliseconds. Against the 1,800 millisecond poor threshold that is a factor of about 33. Against the 800 millisecond good threshold it is a factor of 75.
Neon's reactivation is in a different league and still worth computing. A few hundred milliseconds, taken at 300 to 500, consumes between 37 and 62 percent of the entire 800 millisecond good budget before the application has issued its first query, let alone rendered anything. That is a well-engineered number doing exactly what its vendor claims, and it is still most of the budget.
Neither of these is a defect. Both are documented, deliberate trade-offs, chosen by their vendors and described accurately. The axis is not about whether a vendor made a bad choice. It is about whether the application built on top of them knows which choices it inherited.
4. At the far end the application does not answer slowly, it does not answer at all
The Supabase threshold is different in kind, not just in duration. A paused project is not a slow project. Restoring one is a human action taken in a dashboard: "You can restore a paused project for up to 1 year after it was paused", by selecting the organisation and the paused project and clicking Resume project.
So the idle path has an end state that is not a latency figure. Past a documented threshold, no request wakes anything, because waking requires somebody to log in and click a button. The documentation also draws the line by plan: "Paid projects cannot be paused and are not subject to pausing for inactivity."
This is why the axis cannot be scored purely in milliseconds. A rubric that measures only duration reports the worst case as a number when the worst case is an absence.
What the first visitor experiences, and why no layer measures it
Put the documented facts end to end and the shape of the problem is a composition that none of the four documents describes, because none of them is about more than one layer.
A visitor arrives after a quiet period. Depending on which quiet period has elapsed, they encounter a function instance that must be created, or a database compute that must be resumed, or a project that will not resume without human intervention. The waits do not overlap helpfully. The database cannot begin reactivating until the application code is running and issues a query, so where both are cold the costs are sequential rather than concurrent.
Nobody is at fault in that description. Vercel documents its function model accurately. Neon documents its five minutes and its few hundred milliseconds accurately. Render documents its fifteen minutes and its one minute accurately. Supabase documents its week accurately. Every one of those pages is correct, well written, and about exactly one layer.
This is the third time this series has found a property of a generated application that is decided at three or more layers where each layer documents itself correctly and none documents the composition. It is becoming the most reliable place to look for an axis.
The proposed rubric
Seven signals, weighted to 100. This is a proposal. The weights are the part most worth arguing about before anything is scored.
Scroll to see more
| Signal | Weight | What a failing case looks like |
|---|---|---|
| First-request latency after a real idle period is measured at all, rather than inferred from warm requests | 22 | Every timing anybody has is from a request that followed another request, so the only path that has never been timed is the one a new visitor takes |
| Every independently idling layer is enumerated, with its own threshold written down | 20 | The compute platform's idle behaviour is known, the database's is not, and the project-level pause has not been read by anyone on the team |
| The idle thresholds are the ones the application chose, not the ones it inherited | 16 | The thresholds are whatever the free tier sets, nobody has read them, and on at least one layer the plan does not permit changing them |
| Something other than a human exercises the cold path on a schedule | 14 | The only traffic that ever reaches the application is a person deciding to look at it, so the cold path is exercised only by the visitors it fails |
| The wait the visitor sees is the application's own, not a platform interstitial | 11 | The hosting platform renders its own starting page, so the loading state the application ships is never shown and the wait is unbranded and unexplained |
| The degraded state is bounded and recoverable without human action | 10 | Past some documented idle threshold the application stops responding entirely until somebody logs into a dashboard and clicks a button |
| The first-request cost is disclosed to whoever operates the application | 7 | The person who will be asked why the link was slow has never been told that a cold path exists or what it costs |
Three weighting decisions worth arguing about now
Why measurement outranks improvement. The heaviest signal is not the speed of the cold path but whether anybody has ever timed it. That ordering is deliberate. An application with a nine second cold start that is measured, known and written down is in a better position than one with an unmeasured cold start that happens to be fast today, because the second one has no way to notice when it stops being fast. It is also the signal most likely to be failed by an otherwise careful team, for the reason the named trap below sets out.
Why enumeration is weighted so close to measurement. Timing the cold path once tells you about the layers that were cold at that moment. If the database had been queried four minutes earlier it was still awake, and the number you recorded is not the worst case. Without an enumeration of the layers you cannot tell which measurement you took.
Why the platform interstitial signal exists at all, at a modest weight. It is not a performance property, and eleven points reflects that. It earns its place because it is the one part of this axis that is visible to the visitor as an experience rather than as a duration, and because it interacts with an axis we have already proposed in a way that makes a good score there invisible.
Four postures
Level 0, Always cold. No layer's idle policy has been read. The first visitor after any quiet period is the first party to discover what happens. The cold path has never been timed, because timing it would have required waiting long enough for it to exist.
Level 1, Warmed by accident. The application receives enough traffic that the idle thresholds rarely fire. The cold path is real and unmeasured, but it is seldom taken.
Level 2, Enumerated. Every idling layer is identified, its threshold recorded, and the composed worst case written down. The number may be bad. It is known, and it is known which layer contributes what.
Level 3, Bounded. The composed first-request cost is bounded by design, the degraded state recovers without human intervention, and something other than a visitor exercises the cold path on a schedule, so a regression is discovered by the schedule rather than by a user.
The house argument about the bottom of the ladder applies here in an unusual form, and it is worth stating because it cuts against the natural reading. Level 1 is not progress toward Level 2. It is a different kind of thing. Levels 0, 2 and 3 are properties of the application. Level 1 is a property of its traffic. It is conferred by having visitors, which means it is strongest exactly when an application needs it least and absent exactly when an application is most fragile, in the weeks after launch when almost nobody is arriving. It can also regress to Level 0 without a single line of code changing, purely because a quiet month went by. An application that moves from 0 to 1 has not made a decision; it has acquired an audience.
The measurement protocol
Ten steps. The protocol is the part we would run first, and steps 3 and 5 are the ones that make the rest meaningful.
- Deploy the generated application unchanged, on whatever the builder's default target is. Record the compute platform, the database provider and any project-level host as three separate facts.
- For each of those three, locate and record the vendor's documented idle threshold and its documented recovery behaviour. Record it as a quotation with a URL and a date read, not as a remembered number.
- The looks-correct control. Before any idle period, request the application twice in quick succession and record both timings. These are the numbers a developer would otherwise report. Keep them, labelled as warm, and treat any later disagreement with them as the finding rather than as noise.
- Leave the application entirely untouched for a period longer than the shortest documented threshold plus a margin. Touch nothing: no dashboard, no deploy, no database console. Each of those resets a clock.
- Request once and record the full time to first byte. One request. The second request is a different measurement, and taking it first destroys the only observation that matters.
- Repeat from step 4 with a period longer than the longest documented sub-day threshold, to separate the layers. A wait that exceeds the compute threshold but not the database threshold isolates one contribution.
- Record what the visitor saw during the wait: the application's own loading state, a platform interstitial, a blank page, or an error. This is an observation about rendering, not timing, and it is made with a browser rather than a command-line client.
- Where a documented threshold is measured in days, do not simulate it. Record the documented behaviour and the plan it applies to, and mark the signal as documented rather than measured. An axis should not pretend to have waited a week.
- Repeat the cold measurement at least three times on different days. A single cold timing is one draw from a distribution that includes provider load, region and time of day, and this axis is more exposed to that variance than most.
- Report the warm timings, the cold timings and the documented thresholds side by side. The gap between the first two is the finding, and the third is what tells you whether you measured the worst case or a convenient one.
The named trap: the never-idle illusion
Call it the never-idle illusion. Every observation available to the person building the application is generated by a system that has not been idle, because they are the reason it has not been idle.
This is the ninth trap in this series that shares the one-actor structure, and it may be the purest instance of it, because here the act of observing does not merely fail to reveal the defect. It actively prevents the condition the defect requires.
The arithmetic is worth doing explicitly. The shortest documented threshold among the layers above is five minutes. A developer iterating on an application touches it far more often than once every five minutes: a save, a refresh, a query in a console, a deploy. Every one of those resets a clock. Over a working session, not one of the three thresholds can fire. The cold path is not merely untested; it is unreachable from inside the workflow that would test it.
Two further properties make it worse than the average member of the family.
Level 0 and Level 3 are indistinguishable under the only test that gets run. An application with no idle policy at all and one with a rigorously bounded cold path return the same timing to a developer who has just used them. The measurement that separates them requires doing nothing for fifteen minutes, which does not feel like testing.
The environment where the check would happen is documented as behaving differently. By Vercel's own statement, bytecode caching is not available in development or preview deployments. So even a developer who thinks to test the cold path in a preview environment is testing a configuration the vendor has told them is not the production one.
The visitor who finds the defect is, by construction, the person nobody was watching: the one who arrived after everyone else had stopped looking.
Two documented tensions worth stating
Tension one, and it involves one of our own axes. Our HTTP caching and revalidation proposal rewards responses that are cacheable and correctly scoped. A cached response served from an edge never reaches the origin, so a well-cached application appears to have no cold-start problem at all.
The difficulty is not that caching masks the problem. It is that the masking fails precisely when the problem is real. A cache entry expires after a period of no requests. A compute instance spins down after a period of no requests. These are the same period. The moment a cache is coldest is the moment the origin behind it is also coldest, so the mitigation is weakest exactly where it is needed and strongest exactly where it is not.
The resolution is scope rather than choice. Both axes are right, and this one must measure the first request on a deliberate cache miss, or it is measuring the cache.
Tension two, and it is about whose experience is being scored. Render's documentation states that it "displays a loading page to connecting browsers while a service is spinning up." That is a good platform behaviour: a visitor sees something rather than a hung tab.
It also means the application's own loading state, which our state-handling axis scores and rewards, is not what the visitor sees in this case. An application can score full marks for its skeleton screens and have them never render during the one wait long enough to need them. The resolution is classification rather than choice: the two axes are asking different questions, and this one records whose wait it was as a separate observation from how long it lasted.
The seam between the documents
Four primary sources, and the resolution is in none of them.
The web performance literature says what good looks like, in milliseconds, and is explicit that time to first byte is a floor under other metrics rather than a metric that ranks sites. The compute platforms say what their own start-up behaviour is and, in Vercel's case, exactly which request their mitigation does not help. The database says what its own idle threshold and recovery are. The project host says what happens after a week and that recovery is manual.
Each is accurate. Each is scoped to one layer. The quantity the visitor experiences is the composition, and the composition is nobody's documented responsibility, which is what makes this an axis rather than a bug report.
What this axis is not, and what we are not claiming
This is a pre-registration. No builder is scored here, no result set is published, and no product is assigned a posture. The rubric, the postures and the protocol are proposals, published before measurement rather than after, so that the method can be argued with before any number exists to defend.
We are not claiming that cold starts are a defect. Every behaviour quoted above is a documented, deliberate engineering trade-off, and the platforms that idle aggressively do so to make an application free or nearly free to host, which is a real benefit to the person who prompted it. Scale to zero is a feature, and it is sold as one.
We are not claiming any builder is worse than any other on this axis. We have not measured it. The point of publishing the method first is that the method should be criticised before it is used.
We are not claiming the thresholds quoted here are universal. They are the documented values of particular plans on particular platforms, read on the date recorded in the sources below, and both the numbers and the plans they apply to change.
We are also not claiming this is measurable to the millisecond. It is not. Cold-path timings vary with provider load, region and time of day more than most things this lab tracks, which is why the protocol asks for repeated measurements on different days and why the heaviest signal rewards knowing the number rather than achieving a particular one.
Open questions for anyone who wants to argue with this before it is scored. Should the composed worst case be the headline figure, or should each layer be reported separately, as our deploy-quality axis reports its three dimensions rather than blending them? Is fourteen points too much for the scheduled-probe signal, given that a scheduled probe also prevents the condition it is meant to detect and therefore has to be designed not to keep the application warm? And where a platform's idle behaviour cannot be changed on the plan the builder defaults to, is that the application's failure or the plan's, and should the rubric say so?
References
- Vercel, Fluid compute. The cold start optimisations list, the bytecode caching mechanism, the statement that the first request is not cached yet, and the exclusion of development and preview deployments. Read 17 September 2026. https://vercel.com/docs/fluid-compute
- Cloudflare, How Workers works, Workers reference documentation. The elimination of virtual-machine cold starts, isolate start-up speed, and the hundred-times comparison against a Node process. Read 17 September 2026. https://developers.cloudflare.com/workers/reference/how-workers-works/
- Neon, Scale to Zero. The five minute inactivity threshold, the few-hundred-millisecond reactivation, the 16 CU size limit, and the fixed setting on the Free plan. Read 17 September 2026. https://neon.com/docs/introduction/scale-to-zero
- Render, Free Instance Types. The fifteen minute spin-down threshold, the approximately one minute spin-up, and the platform loading page shown to connecting browsers. Read 17 September 2026. https://render.com/docs/free
- Supabase, Project Pausing. The one week inactivity definition for Free plan projects, the one year restore window, the manual Resume project action, and the statement that paid projects cannot be paused. Read 17 September 2026. https://supabase.com/docs/guides/platform/free-project-pausing
- Google Chrome team, Time to First Byte (TTFB), web.dev. The definition, the 0.8 and 1.8 second thresholds, and the statement that time to first byte is not itself a Core Web Vital. Read 17 September 2026. https://web.dev/articles/ttfb
- BuilderProof, Speed-to-first-paint across AI app builders (June 2026). The two-clock generation-time protocol, and the caveat isolating startup latency from generation volume. https://www.builderproof.org/benchmarks/speed-to-first-paint-ai-app-builders-june-2026
Written by
BuilderProof editorial teamCite this benchmark
BuilderProof editorial team. "Does It Answer the First Visitor? A Proposed Axis for Cold Start and First-Request Latency (September 2026)". BuilderProof, September 2026. https://www.builderproof.org/benchmarks/does-it-answer-the-first-visitor-cold-start-axis-september-2026.
@misc{builderproof-does-it-answer-the-first-visitor-cold-start-axis-september-2026,
title = {{Does It Answer the First Visitor? A Proposed Axis for Cold Start and First-Request Latency (September 2026)}},
author = {{BuilderProof editorial team}},
year = {2026},
month = {sep},
howpublished = {\url{https://www.builderproof.org/benchmarks/does-it-answer-the-first-visitor-cold-start-axis-september-2026}},
note = {BuilderProof, builderproof.org}
}Frequently asked questions
Is this not the same as the speed of the AI app builder?
No, and the two can rank a field in opposite directions. The speed axis times the builder: wall clock from prompt submission to a rendered preview and to a working app. Both clocks stop at the moment you have an application. This axis starts there and times the application that was produced, answering somebody who is not you, at a moment you are not present. A builder can generate in eight seconds and emit an app whose first request after an idle weekend takes a minute, because generation speed and runtime warmth are properties of different machines owned by different companies.
Why would a rubric weight measuring the cold path above making it fast?
Because an application with a nine second cold start that is measured, known and written down is in a better position than one with an unmeasured cold start that happens to be fast today, since the second has no way to notice when it stops being fast. It is also the signal an otherwise careful team is most likely to fail, because the shortest documented idle threshold among the common layers is five minutes and a developer touches their application far more often than that. Every save, refresh, console query and deploy resets a clock, so the cold path is not merely untested, it is unreachable from inside the workflow that would test it.
Are cold starts a defect that vendors should fix?
No, and this axis does not claim they are. Every behaviour it quotes is a documented, deliberate engineering trade off. Scaling to zero is a feature, sold as one, and it is a large part of why a generated application can be free or nearly free to host. Two vendors quoted here take opposite architectural positions and both describe theirs accurately. The axis is not about whether a vendor chose well. It is about whether the application built on top knows which choices it inherited, since the person who prompted it chose none of them.
Why can this axis not simply be scored in milliseconds?
Because the far end of the idle path is not a latency figure. One documented project level threshold is a week of inactivity, after which the project is paused, and restoring it is a human action taken in a dashboard rather than something a request can trigger. A rubric that measured only duration would report the worst case as a number when the worst case is an absence. That is why the rubric scores whether the degraded state is bounded and recoverable without human action as a signal in its own right.
Related benchmarks
Does the Whole Name Get Saved? A Proposed Axis for Length and Truncation Correctness (September 2026)
Every length limit is checked with a value the developer typed themselves, and on that value all four units agree. A pre-registered seven-signal axis for whether the value submitted is the value stored.
Speed-to-first-paint across AI app builders (June 2026)
The BuilderProof speed protocol uses two stopwatches rather than one number: speed-to-first-paint (prompt to first rendered preview frame) and time-to-working-app (prompt to all acceptance checks passing with zero manual edits), across five cold runs on a fixed network profile. The June 2026 timing table was placeholder data and was withdrawn on August 21, 2026.
State-handling completeness: a proposed benchmark axis for AI app builders (August 2026)
State-handling completeness is a proposed BuilderProof benchmark axis (August 2026) that scores how well an AI app builder generates the non-ideal runtime states of the apps it produces: loading, empty, and error states. It is a 20-point axis across five sub-criteria, measured reproducibly by giving all five commercial builders (v0, Lovable, Replit, Base44, Bolt.new) an identical fixed prompt and then inspecting the generated app under a throttled network, an empty account, and a forced request failure.