Realtime Subscription Correctness: A Proposed Axis for Whether a Generated Live View Ever Notices It Stopped Being Live (September 2026)

An AI app builder will generate a live dashboard, a shared task board, a chat panel and an order feed, and every one of them will update in front of you the moment you change something in a second tab. The question this axis proposes to measure is what those views do during the ordinary seconds when the connection is not there: when the laptop lid closes, when a phone switches from wifi to cellular, when a tab sits in the background for a minute. We are not scoring builders today. We are publishing a candidate axis, its rubric, its posture levels and a reproduction protocol, and opening all of it for comment before it enters the composite.

Quick Answer

Realtime subscription correctness is a proposed BuilderProof benchmark axis, drafted September 4, 2026, that scores whether a generated application's live views converge back to the true server state after the connection carrying their updates is interrupted. It is measured from the deployed artifact under induced disconnection, not from the preview. The rubric weights seven signals: whether the client refetches to close the gap after reconnecting, whether connection state is visible to the user at all, whether the initial fetch and the subscription are ordered so no change falls between them, whether the subscription is scoped to what the viewer may read, whether the code's assumptions about delivery match what the transport documents, whether subscriptions are torn down on unmount, and how incoming events interact with local edits. The failure mode it targets is not a crash and not a slow page. It is a view that keeps rendering confidently, with no spinner, no empty state and no error, while the data behind it has been frozen for several minutes. The primitives underneath, from the database notification layer upward, are documented as delivering to whoever is connected at that instant, with no backlog for anyone who was not. This page is an axis proposal open for community edits, not a leaderboard.

BuilderProof is an independent, community-editable benchmark for AI app builders. It accepts no vendor payment or sponsorship. This post proposes a new axis, defines how it would be scored, and cites primary sources read on September 4, 2026. It does not score any builder on this axis yet, and it does not name a winner.

Why realtime correctness deserves its own axis

Every axis we have proposed asks a question about a request. Does the response say who may cache it. Does the list return every row once. Do two overlapping writes both survive. All of those presuppose that somebody asked for something.

A live view asks for nothing. It is opened once, and after that it waits. Everything it shows from the second minute onward arrives unrequested, over a connection the user never thinks about and the developer stops thinking about the moment the demo works. That inverts the usual relationship between a defect and its evidence: when a request fails you get a status code, an exception, a rejected promise, something. When a subscription dies you get silence, and silence is exactly what a working subscription also produces when nothing has happened.

That is the structural argument for a separate axis. The defect has no error to catch, no status code to check and no failing assertion, because from the client's position an empty channel and a dead channel are the same observation.

The problem is not obscure and it is not vendor-specific, but the clearest public account of it happens to be on a vendor's own discussion board. In a thread on the Supabase organisation still ranking first for the question, a Supabase collaborator wrote that "just doing the standard examples of starting a subscription and letting realtime handle re-connections in the background will cause loss of data changes," because the client "attempts to reconnect and resume BUT loses any updates on server during this time." The operative sentence for a benchmark is the next one: "Basic realtime examples in Supabase docs will all look like they are running normally when tab returns" (Supabase discussion 5641, February 2022). The same author reports testing across Windows, Android and iOS and finding subscriptions lost in every case tested except one when the tab was backgrounded or the device slept.

Note what that describes. Not an outage. A tab in the background, and a phone going to sleep, which is the ordinary lifecycle of a mobile session rather than an edge case.

What "realtime subscription correctness" means here

The axis scores the deployed artifact and its source, not the vendor's marketing and not the chat experience. It asks, of the untouched build:

When a change is committed on the server while a client's connection is interrupted, does that client end up displaying the change, and if it cannot, does it say so?

Four boundaries keep this honest.

• It measures convergence, not latency. An application that polls every thirty seconds and always converges scores well here. One that pushes in twelve milliseconds and never recovers from a dropped socket scores badly. Delivery speed is a performance question and belongs to a performance axis.
• It measures the emitted artifact. Reconnection handled entirely inside a client library is recorded as inherited rather than authored, in the same way our other axes separate what the builder wrote from what the framework supplied. The distinction matters more here than elsewhere, because the library's reconnect is real and the gap it leaves is the defect.
• It measures behaviour under interruption. A subscription that is never interrupted cannot be scored. The protocol below requires induced disconnection, which is the step that makes this axis cost more to run than a header sweep.
• It is not an architecture preference. Polling, server-sent events, websockets and no live updating at all are legitimate postures. A static view that never claims to be live cannot fail this axis. The axis asks whether the application's recovery behaviour matches the freshness it presents.

The finding: every layer documents delivery to whoever is connected, and nothing documents delivery to whoever was not

This is the part we think is genuinely unmeasured, and it comes from reading transport documentation against framework documentation rather than from any new experiment. Read in sequence, three independent layers say the same thing in three different vocabularies, and the fourth layer, the generated application, is the only one in a position to do anything about it.

The database layer signals to current listeners only

Most generated realtime ultimately rests on PostgreSQL's notification mechanism. Its documentation is unusually direct about the delivery model. The LISTEN reference states that when a notification is issued, "all the sessions currently listening on that notification channel are notified, and each will in turn notify its connected client application," and that "a session's listen registrations are automatically cleared when the session ends."

Read that precisely. The set of recipients is the set of sessions listening at the instant of the signal. A session that is not listening is not a recipient, there is no queue held on its behalf, and when it comes back it is a new session with no registrations and no history. The mechanism is not lossy by accident. It is specified as delivery to the currently connected, and nothing about it is designed to fill a gap.

The NOTIFY reference adds three further properties that matter to anyone writing code against it. Notifications are transactional: "if a NOTIFY is executed inside a transaction, the notify events are not delivered until and unless the transaction is committed," and "notification events are only delivered between transactions," which is why the documentation advises that "applications using NOTIFY for real-time signaling should try to keep their transactions short." Identical signals collapse: "if the same channel name is signaled multiple times with identical payload strings within the same transaction, only one instance of the notification event is delivered to listeners." And the backlog that does exist is shared and finite: "there is a queue that holds notifications that have been sent but not yet processed by all listening sessions," and "if this queue becomes full, transactions calling NOTIFY will fail at commit."

That last one is worth pausing on, because it inverts the expected failure direction. The overflow does not silently drop a notification. It fails the write.

The browser layer specifies the client half of recovery and stops there

The one transport in this space with a standardised gap-recovery primitive is server-sent events, and reading its specification closely is what turns this axis from a hunch into a measurement.

The HTML Living Standard, in its server-sent events section, gives every EventSource object "a reconnection time, in milliseconds" and "a last event ID string." When a connection drops, the user agent is required to reestablish it, and on doing so, "if the EventSource object's last event ID string is not the empty string," it sets the Last-Event-ID header on the new request. The specification defines that header as one that "reports an EventSource object's last event ID string to the server when the user agent is to reestablish the connection." The server can tune the delay by sending a retry field, which sets "the event stream's reconnection time."

So the browser reconnects on its own, and it hands the server a token saying exactly where the client left off. That is a genuinely well-designed recovery primitive, and it is free.

Now read what the specification does not say. It places no obligation on the server to do anything with that header. It defines the client half completely and leaves the half that actually closes the gap entirely to application code. A generated application that emits a stream and ignores Last-Event-ID is fully conformant, reconnects perfectly, and resumes from the present moment rather than from where the client stopped. The platform did its part, visibly and correctly, and the gap survives.

The same section is candid that the connection will in fact be interrupted in ordinary conditions. Its authoring notes warn that "legacy proxy servers are known to, in certain cases, drop HTTP connections after a short timeout," and advise sending a comment line "every 15 seconds or so" to keep the connection open. Interruption is treated by the specification as routine, not exceptional.

The platform layer offers recovery, opt-in and bounded

At the layer most generated applications actually touch, the picture is the same shape, with the recovery mechanisms present but off by default and narrower than they first appear.

The Supabase Broadcast documentation describes an acknowledgement setting, and is explicit about the default: "if the ack config is not set to true when creating the channel, the promise returned by channelD.send will resolve immediately." A send that resolves immediately has confirmed nothing. It is documented, deliberate and reasonable, and it means the default posture of the sending side is fire and forget.

The same page documents Broadcast Replay, which is the closest thing to a backlog in this stack, and every clause of its scope is a constraint worth recording. It "enables private channels to access messages that were sent earlier," but "only messages published via Broadcast From the Database are available for replay." It requires a since timestamp, caps limit at 25 messages, and stores messages in daily partitions such that "a message stays available for at least 72 hours and at most 4 days, depending on the time of day it was sent." It is available "only in the Supabase JavaScript client version 2.74.0 and later."

That is a real and useful mechanism. It is also opt-in, private-channel-only, restricted to one publication path, capped at 25 messages, and dependent on a client version floor. A generated application acquires none of it by default.

One more property from the Postgres Changes documentation belongs in the rubric rather than in this section, and it is a scope question rather than a delivery one. That page states that "RLS policies are not applied to DELETE statements, because there is no way for Postgres to verify that a user has access to a deleted record." Row-level security is the mechanism most generated applications rely on to keep one tenant's rows away from another, and on this event type the documentation says plainly that it does not apply. The same page notes that receiving previous values on updates and deletes requires setting the table's replica identity to full, which widens what a delete event can carry.

The same documentation is also clear about the cost model: "Postgres Changes authorizes every event against each subscriber," so "throughput scales with the number of subscribers, not the write rate," and "changes are also processed on a single thread to preserve their order." Ordering is preserved by construction, which is a genuine guarantee and one the rubric should credit rather than test for.

The honest counterweight

We would be overstating this if we stopped there. None of the above is a defect in PostgreSQL, in the HTML standard, or in Supabase. Each layer documents its behaviour accurately, and each provides the primitive the layer above needs: a signal, a resumption token, a replay window, an acknowledgement flag.

The claim is narrower and it is about the top layer only. Every one of those recovery primitives has to be picked up by application code, and the application code in question was written by a generator optimising for a demo in which the connection never drops. Nothing in that loop rewards closing a gap that the loop cannot produce.

The proposed rubric

Seven signals, weighted to 100, scored from the deployed artifact's behaviour under induced disconnection and from the emitted source. Weights are a proposal and are the part we most want argued with. The third column describes what a failing case looks like, because a failing case is easier to check against your own build than an abstract ideal.

Three weighting decisions are worth arguing about now rather than after the axis ships.

Why gap recovery carries the most. It is the only signal whose failure produces a confidently wrong screen with no recovery path short of a manual reload. Every other row costs correctness at the margin or costs privacy in a narrow case. This one means the primary artifact of the application, the live view, is silently false for as long as the user leaves it open.

Why connection-state visibility outranks scope. This is the row we expect to be argued with hardest, because scope is a privacy signal and visibility is only an interface signal. We rank it higher because visibility is what converts every other failure on this list from silent to reportable. An application that shows a stale-data banner when its channel drops has a bounded defect that a user can act on. One that does not has an unbounded defect that nobody will ever file a bug for.

Why local-edit interaction is only 7. It is genuinely annoying and genuinely common, and it is also self-revealing. A user whose text box is overwritten mid-sentence notices immediately, reports it, and it gets fixed. The rubric weights silent failures above loud ones throughout, and this is the loudest row on the list.

The four postures

Scores roll up into a posture level so the axis reads at a glance without collapsing the detail.

Level 0, Optimistic. A subscription is opened on mount and its callback mutates local state. There is no error handler on the subscription, no teardown, and no notion of the connection having a status. The view is live exactly as long as nothing goes wrong.

Level 1, Library-reconnecting. The client library's automatic reconnection is doing real work, and the connection does come back. Nothing refetches, so the view resumes from the present and the gap persists silently. This is where we expect a well-formed default build to land, because it is what the standard examples produce, and it is the posture the Supabase collaborator quoted above was describing.

Level 2, Gap-aware. The application treats reconnection as an event that requires action. It refetches the affected data on resubscribe, and it surfaces connection state to the user so that a period without updates is distinguishable from a period without changes. The gap is closed, even if bluntly.

Level 3, Reconciled. Everything in Level 2, plus the refetch is bounded rather than total, using a cursor, watermark or resumption token so that recovery cost does not scale with the size of the collection. Event application is idempotent, so a repeated delivery is harmless. Subscriptions are scoped to the viewer's read permissions and torn down on unmount.

We have no prediction to publish about where the cohort sits, because we have not measured it, and an axis proposal that pre-announced its result would be a poor proposal.

How to reproduce it

The protocol requires a deploy and an induced disconnection. It is more expensive to run than a static read, and that cost is the reason this property is unmeasured rather than any subtlety in what it asks.

1. Build the reference application from the fixed prompt. The brief describes a shared list that several people can edit, viewed live. It says nothing about reconnection, offline behaviour, subscriptions or reliability. Asking for the safeguard would measure prompt compliance, not default posture.
2. Deploy it. Reconnection behaviour is a property of a real network path. A local development server on the same machine does not drop connections in the way this axis is about.
3. Open the view in two independent clients, one designated the observer and one the writer, authenticated as different users where the application supports it.
4. Establish the baseline. Write from the writer, confirm the change appears on the observer, and record the latency. This step exists to prove the subscription works at all, so that a later absence is attributable to the interruption rather than to a broken build.
5. Interrupt the observer for sixty seconds. Use three separate interruption modes and record each, because they are not equivalent: sever the network at the interface, background the tab, and let the device sleep. The Supabase report cited above found the last two sufficient on their own.
6. Write from the writer during the interruption. Make at least three changes of different kinds: an insert, an update to a row already on screen, and a delete of a row already on screen.
7. Restore the observer and wait sixty seconds. Do not reload. Record what the view shows: which of the three changes are present, which are absent, and whether anything at all indicates the view was ever disconnected.
8. Reload the observer. Whatever appears only after this step is the measured gap.
9. Read the subscribe path in the source. Record whether an error or status handler exists, whether a refetch is wired to resubscription, whether teardown exists, and whether the subscription's scope matches the viewer's read scope.
10. Repeat on a second independent generation from the same brief and report both results before averaging, because per-run variance on a property nobody prompted for is itself a finding.

Step 7 is the measurement. Step 8 is the control that proves the data was there to be shown. A tester who reloads out of habit before recording step 7 has destroyed the only evidence the axis collects, which is our best guess at why this property has gone unbenchmarked.

The named trap: the quiet-channel illusion

Each axis in this series names the illusion that hides its defect. Tests that pass without asserting are the green-check illusion. A surface whose only consumer wrote itself is the sole-consumer illusion. A short page mistaken for the end of a collection is the first-page illusion. A cache that is private only because you have never met a shared one is the private-cache illusion.

The trap here is the quiet-channel illusion, and it is the simplest of the set: a channel delivering nothing is indistinguishable from a channel over which nothing has happened.

Both produce an identical observation, which is no callback firing. The interface renders the same pixels either way. There is no loading state, because nothing is loading. No empty state, because the list is populated. No error state, because nothing threw. Every affordance an application has for telling a user that something is wrong is bypassed, not because the developer forgot them, but because from the client's position nothing is wrong: it is displaying the last thing it was told, correctly.

This is why the defect survives ordinary testing so reliably. During development the connection does not drop for long enough to matter, the tab is in the foreground because you are looking at it, and the machine does not sleep because you are using it. Every condition the failure requires is a condition your development loop structurally excludes, and the only signal it would produce is silence, which your working system also produces.

How this relates to our existing axes

Two neighbouring axes touch this territory closely enough that the boundaries have to be stated.

It is not our state-handling completeness axis, and this is the sharpest boundary in the set because the two are most easily confused. That axis asks what a view shows while a request is in flight, when a query legitimately returns nothing, and when a request fails. All three of those are states of a request. A frozen live view is in none of them. It made its request, the request succeeded, the data arrived, and the view is rendering it. The two scores move in opposite directions cleanly: a build can score full marks there, with skeletons on every async view, purposeful empty states and error boundaries throughout, and still sit at Level 0 here, because none of those three states is ever entered by a subscription that has quietly stopped. The reverse also holds. A build can converge perfectly after every disconnection and still render a blank region on an empty result.

It is not our HTTP caching and revalidation correctness axis, although both concern a reader seeing something other than the current truth. The cleanest way to see the split is that they fail on opposite sides of a request. That axis has a stored response and asks whether it may still be reused, which is why its post-write invalidation signal is about discarding a cache entry that a write has superseded. This axis has no request, no response and no cache entry: the client is holding an open connection and the question is whether an event reaches it. The scores move independently. A build can invalidate perfectly on every write, so any reader who asks again gets the new value, and still show a stale dashboard indefinitely to a reader who is not asking again because they believe they are subscribed.

It is also not our concurrent-write safety axis, which asks whether two overlapping writers both keep their effect. That axis is about writer against writer, and its concern is the write that disappears. This axis assumes the write landed perfectly and asks whether a reader who was not connected at that moment ever finds out. And it is not our observability and logging posture axis, which covers what a running application records for its operators. A dead subscription that is faithfully logged server-side is still a defect on this axis, because the person looking at the frozen screen is not reading the logs.

The frame around all of these, including how an axis enters the composite, is documented in our benchmark methodology.

What we are not claiming

We are not claiming that any builder in the cohort ships broken realtime. We have not scored this axis. The proposal exists precisely so the rubric can be attacked before any number is attached to a vendor name.

We are not claiming that any of the platforms cited is at fault. PostgreSQL, the HTML standard and Supabase all document their delivery model accurately, and in the case of server-sent events and Broadcast Replay they supply a recovery primitive the application can use. The gap we are proposing to measure is in the application code that does or does not pick those primitives up.

We are not claiming this defect is universal. We are claiming it is silent by construction, structurally unreachable during development, and unmeasured by any benchmark we could find, which is a different and more modest statement.

We are not claiming every application needs Level 3. A single-user notes app with a live view has almost nothing at stake in a sixty-second gap. What the axis measures is whether the recovery posture matches the freshness the interface is presenting.

Limitations and open questions

Induced disconnection is not one thing. Severing an interface, backgrounding a tab and sleeping a device produce different behaviour, and a rubric that scores only one of them measures that one. Our protocol requires all three and reports them separately, which raises the cost of a run and may prove too heavy in practice.

Inherited recovery is hard to attribute. A builder whose default client library reconnects aggressively will look better than one whose library does not, for reasons that have nothing to do with generated code quality. The posture levels attempt the separation by treating library reconnection as Level 1 rather than as credit toward Level 2, and we would like the argument against that split.

The sixty-second window is arbitrary. It is long enough to exceed typical reconnect backoff and short enough to run repeatedly. It is not derived from anything, and a defensible threshold would be better than a convenient one.

We have not measured the cohort. Everything above is read from specifications, vendor documentation and one practitioner report. Documentation is evidence of design intent, not of behaviour on a given run, and the gap between them is exactly what the protocol exists to measure.

The practitioner report we lean on is dated. The Supabase discussion quoted above was opened in February 2022 and describes client behaviour at that time. We cite it because it remains the top-ranked public answer to the question and because it describes the failure shape precisely, not as evidence about any current client version. Anyone with a recent measurement that contradicts it should send it to us, and we will publish the correction.

The weights are a first draft. In particular, 14 points for subscription authorization scope may be too low for a signal whose failure is a privacy failure rather than a correctness one, and our own caching axis weights its privacy row highest for exactly that reason. The counterargument is that scope on a subscription is usually inherited from the same row-level policy that governs ordinary reads, so the row may be measuring the access-control axis a second time. We are not confident either way.

Corrections, counterexamples from real deployed builds, and rubric edits are welcome. The most useful thing you can send us is a step 7 observation from your own generated application: what the view showed after sixty seconds offline, before you reloaded it.

BuilderProof Editorial Team. "Realtime Subscription Correctness: A Proposed Axis for Whether a Generated Live View Ever Notices It Stopped Being Live (September 2026)". BuilderProof, September 2026. https://www.builderproof.org/benchmarks/realtime-subscription-correctness-axis-proposal-september-2026.

@misc{builderproof-realtime-subscription-correctness-axis-proposal-september-2026,
  title  = {{Realtime Subscription Correctness: A Proposed Axis for Whether a Generated Live View Ever Notices It Stopped Being Live (September 2026)}},
  author = {{BuilderProof editorial team}},
  year   = {2026},
  month  = {sep},
  howpublished = {\url{https://www.builderproof.org/benchmarks/realtime-subscription-correctness-axis-proposal-september-2026}},
  note   = {BuilderProof, builderproof.org}
}