Does the scheduled job actually run? A background-work correctness axis for AI app builders

An AI app builder will happily generate a nightly digest email, a weekly report, an expiring-invite cleanup and a "retry the failed payments" routine. Every one of them looks finished, because the chat says it is finished and the code reads exactly like code that would work. The question this axis proposes to measure is whether that work runs at all once nobody is watching: whether a task started after a response survives the runtime, whether a missed scheduled run is ever noticed, and whether a run that fires twice does its job twice. We are not scoring builders today. We are publishing a candidate axis, its rubric, its posture levels and a reproduction protocol, and opening all of it for comment before it enters the composite.

Quick Answer

Background-work correctness is a proposed BuilderProof benchmark axis, drafted September 5, 2026, that scores whether the scheduled and deferred work an AI app builder emits actually executes, and executes once, on the runtime it was deployed to. It is measured from the exported project and the deployment configuration, not from the chat transcript. The rubric weights seven signals: whether work started after a response is held open rather than abandoned, whether a missed run is recovered by reprocessing outstanding work, whether a duplicated run is harmless, whether a job that outlasts its own interval is prevented from overlapping itself, whether the scheduled entry point is authenticated, whether the emitted schedule is actually runnable on the target plan and timezone, and whether a run outcome is recorded anywhere a person could query. The failure mode it targets is not a crash and not a slow page. It is a task that was never attempted, on a system where nothing anywhere records that it was not attempted. The platforms document this directly: one runtime states that an un-awaited asynchronous call can be cancelled and fail silently, and another states that a missed scheduled run produces no runtime log at all. This page is an axis proposal open for community edits, not a leaderboard.

BuilderProof is an independent, community-editable benchmark for AI app builders. It accepts no vendor payment or sponsorship. This post proposes a new axis, defines how it would be scored, and cites primary sources read on September 5, 2026. It does not score any builder on this axis yet, and it does not name a winner.

Why scheduled and deferred work deserves its own axis

Every axis we have proposed in this series so far scores something a person can see. A layout is wrong, a form accepts bad input, a page shows a stale number, a live view stops updating. Even the quiet failures are quiet in front of somebody.

Background work is different in kind, and the difference is the whole argument for a separate axis. There is no user in the room. Nothing renders. Nothing returns a status code to anyone who would read it. If a nightly job does not run, the only evidence is an absence: an email that did not arrive, a row that was not cleaned up, a total that is slightly wrong in a report nobody reconciles. The feedback loop that catches every other defect class is missing by construction.

That matters more for generated code than for hand-written code, and again the reason is structural rather than a criticism of any vendor. A model optimising for a working preview has no gradient toward background-work correctness, because a preview has no tomorrow. You cannot demonstrate a nightly job in a chat window. The generator writes the handler, the handler compiles, the chat reports success, and the first honest test of the claim is twenty-four hours later on somebody else's infrastructure.

There is a second reason, and it is the one that convinced us to draft the axis now. The platforms these apps deploy to have written down, in their own documentation, exactly what a correct background job must do. This axis does not need us to invent a standard. It only needs us to check whether the emitted code does what the runtime it targets already says it must.

What "background-work correctness" means here

The axis scores the emitted artifact and its deployment configuration, not the platform's marketing and not the chat experience. Specifically it asks, of the untouched export:

For each unit of work the application performs without a user present, does the generated code and configuration ensure that the work is attempted, that it completes at most once in effect, and that a failure to attempt it is discoverable?

That definition deliberately covers two distinct shapes, and it is worth separating them at the outset because they behave in opposite directions.

1. Deferred work. Work started inside a request but intended to outlive the response. Sending a welcome email after returning 201. Writing an audit row. Warming a cache. Calling a third-party webhook. The user gets their answer immediately and the work is supposed to finish afterwards.
2. Scheduled work. Work with no originating request at all. A cron expression, a database scheduler, a queue consumer. Nightly cleanups, weekly digests, hourly syncs, retry sweeps.

Both are "background work" in ordinary speech and both fail silently, but the mechanisms differ enough that the rubric scores them separately.

The finding: every layer documents its own failure mode, and the missing-run half leaves no trace

We read the documentation for the request runtimes, the schedulers and the database primitives that sit under the apps these builders emit. The pattern is consistent and it is not hidden. Each layer states plainly what it will and will not do. What is striking is how much of that stated behaviour has no visible consequence when it goes wrong.

The request runtime cancels what you do not hold onto

The clearest statement comes from Cloudflare. Its Workers context documentation describes ctx.waitUntil() as a primitive that "extends the lifetime of your Worker, allowing you to perform work without blocking returning a response, and that may continue after a response is returned." It then states the consequence of not using it, in its own words: an async call "that is neither awaited nor passed to ctx.waitUntil() can be canceled when the invocation ends, dropping logs, leaving writes unfinished, or failing silently."

Read that carefully. The phrase is failing silently, written by the runtime vendor, about the most idiomatic mistake in the language. A floating promise is not a lint error. It is not a type error. It is the shape that a generator produces when it writes sendEmail(user) on its own line without an await, which is exactly what "send them a welcome email after signup" invites.

The same document notes an important qualification we want to record rather than flatten: "If the client is still receiving the response, including a streamed response body, the Worker invocation remains active without ctx.waitUntil()." So the defect is not universal across every shape of handler, and a fair rubric must say so.

The scheduler documents best-effort delivery in both directions

Vercel's cron job documentation is unusually explicit, and it is the single best source for this axis because it states both failure directions in adjacent paragraphs.

On missed runs, verbatim: "Cron job delivery is best effort. Most invocations run as scheduled, but occasional transient network errors can prevent a request from reaching your function. In those cases, your function does not execute, and no runtime log is created for that scheduled run."

On duplicate runs, in the same section: "Cron delivery can also occasionally invoke the same scheduled run more than once."

The document then prescribes the remedy, and the prescription is the reason this axis has a rubric at all: "Design your operations to be idempotent and reconciliation-based so each run can safely reprocess outstanding work since the last successful run." It gives a worked pair, again verbatim: "Good: 'Set user status to active' (running twice has the same effect). Bad: 'Increment user credit by 10' (running twice doubles the credit)." And it names catching up as a separate obligation: "Query and process all work since the last successful run to catch up after a missed invocation."

Two more statements from the same page bear directly on the rubric. On failure, verbatim: "Vercel will not retry an invocation if a cron job fails." On overlap: "If your cron job runs longer than the interval between invocations, Vercel can trigger a second instance while the first is still running. This can lead to race conditions, duplicate processing, or data corruption," with the recommendation to "use a lock mechanism." The page closes the loop by asking for both at once: "Use both locks (to prevent concurrent runs) and idempotent reconciliation (to handle duplicate or missed runs safely) for the most reliable cron jobs."

Cloudflare's model differs in a way worth recording, because a rubric that assumes one scheduler is a rubric that measures the wrong thing on the other. Its Cron Triggers documentation maps a cron expression to a dedicated scheduled() handler rather than to an HTTP route, states that "Cron Triggers execute on UTC time," and documents a noRetry outcome field that "is true when the scheduled handler calls controller.noRetry()." The scheduled handler reference adds that "the runtime waits for the promise returned by the scheduled() handler to resolve (up to the 15-minute duration limit)." We note the divergence and decline to rank it: one platform says plainly that it does not retry, the other exposes an explicit way for a handler to decline a retry. Those are different designs, and an axis should score the code against whichever one it was deployed onto.

The trigger is often a public URL, and the lock on it is opt-in

Vercel's cron overview states the mechanism: "To trigger a cron job, Vercel makes an HTTP GET request to your project's production deployment URL, using the path provided in your project's vercel.json file."

That is a design decision with a consequence the generator has to act on. The scheduled entry point is a route on the public internet. The platform provides the means to protect it and documents that the protection is something your code must perform: adding a CRON_SECRET environment variable means "the value of the variable will be automatically sent as an Authorization header when Vercel invokes your cron job," and then "your endpoint can then compare both values, the authorization header and the environment variable, to verify the authenticity of the request."

The header arrives on its own. The comparison does not. A handler that omits the check is a public, unauthenticated, side-effect-producing endpoint that any stranger can invoke as often as they like, which is a different and worse property than merely running on the wrong schedule. That is a distinct property from whether the job runs on the right schedule, and it is the reason trigger authorization is scored separately below rather than folded into the schedule signal.

Two smaller behaviours from the same page belong in a reproduction protocol because they turn a wrong guess into a silent no-op rather than an error. Verbatim: "Cron jobs do not follow redirects. When a cron-triggered endpoint returns a 3xx redirect status code, the job completes without further requests." And: "If you create a cron job for a path that doesn't exist, it generates a 404 error. However, Vercel still executes your cron job."

The database layer offers the primitives and names their cost honestly

Under most of what these builders emit sits PostgreSQL, and PostgreSQL has had the queue primitive for years. Its SELECT documentation describes SKIP LOCKED precisely, and the sentence is more careful than most write-ups that cite it: "With SKIP LOCKED, any selected rows that cannot be immediately locked are skipped. Skipping locked rows provides an inconsistent view of the data, so this is not suitable for general purpose work, but can be used to avoid lock contention with multiple consumers accessing a queue-like table."

That is the canonical claim-a-row-without-blocking pattern, documented by the database, with its cost stated in the same breath. A generated worker that selects pending rows without it will hand the same row to two consumers. A generated worker that uses it has an answer to the overlap signal that costs nothing extra.

Scheduling also exists inside the database. pg_cron is "a simple cron-based job scheduler for PostgreSQL (10 or higher) that runs inside the database as an extension," and it "creates a background worker that tracks jobs in the cron.job table." Two of its defaults matter for this axis. Its cron.timezone setting defaults to GMT, so a job written as "9am" is 9am GMT unless somebody said otherwise. And cron.max_running_jobs defaults to 32, which is a real ceiling rather than an abstraction.

The observability property is the one we did not expect to find and which improved the rubric. pg_cron records every run in cron.job_run_details with a status and a return message, including failures with messages such as server restarted and job canceled. Supabase Cron, which "uses the pg_cron Postgres database extension" underneath, states the same: "Every Job's run and its status is recorded on the cron.job_run_details table," and adds an operational recommendation of "no more than 8 Jobs run concurrently" with "each Job should run no more than 10 minutes."

So on that path, "did last night's job run?" is a query. On an HTTP-triggered path where a missed invocation produces no log line, the same question has no answer at all. That asymmetry is a real, checkable property of an emitted design, and it became signal seven.

Durable queueing is likewise available rather than exotic. Supabase Queues describes itself as "a Postgres-native durable Message Queue system with guaranteed delivery built on the pgmq database extension," and states "Exactly Once Message Delivery: A Message is delivered exactly once to a consumer within a customizable visibility window." We quote that qualification deliberately. The guarantee is scoped to a window, which is the ordinary and honest shape of such a guarantee, and a consumer written as though it were unconditional is a consumer that will eventually double-process.

The honest counterweight

Three things cut against the strong version of this argument and we would rather state them than have a reader find them.

First, the platform limits are documented, discoverable and not hidden behind support tickets. Everything above came from public pages in a single afternoon. A builder that reads its own target platform's documentation could satisfy most of this rubric without inventing anything.

Second, some of these failure modes are bounded by the plan rather than by the code. Vercel's cron usage page, last updated July 15, 2026, states that Hobby accounts are "limited to cron jobs that run once per day" and that more frequent expressions "will fail during deployment." A deploy-time failure is loud. It is the one failure in this whole area that a person will actually see, and it should count in a builder's favour that the platform refuses rather than silently degrades.

Third, and most important: we have not measured any of this yet. Everything above is a reading of what the runtimes promise. Whether generated code honours it is precisely the open question this proposal exists to answer, and we would be doing the thing we criticise if we asserted the answer in advance.

The proposed rubric

Seven signals, weighted to 100. Every signal is scored from the exported code and the deployment configuration, so any reader with the same export can check the score and dispute it.

Two notes on the weighting, because both are contestable and we would rather be argued with.

The top two signals are weighted above the rest because they are the two that produce nothing, as opposed to producing the wrong thing. Everything below them describes work that happened incorrectly. The first two describe work that never happened, which is harder to notice and harder to reconstruct after the fact.

Run observability is weighted lowest at 6 despite being, in our view, the signal most likely to save an operator's week. It is low because it is a mitigation rather than a correctness property: a job that reliably records that it failed is still a job that failed. We can be persuaded to raise it.

The four postures

Scores group into four postures. The labels are for reading; the number is the rubric.

Absent. Background work is written as though the runtime were a long-lived server. Deferred work is fired without being awaited. Scheduled work, if it exists at all, is a timer created inside a request handler or a module-level interval. Nothing is authenticated, nothing is recorded, and there is no schedule declared in any deployment configuration at all.

Declared. A real scheduler is configured. A cron expression exists in the deployment configuration or in the database, and it points at a real handler. The handler does the work for the window it fired in and nothing else. A missed run is lost permanently, a duplicate run does the work twice, and the endpoint accepts any caller.

Defended. The handler is authenticated, the schedule is valid for the plan and states its timezone intent, and the operation is written so a duplicate delivery is harmless. Overlap is prevented by a lock or a row claim. A missed run is still simply missed, because the query is bounded to the current window rather than to work outstanding since the last success.

Reconciling. The handler asks what work is outstanding rather than what happened in the last interval, so a missed run is absorbed by the next one. Duplicate delivery is harmless. Overlap is prevented. The entry point is authenticated. The outcome of each run is written somewhere queryable, so a silent gap is visible after the fact rather than inferred from a customer complaint.

We expect the honest distribution across a real cohort to be weighted toward the first two, and we are prepared to be wrong about that. Publishing the expectation before the measurement is the point.

How to reproduce it

The protocol is written so that a skeptic with the same export reaches the same score, or shows us where it is wrong.

1. Prompt for background work explicitly, in ordinary language. Ask for a welcome email after signup, a nightly cleanup of expired records, and a weekly summary. Do not use the words "cron", "queue", "idempotent" or "background job" in the prompt. The axis is about what the builder does when a normal person describes a normal requirement.
2. Export the project untouched. Score the artifact, not the chat.
3. Enumerate every unit of background work. Record for each whether it is deferred or scheduled, and where it is declared: a deployment configuration file, a database scheduler, a platform dashboard setting, or nowhere.
4. For each deferred unit, trace the promise. Record whether it is awaited, passed to a lifetime-extension primitive, streamed with the response, or none of those. Match the finding against the documented behaviour of the runtime the project is configured to deploy to, because the answer differs between them.
5. For each scheduled unit, read the selection query. Record whether it selects work by a window anchored to now, or by outstanding state. This single question separates Defended from Reconciling and it is usually answerable from one line of SQL.
6. Run the same handler twice against a seeded database and diff the resulting rows. This is the duplicate-run test and it needs no scheduler at all. An operation whose second run changes nothing passes.
7. Simulate a missed run by advancing the clock past a scheduled window without invoking the handler, then invoke it once. Record whether the skipped work is picked up or permanently skipped. This is the single most diagnostic step in the protocol and it is the one no chat transcript can answer.
8. Invoke the scheduled entry point directly with no credentials. Record the status code and, more importantly, record whether the side effect happened.
9. Check the declared schedule against the target plan. Record whether the expression deploys at all, what precision the platform documents for it, and whether any timezone was expressed rather than inherited from a default.
10. After all of the above, ask where the evidence of each run lives. If the answer is an ephemeral log stream, record that a missed run leaves nothing. If the answer is a table, record that it can be queried.

Steps 6, 7 and 8 are cheap, deterministic and independent of any vendor's infrastructure, which is why we would run them first if the protocol has to be cut down.

The named trap: the fire-and-forget illusion

Every axis in this series names the illusion that hides its defect. This one is the fire-and-forget illusion.

The shape is this. The code reads correctly. The function is called. In local development it even works, because a local development server is a long-lived process that has no reason to stop when a response is flushed. The endpoint returns 200. The chat says the feature is done. A reviewer reading the diff sees a call to a function that sends the email, and the call is right there on the line.

Nothing in that chain is a lie, and nothing in it is evidence. The runtime, not the code, decides whether the call outlives the response, and the runtime has written down that it will cancel work you did not explicitly hold open. The illusion is not that the code looks like it works. It is that calling a function feels like doing the thing, and in a serverless request lifecycle those are two different claims.

The scheduled half of the trap is quieter still. A cron expression in a configuration file looks like a promise from the platform that the work will happen. It is a best-effort request, documented as such, on a route that may not be protected, at a precision that may be an hour wide, on a plan that may refuse the expression outright, with no log line produced in the one case you would most want to know about.

How this relates to our existing axes

This axis overlaps two we have already proposed, and in both cases the boundary is worth stating in both directions rather than asserted once in our favour.

Concurrent-write safety. Our concurrent-write safety proposal weights idempotency of retried mutations and retry discipline, and a reader could reasonably ask whether duplicate-run safety here is the same signal wearing a different hat. It is not, and the cleanest evidence is that the platform documentation treats them as two separate obligations in two adjacent sentences: make the operation idempotent, and query all work since the last successful run. The first is about too many effects. The second is about too few, and it has no analogue in the concurrent-write axis at all.

They also move in opposite directions, which is the test we hold ourselves to before proposing anything. An application can guard every user-facing mutation with a version predicate and a unique constraint, scoring well on concurrent-write safety, and still lose an entire night's digest because its scheduled query is anchored to the last twenty-four hours rather than to outstanding work. Idempotency says nothing whatever about catch-up. Conversely, an application can reconcile perfectly against outstanding state, absorbing every missed run, while its user-facing edit path reads a row into memory, mutates it and writes the whole object back, losing updates all day. High on one, zero on the other, in both directions.

There is a second difference that matters for the reproduction protocol. A duplicated user-initiated request has a user attached to it, who will usually notice the double charge. A missed scheduled run has nobody attached to it. The concurrent-write axis can be tested by two overlapping clients. This one has to be tested by not invoking something and then checking whether anyone noticed.

Timezone and date handling. Signal six touches our proposed timezone and date handling axis, and we want to be precise about the split so neither axis claims the other's ground. That axis asks whether stored and displayed instants are correct: whether the app writes an unambiguous instant, whether it renders in the viewer's zone, whether it survives a daylight-saving boundary. This axis asks something narrower and different: whether the schedule was expressed in the zone the requirement meant. A digest requested for "9am" and emitted as 0 9 * * * against a scheduler documented to run in UTC or GMT is not a date-formatting defect. Every timestamp in that application can be flawless and the email still arrives in the middle of the night. Equally, an application can get its schedule zone exactly right and still render every timestamp to every user in the server's zone. Separate properties, separately scored.

Our general approach to proposing, weighting and retiring axes is set out in how we benchmark AI app builders.

What we are not claiming

We are not claiming that any builder fails this axis. We have not run it. No builder has a score on it, no builder is named in any posture above, and nothing here should be read as a placement.

We are not claiming that the platforms are at fault. Every behaviour quoted above is documented, most of it in the section a developer would reach for first, and several of the documents go out of their way to prescribe the remedy as well as describe the risk.

We are not claiming this is a new discovery in software engineering. At-least-once delivery, reconciliation loops and distributed locks are old, settled and well understood. The open question is narrower: whether code generated from a one-line natural-language request applies any of it.

We are not claiming the rubric is right. Seven signals and a set of weights is a first draft, published so it can be attacked before it is used.

Limitations and open questions

The runtime is a variable, not a constant. The same generated handler can be correct on one platform and cancelled on another, because the lifetime rules differ. Scoring "the code" without pinning the deployment target is not meaningful, so the protocol pins it. That makes the score a property of a code-plus-target pair, which is more honest and less quotable, and we have not resolved that tension.

Deferred and scheduled work may deserve separate axes. We have combined them because both fail without an observer, but they have different mechanisms, different remedies and possibly different distributions across a cohort. A reader who thinks these are two axes wearing one name has a real argument and we would like to hear it.

Step 7 has no agreed implementation. Simulating a missed run is easy to describe and fiddly to standardise. Advancing a clock, withholding an invocation and defining "the next run" differ enough between an HTTP-triggered scheduler and an in-database one that two careful people could produce different numbers. Until that is pinned down, signal two is the least reproducible thing in the rubric, which is uncomfortable given it carries the second-largest weight.

We may be over-weighting the public-trigger signal. Twelve points for authentication assumes the trigger is a public route, which is true on one of the schedulers we read and false on another, where the handler is not reachable over HTTP at all. A signal that is inapplicable on some targets should probably be renormalised rather than scored as a pass, and we have not decided how.

We have not looked at cost. A reconciling job that queries all outstanding work is more expensive than one bounded to a window, and on metered compute that difference is a real bill. An axis that rewards correctness without acknowledging its cost is giving incomplete advice.

Contributors are welcome to challenge any of the above through the BuilderProof methodology process. As with every BuilderProof axis, the goal is a measure a skeptic can reproduce and disagree with on the numbers rather than on the definition.

BuilderProof Editorial Team. "Does the scheduled job actually run? A background-work correctness axis for AI app builders". BuilderProof, September 2026. https://www.builderproof.org/benchmarks/does-the-scheduled-job-actually-run-background-work-axis-september-2026.

@misc{builderproof-does-the-scheduled-job-actually-run-background-work-axis-september-2026,
  title  = {{Does the scheduled job actually run? A background-work correctness axis for AI app builders}},
  author = {{BuilderProof editorial team}},
  year   = {2026},
  month  = {sep},
  howpublished = {\url{https://www.builderproof.org/benchmarks/does-the-scheduled-job-actually-run-background-work-axis-september-2026}},
  note   = {BuilderProof, builderproof.org}
}