Secrets-Handling Posture: A Proposed AI Builder Benchmark Axis

Quick Answer

Secrets-handling posture is how an AI app builder stores your API keys and environment secrets and keeps them out of the browser. It is distinct from end-user auth and from data portability, and no public AI-builder benchmark scores it yet. This July 2026 proposal defines a reproducible five-check rubric (dedicated store, server-side-only path, documented encryption at rest, active client-exposure guardrail, rotation guidance) and applies it to five builders (Lovable, Bolt.new, v0, Replit, Base44) using their own documentation. The preliminary read: all five ship a server-side secrets store, but they diverge on documented encryption at rest and on whether a client-exposure guardrail is active or merely conventional.

BuilderProof publishes proposed axes before scoring them, so the methodology is public and criticized before it becomes a leaderboard. This is the sixth axis in the series, after code portability, data-layer portability, iteration fidelity, auth posture, and cost-to-ship. It is a proposal, not a verdict.

Why secrets handling deserves its own axis

We already score how a builder handles the code it generates and the data it stores. Neither of those captures a third, separately-failing thing: the credentials the app needs to talk to Stripe, an email provider, or a model API.

The failure mode is specific and severe. A leaked customer record is a breach of one row. A leaked service-role key or payment secret baked into a client bundle is a breach of the whole account, and it is exploitable by anyone who opens browser devtools. The mechanism is well documented: in a Next.js app, only variables prefixed NEXT_PUBLIC_ are inlined into the JavaScript sent to the browser, and everything else stays server-only (Next.js environment variables docs, 2026). Most AI builders generate exactly this kind of app, so the question "where does my API key end up" has a concrete, checkable answer per platform.

That makes secrets handling a good benchmark axis: it has a clear failure mode, a documented mechanism, and observable, per-vendor behavior. It also sits beside axes we already publish without overlapping them. It is not data-layer portability, which asks whether you can get your data out, and it is not code portability, which asks whether you own the generated code. It is a third, separately-failing property: where the running app's credentials live.

The proposed rubric: five checks

Each builder is scored on five documentation-verifiable checks. Each check is recorded as Documented, Partial, or Not documented. We deliberately avoid a single 0 to 10 number at the proposal stage; the point of publishing the rubric first is to let the cohort and readers argue with the checks before any score is attached.

• S1 — Dedicated secrets store. A first-class, documented secrets or environment manager exists, so a key never has to live in source code.
• S2 — Server-side-only execution path. The docs show secrets consumed by server, edge, or backend functions and state they are not shipped to the browser.
• S3 — Encryption at rest (documented). The vendor explicitly documents that stored secrets are encrypted at rest, not just "kept secure."
• S4 — Active client-exposure guardrail. A mechanism, beyond prose advice, that prevents or flags a secret leaking into client code: automatic detection, an enforced boundary, or "never visible to viewers" sharing behavior.
• S5 — Rotation and hygiene guidance. Documented guidance to rotate keys, remove unused ones, and never hardcode.

A note on scope: a check measures what the vendor documents, not what its infrastructure secretly does. "Not documented" means the public docs did not state it as of July 2026; it is not a claim that the platform is insecure.

Preliminary cohort read (July 2026)

We priced five builders on their own published docs: Lovable, Bolt.new, v0, Replit, and Base44. Values below are documentation reads, each linked to the source.

The cohort converges on S1 and S2: every builder ships a documented secrets store and routes secrets through a server, edge, or backend function rather than the browser. Where they separate is S3 and S4.

Lovable

Lovable documents a Secrets feature and, distinctively, an active guardrail: it "automatically detects API keys pasted into the chat and guides you to store them securely in Secrets instead of hardcoding them into your code," and it steers users to "create an Edge Function to make the API call server-side" (Lovable security docs, 2026). That automatic detection is the strongest S4 in the cohort. Its published docs did not state encryption at rest (S3, Not documented) and emphasized storage over rotation (S5, Partial).

Bolt.new

Bolt provides a dedicated Secrets panel whose values are "never visible to viewers, so you can share projects without exposing secrets," used by "server functions (sometimes called edge functions)" so that "private information never ends up in your users' browsers or devices" (Bolt secrets docs, 2026). It is the only builder whose docs explicitly cover S5 hygiene: rotate keys regularly, remove unused keys, never store secrets in plain text. Encryption at rest was not explicitly stated (S3, Not documented).

v0 (Vercel)

v0 generates Next.js apps that run on Vercel, whose environment variables "are encrypted at rest" and are, by Next.js convention, server-only unless prefixed NEXT_PUBLIC_ (Vercel environment variables docs, June 2026). That earns a documented S3. Its S4 is scored Partial rather than Documented because the browser boundary is a framework convention the developer must apply correctly, not an active detector: a mislabeled NEXT_PUBLIC_ secret is inlined into the client bundle at build time exactly as designed.

Replit

Replit's Secrets tool "automatically encrypts the data and makes it available to your Replit App as an environment variable," using "AES-256 encryption at rest and TLS encryption in transit," and its visibility model hides values from non-owner remixers and viewers (Replit Secrets docs, 2026). Naming the cipher is the most explicit S3 in the cohort. S5 is Partial: the docs warn against hardcoding but say less about rotation.

Base44

Base44 exposes a secrets CLI (secrets set, secrets list, secrets delete) whose values "are available to your backend functions as environment variables via Deno.env.get()" (Base44 secrets docs, 2026). That is a clear S1 and S2. The surrounding hygiene surface is thinner in its public docs (S3 and S5 Not documented, S4 Partial). Consistent with our standing guardrail, thin docs are scored conservatively and are not read as proof of weak infrastructure.

Guardrails on this reading

Three limits, stated plainly so the axis stays honest.

First, documentation is a proxy. A vendor can encrypt secrets at rest without saying so; a "Not documented" is a gap in the public docs on the date read, not a security finding.

Second, a server-side managed backend is not automatically better or worse than a bring-your-own-key model. It is a different trust boundary, and this axis measures documented handling, not architecture preference.

Third, these are point-in-time reads for July 2026. Docs change. Every cell above links to its source so the check can be re-run and disputed. That is the whole design: our benchmark methodology is public first and scored second.

How to reproduce this

1. Open the builder's docs and search for "secret", "environment variable", and "API key".
2. Record S1 through S5 as Documented, Partial, or Not documented, pasting the exact sentence you relied on.
3. For S4, verify whether the guardrail is active (auto-detection, enforced boundary, viewer-invisible sharing) or a convention the developer must apply.
4. Date every read. Re-run quarterly, because this is a moving target.

If you find a doc that changes a cell, that is a contribution, not a correction to argue with. The rubric is meant to be edited.

BuilderProof editorial team. "Secrets-Handling Posture: A Proposed AI Builder Benchmark Axis". BuilderProof, July 2026. https://www.builderproof.org/benchmarks/secrets-handling-posture-axis-proposal-july-2026.

@misc{builderproof-secrets-handling-posture-axis-proposal-july-2026,
  title  = {{Secrets-Handling Posture: A Proposed AI Builder Benchmark Axis}},
  author = {{BuilderProof editorial team}},
  year   = {2026},
  month  = {jul},
  howpublished = {\url{https://www.builderproof.org/benchmarks/secrets-handling-posture-axis-proposal-july-2026}},
  note   = {BuilderProof, builderproof.org}
}