Does the Search Box Find It? A Proposed Axis for Search and Retrieval Correctness (September 2026)

Quick answer. Every search box anyone tests is tested by the person who wrote the data, searching for a word they typed themselves, in a table small enough to read on one screen. Under exactly those conditions a substring match, a lexeme match and a ranked index return the same rows in an order nobody notices. PostgreSQL's own documentation states the three ways that stops being true, and states them in the order a growing application will meet them. This page proposes a benchmark axis for it. It publishes no scores.

Search is the feature a generated application is most likely to have and least likely to have thought about. Ask a builder for a customer list and you get a search box, because every list view in every reference design has one. What sits underneath it is almost always a single line, and that line is a decision about linguistics, ordering and indexing made by a model that was asked for a list.

The decision is invisible for a specific and measurable reason. It is not that the failure is rare. It is that the only search anyone performs before shipping is the one search that cannot distinguish a good implementation from a bad one.

What the database says about the obvious approach

The most likely generated line is a pattern match: a case-insensitive comparison against a wildcard on both sides of whatever the user typed. It works. It is also the approach the database's own manual opens by rejecting, and the rejection is unusually direct.

PostgreSQL's introduction to full text search names the operators a generated application will reach for and then lists what they lack. Verbatim: "PostgreSQL has ~ , ~ , LIKE , and ILIKE operators for textual data types, but they lack many essential properties required by modern information systems."* Three properties follow, and each one is invisible in a seeded table.

The first is linguistic. Verbatim: "There is no linguistic support, even for English. Regular expressions are not sufficient because they cannot easily handle derived words, e.g., satisfies and satisfy. You might miss documents that contain satisfies, although you probably would like to find them when searching for satisfy."

The second is ordering. Verbatim: "They provide no ordering (ranking) of search results, which makes them ineffective when thousands of matching documents are found." Read that clause carefully, because it contains the condition under which the defect becomes visible. Thousands. Not five.

The third is cost. Verbatim: "They tend to be slow because there is no index support, so they must process all documents for every search."

Those three sentences describe a feature that is correct on the day it is built and degrades along three independent axes as the application succeeds. None of the three can be observed by the person who built it, on the data they built it with.

The unit the search is performed in is not chosen in the code

The alternative is to search lexemes rather than characters, and this is where a second decision hides. Converting a document to a searchable vector takes an optional configuration argument. That argument decides whether a stemmer runs at all.

PostgreSQL documents the default plainly. Verbatim: "Selects the text search configuration that is used by those variants of the text search functions that do not have an explicit argument specifying the configuration ... The built-in default is pg_catalog.simple, but initdb will initialize the configuration file with a setting that corresponds to the chosen lc_ctype locale, if a configuration matching that locale can be identified."

The simple configuration is not a stemmer. Verbatim, from the dictionaries chapter: "The simple dictionary template operates by converting the input token to lower case and checking it against a file of stop words. If it is found in the file then an empty array is returned, causing the token to be discarded. If not, the lower-cased form of the word is returned as the normalized lexeme." Lower case and stop words. No suffix removal. Under that configuration, searching for satisfy still misses satisfies, which is the first failure the manual listed.

So whether a generated search handles derived forms is decided, when the code omits the argument, by the locale the database server happened to be initialised with. That is a property of the hosting environment, not of the project. The same emitted line can stem on one host and not on another, and nothing in the emitted project records which it expected.

There is a further edge worth naming because it fails silently rather than loudly. A stemmer is language-specific and PostgreSQL says what happens when it is asked for a word it cannot handle. Verbatim: "A Snowball dictionary recognizes everything, whether or not it is able to simplify the word." An English configuration pointed at Spanish text does not raise an error. It returns lexemes. They are simply the wrong ones.

The parser that a space breaks

The sharpest failure in this territory is not a wrong result. It is an error, on the second-simplest query a person can type.

PostgreSQL provides four functions for turning text into a query, and they differ in how much they forgive. The documentation is explicit about the trade. Verbatim: "to_tsquery offers access to more features than either plainto_tsquery or phraseto_tsquery, but it is less forgiving about its input." The specific intolerance is stated a few paragraphs later. Verbatim: "Without quotes, to_tsquery will generate a syntax error for tokens that are not separated by an AND, OR, or FOLLOWED BY operator."

Two words separated by a space are two tokens not separated by an operator. A user who types red shoes into a box wired to the strict parser does not get zero results. They get a server error. A developer who tests with shoes never sees it.

This is not a hypothetical wiring. PostgREST, which sits under a large share of builder-generated data access, exposes four filter operators that map one to one onto those four functions: fts to to_tsquery, plfts to plainto_tsquery, phfts to phraseto_tsquery, and wfts to websearch_to_tsquery. The plainest operator name, fts, is bound to the least forgiving function. PostgREST also makes the stemming language an optional part of the filter, describing the available options as verbatim: "the choice of plain vs phrase search and the language used for stemming." Optional, again, which returns the question to the server's configured default.

What the practical guide says, measured

The document a generated application is most likely to be derived from is not the database manual. It is the platform's how-to guide. We counted the calls in Supabase's full text search guide rather than characterising it, and the counts are worth stating precisely.

Of 21 calls that convert a document to a search vector, 6 state a configuration and 15 do not. Of 19 calls to the strict query parser, zero state a configuration. Of 7 calls to the web-search-syntax parser, all 7 do. The guide is scrupulous about the argument on the forgiving function and silent about it on every single call of the strict one.

The words stemming, stem, lexeme and default_text_search_config appear zero times in that document.

We want to be fair about what that is and is not. It is a practical guide whose job is to get a reader to a working query, it recommends the web-search parser for user-facing search, and it covers ranking, weighting and indexes properly. The point is not that it is wrong. The point is that the single decision that determines whether satisfies matches satisfy is not visible in it, and a model generating a search box from material like this has no reason to make that decision deliberately.

One more detail from the same guide illustrates where the strict parser pushes the problem. Its documented way to search for a phrase is to have the person typing use a plus sign in place of a space. That is a correct workaround and it relocates a parser constraint onto the user of the application.

Seven signals, weights summing to 100

The failing description is what a zero looks like, so the rubric can be applied by reading the emitted source and by probing a running application, rather than from any vendor's description of itself.

Why query parsing carries the most. Every other row on the list produces a degraded result. Only this one produces no result at all, and it does so through an error rather than an empty list, which means the failure is attributed to the application being broken rather than to the data being absent. It is also the row with the lowest threshold: the trigger is a space.

Why the configuration row outranks granularity. Granularity is a choice, and a defensible application can land either side of it. The configuration row is not a choice at all when it is omitted, and the resulting behaviour is decided somewhere the emitted project cannot see. This rubric scores whether a decision was made, not which decision it was, and an inherited default is the absence of one.

Why indexing is mid-table rather than top. An unindexed search is correct. It returns the right rows in the right order and it will do so forever. What it does is convert the application's own success into latency, which is a real cost and a recoverable one, and PostgreSQL says so in the mild register the situation deserves: verbatim: "Although these queries will work without an index, most applications will find this approach too slow, except perhaps for occasional ad-hoc searches."

Four postures

We are deliberately not assigning builders to these postures. This write-up is drawn from platform and database documentation, and we are not going to characterise any vendor's generated output from documents we have read rather than from builds we have run.

Level 0, Pattern match. A case-insensitive wildcard comparison against one column. No linguistic handling, no ranking, and no index that the operator can use. Correct on the data it was written against.

Level 1, Indexed pattern match. Still a character comparison, but the pattern is anchored or a trigram index is present, so the cost stops growing with the table. The result set is byte-for-byte what Level 0 returned.

Level 2, Lexeme search. A search vector, a stated configuration, a forgiving query parser and an index that serves the matching operator. Derived forms match. Order is still whatever the plan returns, or a single ranking call bolted on at the end.

Level 3, Stated retrieval contract. Level 2, plus the matching granularity is written down, the order is defined and deterministic, the searched fields are the displayed fields, and what the system did to the query is visible to the person who typed it.

The step from Level 0 to Level 1 is the only one on this ladder that a user cannot see, and the step from Level 1 to Level 2 is the only one that changes which rows come back. That is worth stating because the two rungs buy different things and are easy to confuse: the first buys durability, the second buys correctness. A team that reaches Level 1 has bought the half that their own testing could never have told them was missing, and has not touched the half it also could not have shown them.

A ten-step protocol

1. Generate the application from a fixed, version-pinned brief asking for a collection with a free-text search box over records that have more than one text field.
2. Enumerate every search path from the emitted source: the operator, the columns covered, the parser function, the configuration argument if any, and the index if any.
3. Run the control that establishes the baseline. Search for a single ASCII word exactly as stored and confirm the matching row comes back. Everything after this is measured against a working search, so that a later zero is a finding rather than a broken build.
4. Type two words separated by a space. Record whether the result is rows, no rows, or an error, and record which.
5. Search for a singular where the stored value is plural, then the reverse. This places the implementation on the granularity row in one direction.
6. Search for a sequence of letters from the middle of a stored word, such as cat against education. Together with step 5 this places granularity in both directions, and the two results are more informative than either alone.
7. Load the collection to a size the application would reach if it succeeded, then repeat step 3 and record wall time against the step 3 measurement.
8. Issue a query that matches many rows. Record the order. Issue it again unchanged and compare the two orders.
9. Search for a value that is visible in the rendered list but stored in a column the box does not cover.
10. Type a stop word on its own, then a query containing an apostrophe and an ampersand. Record what the application tells the reader about what was actually searched for.

The named trap: the seeded-corpus illusion

Every axis we propose names the illusion that hides its defect, because the illusion is usually more useful than the rubric.

Here it is the seeded corpus. The person evaluating the search box wrote the rows it searches. They therefore search for a word they know is present, spelled the way they spelled it, in their own alphabet, in a table of perhaps twenty records. Under that single query, all four postures above return the same rows. Ranking is invisible because there is nothing to rank. Indexing is invisible because a sequential scan of twenty rows is instant. Stemming is invisible because they typed the stored form. The parser's intolerance is invisible because they typed one word.

This belongs to a family we keep meeting: the only observations available at test time are generated by the same actor who built the thing. What makes this member unusual is that the corpus is not merely small, it is authored by the searcher, so the query and the data are correlated in a way they never will be again. The moment a second person types into that box, the query stops being drawn from the same distribution as the content, and every one of the three properties the database manual listed starts to matter at once.

There is a second aggravating property. The condition under which the defect appears is the condition the project is trying to reach. A search that is wrong in all three ways is indistinguishable from a correct one until the application has enough data and enough users to be worth fixing, at which point the fix is a schema change, a backfill and an index build rather than a patch.

What this axis is not

It is not text comparison and collation. Our text-comparison and collation proposal asks whether the system can decide that two strings are one value. This one asks whether a string a reader typed should retrieve a different string somebody else stored, and how the results should be ordered. Stated both ways: an application with a flawless accent-insensitive comparison rule can still return nothing for a plural, because equality is not the operation a search performs, and an application with excellent stemming, ranking and indexing can register one person twice, because the uniqueness check is a different operator on a different column.

That axis is explicit about the boundary from its own side. Its scope note draws four boundaries and search is in none of them, and its prose argues the deprioritisation directly: verbatim: "A search that misses a row is annoying and recoverable: fix the query, and the next search finds it." We think that is correct about repairability, and it is exactly why the retrieval question needs a rubric of its own rather than a single row inside one built for equality.

The two axes also collide, and that neighbour found the collision first, quoting the database: verbatim: "certain operations are not possible with nondeterministic collations, such as some pattern matching operations." The remedy that makes equality behave the way a reader expects can remove the operator a Level 0 search depends on. We resolve it by scope. Score the search path on the operator it actually uses, and score the identity column separately, because an application can and often should make different choices in the two places.

It is not pagination and large-collection reads. Our pagination proposal asks whether an ordering is total, so that a traversal neither repeats nor skips. This one asks whether an ordering is relevant. Stated both ways: a search can have a perfectly unique, stable ordering key and still put the best match on page nine, because a creation timestamp is total and says nothing about relevance, and a search can rank impeccably and still duplicate rows across page boundaries, because a relevance score is not unique. That proposal also concedes the indexing half from its own side, in its limitations: verbatim: "A slow page late in a collection may be the paging technique or may be an unindexed filter that would have been slow anywhere, and the rubric as written does not separate them."

It is not URL-state addressability. Our URL-state proposal asks whether the view a reader is looking at has an address they can send to somebody. This one asks whether the rows in that view are the right rows. Stated both ways: a search whose query lives in the address bar, survives a reload and can be shared can still return the wrong set, and a search with excellent retrieval can be unlinkable because the query was held in component state. The two compose rather than overlap, and the composition is the ordinary case: a shared search URL is only useful if the search behind it is deterministic, which is the order row on this rubric rather than anything that axis measures.

It is not input validation, and it is not API-design consistency. Validation asks whether a value should be accepted. A search query is a legitimate value by construction, and this axis begins after it has been accepted. API-design consistency asks whether filtering is expressed the same way on every collection, which is a question about the shape of the interface rather than about the contents of the result set. An application can express filtering identically across forty endpoints and return nothing for a plural on all forty.

There is also a composition worth recording rather than a boundary. An unindexed search is a sequential scan, and a sequential scan holds a pooled database connection for its whole duration. The index row on this rubric and the pool-sizing row on our database connection and query cost proposal therefore multiply rather than add, and neither axis can see that on its own.

Three layers, and nobody owns the join

The pattern underneath all of this is one we have now met several times. The database documents what a lexeme is and what its own default configuration does. The data-access layer documents four operators and makes the stemming language optional. The client library and the generated code pick one of the four, usually without naming a configuration. Every one of those three documents is accurate about its own layer and silent about the composition, and the composition is the only thing the person using the search box experiences.

That is why this is proposed as an axis rather than filed as a bug. There is nothing here to report to a vendor. There is a decision that nobody is currently required to make, and a rubric can ask whether it was made.

Open questions

• Granularity has no correct answer, only a stated one. A catalogue search probably wants prefixes and substrings. A document search probably wants lexemes. The rubric scores whether the choice is visible, which is weaker than scoring the choice, and we are not confident that is the right trade.
• The index row may be unmeasurable statically. Whether an operator can use an index depends on the index type, the pattern shape and planner statistics. A rubric applied by reading source will sometimes be wrong about it in both directions.
• The order row may be too generous to a single ranking call. Adding one relevance call and sorting by it is a large improvement and still leaves ties undefined. We have not decided whether that deserves most of the fourteen points or half of them.
• We have not established how often the strict parser is actually wired to a user-visible box. The mapping makes it possible and the naming makes it likely. Neither is a measurement, and a future pass should count it in generated builds rather than infer it from documentation.

No scores today. This page proposes the axis, the rubric, the weights, the postures and the protocol. It ranks nobody and it assigns no builder a posture. Publishing the rubric before any result is deliberate, so that the weights can be argued with before they decide anything.

Corrections, counterexamples from real deployed builds, and rubric edits are welcome. The most useful thing you can send us is a step 4 observation from your own generated application: type two words into its search box and tell us which of the three outcomes you got.

BuilderProof editorial team. "Does the Search Box Find It? A Proposed Axis for Search and Retrieval Correctness (September 2026)". BuilderProof, September 2026. https://www.builderproof.org/benchmarks/does-the-search-box-find-it-search-retrieval-axis-september-2026.

@misc{builderproof-does-the-search-box-find-it-search-retrieval-axis-september-2026,
  title  = {{Does the Search Box Find It? A Proposed Axis for Search and Retrieval Correctness (September 2026)}},
  author = {{BuilderProof editorial team}},
  year   = {2026},
  month  = {sep},
  howpublished = {\url{https://www.builderproof.org/benchmarks/does-the-search-box-find-it-search-retrieval-axis-september-2026}},
  note   = {BuilderProof, builderproof.org}
}