Promythicus FR root@promythicus.com

Promythicus / Blog

One AI model that searches and answers: no separate search engine

By Bardia Shahrestani, Promythicus Inc. · Published

On the CUAD contract-review test, we let the answering model do its own search. Its internal states, read as they are, found 76% of the marked clauses; adding its own yes/no check on the 30 best passages raised that to 88%. End to end it quoted 83% of the clauses, level with reading the whole contract (82%) and with our outline method (81%), while reading a seventh of the text. Nothing was trained and no second model was loaded. It is slower: about 9.5 seconds a question against 4.3 for the outline.

Summary: the answering model's own search finds 76% of clauses, 88% with its own yes/no check; 83% of clauses quoted end to end

The question

A private AI server usually runs two or three models: one that answers, and one or two small ones that search (an embedding model and a reranker). On a single graphics card they compete for memory, and something has to be swapped in and out. Can the answering model do the search as well?

The test

Step one: the model’s own states as a search index

When a language model reads a passage, each of its layers holds a list of numbers describing what it has read. We took those numbers at one middle layer, averaged over the passage, and compared them with the same numbers for the question. The closest passages are read. No training, no extra model.

The layer was chosen on the 40 practice contracts (layer 36 of 64) and then used once on the 62 others.

Search, nothing trained on contractsMarked clauses found
A released general-purpose add-on for an 8B model (CLM)29%
Embedding model (Qwen3-Embedding-0.6B)67%
The same with its reranker74%
The answering model’s own states76%
The same, then its own yes/no check on the 30 best passages88%

Layers 36 to 40 and 52 gave nearly the same result. The last layer was poor (24%): by then the model is thinking about its next word, not about what the passage says.

Step two: the model checks its own shortlist

The 30 best passages are put to the same model one at a time: “Does this passage contain a clause of this kind?” We do not read its answer as text. We read how strongly it leans to “yes” against “no”, and sort the passages by that.

What we dropped: add-ons fitted on contracts

We also fitted a small add-on to the embedding model using the practice contracts. It scored 88%. But CUAD asks the same 41 kinds of question of every contract, so we fitted it on half the kinds and tested on the other half: 90% on the kinds it had seen, 74% on the kinds it had not (71% without it). It had learnt the question list, not contracts. We dropped it. The one-model result above needs no fitting, so it has nothing to memorise.

End to end

Finding the passage is half the job. On the ten contracts and 410 questions of our earlier articles, same prompt and scoring:

Way of readingMarked clauses quotedSeconds a question
Separate search models, 12 passages71%3.1
The model’s own states, 12 passages73%3.0
Outline, 4 parts81%4.3
Whole contract82%3.3 (8.3 for the first question)
The model’s own states and yes/no check, 12 passages83%9.5

With 248 marked clauses, the last three rows are level: a point or two either way is noise. Quotes were word for word 95% of the time.

The cost: speed

The yes/no check reads 30 passages for every question, about 6 seconds. We tried sending them all at once and running several at a time; it made no real difference, because the card is already busy reading. Checking 20 passages instead of 30 saves a third of the time and loses three points. Keeping each passage’s reading from when the contract is taken in, so that only the question is new work, is the next thing to try.

And for a single contract that fits in the model’s memory, reading it whole remains the simplest: it is read once, then each question takes about 3 seconds. The search matters when the document is too long, or when the question covers many documents.

Limits

Questions and answers

Can one AI model do both the search and the answer?
In our test, yes. The answering model’s own internal states found 76% of the marked clauses, and with its own yes/no check 88%, more than a separate embedding model and reranker (74%). Nothing was trained.
Why does that matter on a private server?
A single graphics card has limited memory. If one model does everything, nothing has to be swapped in and out, and there is one model to install, test and keep up to date.
What is the catch?
Speed. The yes/no check on 30 passages takes about 6 seconds a question on a 16 GB card. For a single contract that fits in memory, reading it whole is faster.
Why not train a small add-on on contracts instead?
We tried. It scored well on the kinds of question it was fitted on (90%) and barely helped on kinds it had not seen (74% against 71%). It would not carry to a firm’s own questions.

Sources