One AI model that searches and answers: no separate search engine
On the CUAD contract-review test, we let the answering model do its own search. Its internal states, read as they are, found 76% of the marked clauses; adding its own yes/no check on the 30 best passages raised that to 88%. End to end it quoted 83% of the clauses, level with reading the whole contract (82%) and with our outline method (81%), while reading a seventh of the text. Nothing was trained and no second model was loaded. It is slower: about 9.5 seconds a question against 4.3 for the outline.
The question
A private AI server usually runs two or three models: one that answers, and one or two small ones that search (an embedding model and a reranker). On a single graphics card they compete for memory, and something has to be swapped in and out. Can the answering model do the search as well?
The test
- Data: CUAD, a public set of commercial contracts in which lawyers marked the clauses that answer 41 standard review questions. The 102 test contracts were split once: 40 to try things on, 62 never looked at until a method was fixed.
- Machine: one workstation, one 16 GB graphics card, no cloud.
- Model: a mixed-precision variant of Qwen 3.8 27B that runs on a single 16 GB GPU.
- Score: the share of marked clauses that fall inside the 12 passages read (about 6,000 characters).
Step one: the model’s own states as a search index
When a language model reads a passage, each of its layers holds a list of numbers describing what it has read. We took those numbers at one middle layer, averaged over the passage, and compared them with the same numbers for the question. The closest passages are read. No training, no extra model.
The layer was chosen on the 40 practice contracts (layer 36 of 64) and then used once on the 62 others.
| Search, nothing trained on contracts | Marked clauses found |
|---|---|
| A released general-purpose add-on for an 8B model (CLM) | 29% |
| Embedding model (Qwen3-Embedding-0.6B) | 67% |
| The same with its reranker | 74% |
| The answering model’s own states | 76% |
| The same, then its own yes/no check on the 30 best passages | 88% |
Layers 36 to 40 and 52 gave nearly the same result. The last layer was poor (24%): by then the model is thinking about its next word, not about what the passage says.
Step two: the model checks its own shortlist
The 30 best passages are put to the same model one at a time: “Does this passage contain a clause of this kind?” We do not read its answer as text. We read how strongly it leans to “yes” against “no”, and sort the passages by that.
What we dropped: add-ons fitted on contracts
We also fitted a small add-on to the embedding model using the practice contracts. It scored 88%. But CUAD asks the same 41 kinds of question of every contract, so we fitted it on half the kinds and tested on the other half: 90% on the kinds it had seen, 74% on the kinds it had not (71% without it). It had learnt the question list, not contracts. We dropped it. The one-model result above needs no fitting, so it has nothing to memorise.
End to end
Finding the passage is half the job. On the ten contracts and 410 questions of our earlier articles, same prompt and scoring:
| Way of reading | Marked clauses quoted | Seconds a question |
|---|---|---|
| Separate search models, 12 passages | 71% | 3.1 |
| The model’s own states, 12 passages | 73% | 3.0 |
| Outline, 4 parts | 81% | 4.3 |
| Whole contract | 82% | 3.3 (8.3 for the first question) |
| The model’s own states and yes/no check, 12 passages | 83% | 9.5 |
With 248 marked clauses, the last three rows are level: a point or two either way is noise. Quotes were word for word 95% of the time.
The cost: speed
The yes/no check reads 30 passages for every question, about 6 seconds. We tried sending them all at once and running several at a time; it made no real difference, because the card is already busy reading. Checking 20 passages instead of 30 saves a third of the time and loses three points. Keeping each passage’s reading from when the contract is taken in, so that only the question is new work, is the next thing to try.
And for a single contract that fits in the model’s memory, reading it whole remains the simplest: it is read once, then each question takes about 3 seconds. The search matters when the document is too long, or when the question covers many documents.
Limits
- One dataset, American commercial contracts in English, short enough to fit. We have not yet tested a collection of hundreds of files.
- The scoring is ours and cannot be set beside published CUAD numbers.
- The released add-on (CLM) was run in reduced precision and we could not confirm its intended prompt format, so its 29% may understate it.
- One run of each method.
Questions and answers
- Can one AI model do both the search and the answer?
- In our test, yes. The answering model’s own internal states found 76% of the marked clauses, and with its own yes/no check 88%, more than a separate embedding model and reranker (74%). Nothing was trained.
- Why does that matter on a private server?
- A single graphics card has limited memory. If one model does everything, nothing has to be swapped in and out, and there is one model to install, test and keep up to date.
- What is the catch?
- Speed. The yes/no check on 30 passages takes about 6 seconds a question on a 16 GB card. For a single contract that fits in memory, reading it whole is faster.
- Why not train a small add-on on contracts instead?
- We tried. It scored well on the kinds of question it was fitted on (90%) and barely helped on kinds it had not seen (74% against 71%). It would not carry to a firm’s own questions.
Sources
- The Atticus Project, CUAD: Contract Understanding Atticus Dataset
- Hendrycks et al., CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review (2021)
- Zhang et al., Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025)
- Contrastive-LM, CLM: Contrastive Language Models (code and released head)
