SOW Contract Intelligence · RAG · MCP · 2026
Can AI Learn What Makes a Project Succeed?
Building contract intelligence with RAG and delivery evidence
A retrieval-augmented generation (RAG) assistant for searching, drafting and reviewing Statements of Work. It combines cited contract text with delivery outcomes, uses proven same-industry clauses where they exist, and sends every draft to an engagement lead, deal desk and legal before issue.
System
Retrieval-augmented generation (RAG) contract assistant
Logic
Verbatim extraction · cited drafting · outcome-ranked precedents
Stack
Claude · PostgreSQL + pgvector · Hugging Face embeddings · Gradio · MCP · LlamaIndex (comparison)
Operations
One container on Hugging Face Spaces · evals on start
Public data
48 fictional SOWs for invented clients
48
fictional SOWs across more than six industries
2,861
clauses and components extracted and verified
127
golden evaluation cases
0
uncited claims allowed in a draft
01 · The problem
Knowledge Trapped in Old SOWs
Services teams repeatedly draft the same commercial and delivery terms without a reliable way to use prior outcomes.
The acceptance clause that held up, the payment schedule that protected margin and the pilot terms that led to paid work sit across old documents and people's memories. Drafting is slow, terms vary between teams, and delivery risk is often found after signature.
Search alone is not enough. The useful precedent is the clause tied to a deal's delivery health, margin, client satisfaction, conversion and time to production. The system therefore keeps contract text and outcomes together.
The product question
Can past contracts, ranked by how those deals actually delivered, draft and review new SOWs without the model inventing a single commitment?
02 · Product proof
Ask, Draft, Review and Compare
The implemented product exposes the main contract workflow: explore the corpus, assemble a new draft, check an existing SOW and find missing clauses.
Example questions show what the corpus can answer
Prompts cover acceptance, handover, pilot conversion, time to production, regulatory assumptions, payment schedules and subcontracting terms.
A new SOW starts with the delivery context
Client, product, industry, size, pricing and objective determine which proven components and same-industry precedents are eligible for the draft.
Existing SOWs can be checked before issue
A document or fictional sample is reviewed in memory. Industry, product and pricing are detected from the text before the clause and precedent checks run.
Clause coverage makes omissions visible
The corpus can be filtered by clause type and presence. This view identifies every fictional SOW missing an AI governance clause and keeps the source filenames visible.
03 · System design
A Cited Contract Pipeline
Extraction, retrieval and generation are separate stages with deterministic checks between them.
Ingest
Convert Word and PDF documents to page text
→Sanitise
Apply aliases, redact contact details and run a leak check
→Extract
Return clauses and reusable components against a JSON schema
→Verify
Keep an item only when its quoted text appears on the source page
→Chunk & embed
Create local bge-small vectors and PostgreSQL full-text records
→Retrieve
Combine vector and full-text search, with same-industry precedents first
→Draft / review
Use Claude with a cite-or-fallback rule
→Human sign-off
Engagement lead, deal desk and legal approve anything issued
C1 · Verbatim extraction guard
def verbatim(snippet: str, page_text: str) -> bool:
return len(snippet) > 20 and _norm(snippet) in _norm(page_text)An extracted clause is retained only when the quoted text appears on the source page.
C2 · Draft only from supplied components
ADAPT_PROMPT = """You adapt approved Statement of Work components to a new client brief.
Rules:
- Use only the components provided. Do not add scope, commitments, prices or dates that are not in them.
- Write one paragraph per component you use, and end each paragraph with its tag, e.g. [C12].
- If a component does not fit the brief, leave it out rather than stretching it."""
problems = check_citations(text, {c["component_id"] for c in comps})
if problems:
text = "\n\n".join(
f"{c['body']} [C{c['component_id']}]" for c in comps
)If the adapted response is not traceable to the supplied components, the draft falls back to their verbatim text.
04 · Grounding
Citation Checks and Fallbacks
Three checks stop an extracted or generated statement from becoming an unsupported contract term.
01
Verbatim extraction
The quote must occur on the source page after normalising whitespace and case.
02
Cited drafting
Every paragraph must point to an allowed component; otherwise the verbatim component replaces it.
03
Cited redlines
A suggested redline must cite an allowed canonical clause; otherwise that canonical text is used.
Empty-to-serving build
2,861
extracted items kept
1 dropped
The quote was not present verbatim on its source page. The check rejected it before indexing.
The extraction model returns structured clauses and components, but its output is not trusted as source text. Drafting has a second check at paragraph level. Review redlines have a third check against the canonical clause IDs supplied to the model. The safeguards are deliberately simple enough to inspect in code and in the resulting document.
The dropped item is the control working
The useful measure is not that extraction produced no errors. It is that an unsupported quote did not pass into retrieval or drafting.
05 · Delivery evidence
Beyond Similarity Search
RAG finds relevant wording. Structured queries connect it to delivery outcomes and enforce complete coverage where a top-k result would be unsafe.
- Outcome-ranked precedents
- Clauses are returned with delivery health, margin, satisfaction, conversion and time to production. Stronger outcomes rank first.
- Commercial controls
- Supplier-funded pilots are checked for a time-box, named success measures, acceptance thresholds, governance and an exit plan.
- Sector overlays
- Financial-services and hospitality briefs add the relevant regulatory, audit, subcontracting, data and exit terms.
Limit: the fictional corpus tests whether the system joins terms to outcomes. It does not show that a contract term caused a delivery result or establish legal sufficiency.
06 · Review a SOW
Structured Review with Human Approval
Review checks the uploaded document against the strongest applicable precedents, then returns a red/amber/green status, clause findings, five priority fixes and cited redlines.
Upload boundary
Memory only
Uploads are sanitised before analysis, held in memory for the review and discarded afterwards. They never enter the contract library.
Do not upload confidential client documents.
The product carries this warning in the review surface.
Overall status
A concise red/amber/green risk summary across expected clauses.
Clause table
Status, finding, canonical example and redline.
Top five fixes
The most material changes before signing.
Cited redlines
Every proposed replacement points to a proven clause.
C3 · A redline must cite
red = out.get("redline")
if red and not (
{int(t) for t in re.findall(r"\[K(\d+)\]", red)} & allowed
):
red = " ".join(
f"{c['quote']} [K{c['clause_id']}]" for c in canon
)If the proposed redline has no allowed canonical citation, the review substitutes the verbatim canonical text.
Illustrative contract review, not legal advice.
07 · Evaluation
Known Answers from a Controlled Corpus
The 127-case golden set was built with the fictional corpus, so expected clauses, files, pages and planted review gaps are known by construction.
127 controlled cases
The suite covers extraction accuracy, retrieval hit@8, clause coverage, precedent recall, citation checks and planted review gaps.
Evaluation boundary
It tests known cases in the fictional corpus. It does not establish legal sufficiency, performance on messy client contracts or improved delivery outcomes.
08 · Framework due diligence
LlamaIndex or a Hand-Built Retriever?
Two retrieval implementations sit behind the same function contract: direct PostgreSQL and pgvector retrieval, and a LlamaIndex version using the same corpus, chunks and embeddings.
The retrieval layer is hand-written: pgvector similarity and PostgreSQL full-text search, fused with reciprocal rank fusion in one SQL query. I also built the same path in LlamaIndex behind the same interface. It adds standard retriever abstractions, metadata filters, events and tracing hooks; the Ask tab can switch between the two implementations.
Operational trade-off
| Measure | Hand-built | LlamaIndex |
|---|---|---|
| Query path | Lower framework overhead | Small framework overhead |
| Instrumentation | Custom spans and metrics | Events, callbacks and tracing integrations |
What the framework gives
What it costs
The design lesson
Keep retrieval behind one small interface and measure it.
The grounding guarantees remain outside retrieval either way. The hand-built path is simpler for this contained corpus; LlamaIndex becomes useful when its connectors, shared abstractions and tracing integrations justify the extra framework overhead.
09 · Production shape
One Explicit Deployment
Claude, local bge-small embeddings, PostgreSQL, pgvector, Gradio, the MCP server and background evaluation run from one entry point in a Hugging Face Space. From an empty CPU Basic Space, the verified start-up log reached serving in about twelve minutes.
The design lesson
The value was not generation. It was making the practice's own delivery history queryable, and refusing to say anything it could not cite.
Discuss the system
Contract knowledge with a source trail
The live demo is password-protected. Access is available on request.