04Grounded RAG Assistant — Contract Q&A With Clause-Level Citations
Contract Q&A over tens of thousands of pages with clause-level citations in under five seconds
Role: Solution Engineer / AI Architect
Executive summary
Delivered a secure, grounded RAG assistant over contracts and knowledge bases using Postgres/pgvector retrieval, GPT-4, Fabric Data Agent and Foundry—answering complex questions with clause-level citations in under five seconds on average.
- Azure Database for PostgreSQL
- pgvector
- Azure OpenAI (GPT-4)
- LangChain
- Fabric Data Agent
- Microsoft Foundry
- Azure Monitor
- Databricks
An insurance enterprise wanted an AI assistant that answers complex questions by combining structured data with unstructured contract documents. The design intent was a graph-aware RAG approach using Azure PostgreSQL with pgvector as the vector store, integrated with Microsoft Fabric Data Agent and Microsoft Foundry, with strong monitoring and enterprise security.
- Stand up Azure Database for PostgreSQL with pgvector to store embeddings of contract text and knowledge-graph entities.
- Build a pipeline to index contracts (PDF/DOCX) by chunking text and computing embeddings via Azure OpenAI.
- Design a grounded RAG agent pipeline: retrieve relevant sections plus graph relationships (parties, dates, obligations), then have GPT-4 compose answers that quote their sources.
- Integrate Fabric Data Agent for secure live-data queries and Foundry for secure deployment.
- Implement monitoring on Foundry and Azure Monitor and ensure compliance (no sensitive data leakage).
I configured Postgres with pgvector and an ETL (Databricks notebooks triggered by Fabric pipelines) to load and update contract vectors. The agent (Azure OpenAI orchestrated by LangChain) first runs vector similarity search to fetch top passages and graph relationships (parties, dates), then prompts GPT-4 with that context and instructs it to cite the clause it relies on. I chose retrieval-with-metadata over a full graph-traversal stack because the contract domain's questions resolve to clause citations, not multi-hop entity walks, and the metadata layer delivered that without a second database to operate. Fabric Data Agent enabled real-time queries when needed, and we deployed on Foundry for secure authentication. Foundry logging and Azure Monitor tracked Postgres and OpenAI usage.
The assistant let analysts ground contract questions—like “which contracts allow early termination and under what conditions?”—in quoted clauses and section references instead of skim-reading documents. Vector search retrieved relevant passages across tens of thousands of pages of contract text, cited answers made review fast enough to trust in daily use, and monitoring kept average response under five seconds while verifying no unauthorized data left the system. It became a blueprint for secure enterprise QA over private data.
Combining structured and unstructured data demands careful schema design; representing domain relationships alongside embeddings boosted relevance, and I learned to distinguish that pattern from full graph-traversal RAG—metadata-graph retrieval answers this class of questions without operating a second graph engine. Monitoring and guardrails are essential for enterprise-wide assistants. Vector search plus agent orchestration is a powerful pattern for knowledge management.