02Fine-Tuned LLM for NL2SQL — From ~60% to ~85% SQL Accuracy Across Dialects
Fine-tuned GPT-4.1 lifted NL2SQL accuracy from ~60% to ~85% and routed each question to the right data source
Role: Solution Engineer / ML Lead
Executive summary
Fine-tuned a GPT-4.1 model and built a multi-agent NL2SQL system that lets business users query Databricks, Fabric, SQL and Oracle in natural language—reaching ~85% SQL accuracy.
- NL2SQL accuracy (eval on known query types)
- ~85% ~60% on the base model
- Azure OpenAI
- GPT-4.1 fine-tuning
- Azure ML
- LangChain / LangGraph
- Azure AI Search
- Databricks
- Fabric Warehouse
- Oracle
- SQL
An enterprise banking client wanted business users to ask questions in natural language and get answers via SQL across multiple platforms (Azure Databricks, Fabric Warehouse, on-prem SQL, Oracle Autonomous DB). Off-the-shelf NL2SQL tools fell short on accuracy, so the solution required customizing an LLM and integrating it into a robust, privacy-aware architecture.
- Curate a training dataset pairing diverse natural-language questions with correct SQL (open datasets plus the client's internal query logs).
- Fine-tune a GPT-4.1 model via Azure OpenAI to specialize in SQL across Databricks, Fabric and Oracle dialects.
- Design a multi-agent architecture: interpret the question and pick the source, generate SQL with context, and retrieve schema/business context (RAG).
- Implement LangChain/LangGraph to coordinate agents and tool use (e.g., validating queries against a database).
- Enforce governance so the model only runs authorized queries, and log queries for compliance.
Using Azure ML and LangChain, I iteratively refined the dataset with client-specific SQL patterns and synonyms for domain adaptation. The LangGraph multi-agent system used careful prompt engineering: an interpreter agent used AI Search (RAG) to fetch relevant examples and context, the main agent generated SQL guided by them, and a validation agent tested queries on a sandbox to catch errors. I ran daily syncs with the client's innovation team and maintained code in Git. Fine-tuning was one of two approaches on the table—the alternative was heavier prompt/RAG engineering on the base model; I pushed for fine-tuning because off-the-shelf tools had already plateaued on the dialect mix, and the eval loop below is what kept that call honest.
On the shared eval set of known query types, the fine-tuned model reached ~85% accuracy, up from ~60% on the base model measured the same way. The multi-agent design enabled dynamic source selection—routing pipeline questions to Databricks and ledger questions to the Fabric Warehouse. RAG let the system add consulting-style explanations by quoting definitions and policies. The governance layer logged queries and enforced authorization so only permitted sources and shapes of SQL could run. User acceptance tests with analysts confirmed the natural-language interface saved time over manual SQL.
The measured gap taught the transferable part — the base model sat at ~60% on the dialect mix while fine-tuning reached ~85%—a delta I could only justify because both were evaluated the same way. I now always size up whether a prompt/RAG baseline can close such gaps before recommending training spend. SQL security and performance discipline—timeouts and resource limits—is essential to prevent runaway queries. Transparency about how the AI forms SQL and sources context built user trust.