02Fine-Tuning an LLM for NL2SQL (Natural-Language-to-SQL Agents)
Role: Solution Engineer / ML Lead
Executive summary
Fine-tuned a GPT-4.1 model and built a multi-agent NL2SQL system that lets business users query Databricks, Fabric, SQL and Oracle in natural language—reaching ~85% SQL accuracy.
- Azure OpenAI
- GPT-4.1 fine-tuning
- Azure ML
- LangChain / LangGraph
- Azure AI Search
- Databricks
- Fabric Warehouse
- Oracle
- SQL
An enterprise banking client wanted business users to ask questions in natural language and get answers via SQL across multiple platforms (Azure Databricks, Fabric Warehouse, on-prem SQL, Oracle Autonomous DB). Off-the-shelf NL2SQL tools fell short on accuracy, so the solution required customizing an LLM and integrating it into a robust, privacy-aware architecture.
- Curate a training dataset pairing diverse natural-language questions with correct SQL (open datasets plus the client's internal query logs).
- Fine-tune a GPT-4.1 model via Azure OpenAI to specialize in SQL across Databricks, Fabric and Oracle dialects.
- Design a multi-agent architecture: interpret the question and pick the source, generate SQL with context, and retrieve schema/business context (RAG).
- Implement LangChain/LangGraph to coordinate agents and tool use (e.g., validating queries against a database).
- Enforce governance so the model only runs authorized queries, and log queries for compliance.
Using Azure ML and LangChain, I iteratively refined the dataset with client-specific SQL patterns and synonyms for domain adaptation. The LangGraph multi-agent system used careful prompt engineering: an interpreter agent used AI Search (RAG) to fetch relevant examples and context, the main agent generated SQL guided by them, and a validation agent tested queries on a sandbox to catch errors. I ran daily syncs with the client's innovation team and maintained code in Git.
The fine-tuned model reached ~85% accuracy on known query types (up from ~60% on the base model). The multi-agent design enabled dynamic source selection—routing pipeline questions to Databricks and ledger questions to the Fabric Warehouse. RAG let the system add consulting-style explanations by quoting definitions and policies. User acceptance tests with analysts were positive, confirming the natural-language interface saved time over manual SQL.
Hybrid approaches (prompt templates + RAG) can complement or even replace full fine-tuning. SQL security and performance discipline—timeouts and resource limits—is essential to prevent runaway queries. Transparency about how the AI forms SQL and sources context built user trust.