SQL Server 2025 Built-in Chunking and Vector Support AI Functions, the Brutal Truth. From SQL to AI series, Part 1

Before we dive into today's topic, if you missed my previous post you can take a look at Check Parallelism Issues in 45 Seconds. From Symptoms to Root Clause. The "45 Seconds DBA Series" – What Real DBAs Check First | Part 10 🥇". 👉 If you found this deep-dive helpful, feel free to check out the ads—your support helps me keep creating high-quality SQL Server content for the community.

SQL Server 2025 Has AI Functions…
But Not Where You Think

What if I told you that SQL Server can now generate embeddings and chunk text… directly in T-SQL?

👉 It’s true. 👉 But also misleading.

⚡ TL;DR (Reality Check)

✔️ AI functions exist (AI_GENERATE_CHUNKS / AI_GENERATE_EMBEDDINGS)

✔️ They are real T-SQL functions

❗ But they work ONLY in cloud-enabled environments

❗ Not fully available in standard SQL Server on-prem

✔️ SQL Server is becoming an AI orchestrator, not an AI engine

🧠 The Illusion

You may have seen code like this:

SELECT 
    C.chunk_id,
    C.chunk_text,
    AI_GENERATE_EMBEDDINGS('text-embedding-3-small', C.chunk_text)
FROM AI_GENERATE_CHUNKS(@doc, 'FIXED', 500, 50) C;

Looks like pure T-SQL magic, right?

No Python. No pipelines. No external services.

But here’s the truth:

This does NOT run inside SQL Server the way you think.

⚠️ Where This Actually Works

These features are real — but they are cloud-first capabilities.

✔️ Supported:

• Azure SQL Managed Instance (Always-up-to-date)
• Azure SQL Database (preview / rollout)
• Microsoft Fabric SQL

❌ Not fully supported:

• SQL Server 2025 on-prem (standard installations)

Translation: this is the future of SQL… but only if you're in the cloud.

🔍 What Is Really Happening

Let’s break it down:

• AI_GENERATE_CHUNKS → splits text into chunks inside SQL
• AI_GENERATE_EMBEDDINGS → calls an external AI model

Yes — external.

The embeddings are NOT generated by SQL Server.

They come from:

• Azure OpenAI
• or other REST-based AI models

SQL Server is simply orchestrating the call.

🧪 The Real Setup

CREATE EXTERNAL MODEL MyEmbeddingModel
WITH (
    LOCATION = 'https://your-openai-endpoint',
    API_KEY = 'your-key'
);

Without an external model → embeddings will NOT work.

🚀 RAG Inside SQL (Almost)

This enables something powerful:

You can orchestrate a full RAG pipeline in T-SQL.

But:

• Data stays in SQL Server
• AI runs outside
• SQL coordinates everything

🧠 The Real Insight

SQL Server is NOT becoming an AI engine.

It is becoming the control layer for AI.

Data lives in the database. Intelligence lives in external models.

🚀 My REAL Strategy

• ⚡ Think Hybrid: Database + AI services → not one or the other
• ⚡ Optimize Chunking: 500 tokens + overlap is a solid baseline
• ⚡ Measure Latency: every embedding call is a network call
• ⚡ Watch Costs: embeddings + storage scale fast

💣 Final Thought

AI is already inside T-SQL.

But only if you're running the right environment.

And that environment… is the cloud or in the next version of SQL Server...

📢 Support the Blog: Did you find this deep-dive helpful? The ads you see here are selected to reflect your interests. If a partner's offer catches your eye, give it a look! Your engagement helps me continue publishing premium SQL Server content for the community.

Biondi Luca @2026 - Sharing over 25 years of Gained Knowledge for Passion. Share if you like my posts!