Mirrors/SqlDatabaseVectorSearch
1
0
Fork 0
mirror of https://github.com/marcominerva/SqlDatabaseVectorSearch.git synced 2026-08-04 09:48:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #17 from marcominerva/agent-framework

Browse Source 
Refactor to Microsoft.Agents.AI and improve embedding workflows
This commit is contained in:
Marco Minerva
2026-07-31 16:39:34 +02:00
committed by GitHub
parent 4dcd7c9fdc d910205339
commit f0c92b32f2
39 changed files with 1434 additions and 1006 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.editorconfig
+3
View File
@@ -124,6 +124,7 @@ csharp_style_prefer_null_check_over_type_check = true:suggestion
# Modifier preferences
csharp_prefer_static_local_function = true:suggestion
csharp_prefer_static_anonymous_function = true:suggestion
csharp_preferred_modifier_order = public,private,protected,internal,static,extern,new,virtual,abstract,sealed,override,readonly,unsafe,volatile,async:silent
# Code-block preferences
@@ -139,9 +140,11 @@ csharp_prefer_system_threading_lock = true:suggestion
csharp_prefer_simple_default_expression = true:suggestion
csharp_style_deconstructed_variable_declaration = false:suggestion
csharp_style_inlined_variable_declaration = true:suggestion
csharp_style_prefer_implicitly_typed_lambda_expression = true:suggestion
csharp_style_pattern_local_over_anonymous_function = true:suggestion
csharp_style_prefer_index_operator = true:suggestion
csharp_style_prefer_range_operator = true:suggestion
csharp_style_prefer_unbound_generic_type_in_nameof = true:suggestion
csharp_style_throw_expression = true:suggestion
csharp_style_unused_value_assignment_preference = discard_variable:none
csharp_style_unused_value_expression_statement_preference = discard_variable:none
README.md
+72 -218
View File
@@ -1,10 +1,10 @@
# SQL Database Vector Search Sample
# SQL Database Vector Search
[![.NET 10](https://img.shields.io/badge/.NET-10-blue)](https://dotnet.microsoft.com/en-us/download/dotnet/10.0)
[![Minimal API](https://img.shields.io/badge/Minimal%20API-Available-green)](https://dotnet.microsoft.com/apps/aspnet/apis)
[![Blazor](https://img.shields.io/badge/Blazor-WebApp-purple)](https://dotnet.microsoft.com/apps/aspnet/web-apps/blazor)
A Blazor Web App and Minimal API for performing RAG (Retrieval Augmented Generation) and vector search using the native VECTOR type in Azure SQL Database and Azure OpenAI.
A Blazor Web App and Minimal API for performing RAG (Retrieval Augmented Generation) and vector search using the native VECTOR type in Azure SQL Database or SQL Server 2025, Azure OpenAI, and [Microsoft Agent Framework](https://github.com/microsoft/agent-framework).
## Table of Contents
- [Overview](#overview)
@@ -23,11 +23,13 @@ A Blazor Web App and Minimal API for performing RAG (Retrieval Augmented Generat
## Overview
This application allows you to:
- Load documents (PDF, DOCX, TXT, MD)
- Generate embeddings and save them as vectors in Azure SQL Database
- Perform semantic search and RAG using Azure OpenAI
- Generate embeddings and save them as vectors in Azure SQL Database or SQL Server 2025
- Perform semantic search and RAG using Azure OpenAI and Microsoft Agent Framework agents
- Interact via a Blazor Web App or programmatically via Minimal API
Embeddings and chat completion are powered by [Semantic Kernel](https://github.com/microsoft/semantic-kernel).
The native `VECTOR` type is available in both Azure SQL Database and SQL Server 2025, so no external vector store is required.
Embeddings and chat completion are orchestrated with [Microsoft Agent Framework](https://github.com/microsoft/agent-framework). The application uses an embedding workflow to import documents, a reformulation agent to rewrite follow-up questions with conversation context, and a RAG agent connected to a SQL vector-search context provider.
## Screenshots
@@ -39,17 +41,24 @@ Embeddings and chat completion are powered by [Semantic Kernel](https://github.c
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/en-us/download/dotnet/10.0)
- [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/single-database-create-quickstart)
- One of the following, both of which support the native `VECTOR` type:
- [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/single-database-create-quickstart)
- [SQL Server 2025](https://learn.microsoft.com/en-us/sql/relational-databases/vectors/vectors-sql-server) or later
- Azure OpenAI resource and API keys
## Project Structure
- `SqlDatabaseVectorSearch/` - Main Blazor Web App and API
- `Components/` - Blazor UI components
- `ContentDecoders/` - Decoders that extract text from PDF, DOCX, TXT and MD files
- `Data/` - EF Core context, migrations, and entities
- `Endpoints/` - Minimal API endpoints
- `Extensions/` - Extension methods and helpers
- `Models/` - Request and response models
- `Services/` - Business logic and integration services
- `TextChunkers/` - Text splitting utilities
- `Settings/` - Configuration classes
- `TextChunkers/` - Text splitting utilities
- `Validations/` - Request validators
- `Workflows/` - Microsoft Agent Framework workflow executors for document import and embedding generation
## Setup
@@ -60,8 +69,8 @@ Embeddings and chat completion are powered by [Semantic Kernel](https://github.c
```
2. Configure the database and OpenAI settings
- Edit `SqlDatabaseVectorSearch/appsettings.json` and set your Azure SQL connection string and OpenAI settings.
- **Important**: The `ModelId` values for both `ChatCompletion` and `Embedding` are used for token counting via `Microsoft.ML.Tokenizers`. These values must be valid model identifiers supported by the tokenizer library (e.g., `gpt-4o`, `gpt-4`, `gpt-3.5-turbo`, `text-embedding-3-small`, `text-embedding-3-large`, `text-embedding-ada-002`). The `ModelId` may differ from the actual deployment name you're using in Azure OpenAI. For example, for gpt-4.1 and gpt-5 models set the `ModelId` to `gpt-4o` for proper token counting.
- Edit `SqlDatabaseVectorSearch/appsettings.json` and set your connection string (Azure SQL Database or SQL Server 2025) and OpenAI settings.
- **Important**: The `ModelId` values for both `ChatCompletion` and `Embedding` are used only for token counting via `Microsoft.ML.Tokenizers`, while the actual calls to the service use the `Deployment` values. `ModelId` must therefore be a model name recognized by the tokenizer library (e.g., `gpt-5`, `gpt-4.1`, `gpt-4o`, `gpt-4`, `gpt-3.5-turbo`, `text-embedding-3-small`, `text-embedding-3-large`, `text-embedding-ada-002`), which is typically different from the deployment name you have chosen in Azure OpenAI. If a model is not recognized, `TiktokenTokenizer.CreateForModel` throws at startup: in that case, fall back to the closest supported model that shares the same encoding (for example `gpt-4o` for newer GPT models).
- If using embedding models with shortening (e.g., `text-embedding-3-small` or `text-embedding-3-large`), set the `Dimensions` property accordingly. For `text-embedding-3-large`, you must specify a value <= 1998.
- If you change the VECTOR size, update both the [ApplicationDbContext](SqlDatabaseVectorSearch/Data/ApplicationDbContext.cs) and the [Initial Migration](SqlDatabaseVectorSearch/Data/Migrations/00000000000000_Initial.cs).
@@ -71,28 +80,38 @@ Embeddings and chat completion are powered by [Semantic Kernel](https://github.c
dotnet run --project SqlDatabaseVectorSearch/SqlDatabaseVectorSearch.csproj
```
5. Access the Web App
- Navigate to `https://localhost:5001` (or the port shown in the console)
4. Access the Web App
- Navigate to `https://localhost:7025` (or the port shown in the console)
## Supported features
- **Conversation History with Question Reformulation**: This feature allows users to view the history of their conversations, including the ability to reformulate questions for better clarity and understanding. This ensures that users can track their interactions and refine their queries as needed.
- **Information about Token Usage**: Users can access detailed information about token usage, which helps in understanding the consumption of tokens during interactions. This feature provides transparency and helps users manage their token usage effectively.
- **Response Streaming**: This feature enables real-time streaming of responses, allowing users to receive information as it is being processed. This ensures a seamless and efficient flow of information, enhancing the overall user experience.
- **Citations**: The application provides citations for the sources used to justify each answer. This allows users to verify the information and understand the origin of the content provided by the system.
- **Microsoft Agent Framework orchestration**: Document import is implemented as a workflow, while question reformulation and RAG are implemented as agents.
- **Conversation history with question reformulation**: The reformulation agent rewrites each question using the conversation context before vector search is performed.
- **SQL vector-search context provider**: The RAG agent receives relevant chunks from Azure SQL Database or SQL Server 2025 through a `TextSearchProvider` backed by native VECTOR search.
- **Information about token usage**: The Blazor chat page and API responses expose token usage for reformulation and final answer generation.
- **Response streaming**: The Blazor chat page and the `/api/ask-streaming` endpoint stream answers, appending tokens as they arrive.
- **Markdown source citations**: Citations are included directly in the generated Markdown answer as a localized sources section with source name, page number when available, and a short supporting excerpt.
## How to Use
- **Web App**: Use the Blazor interface to upload documents, search, and chat with RAG.
- **Web App**: Use the Blazor interface to manage documents and chat with your indexed content. The chat page streams answers, shows token usage, supports conversation reset, and renders source citations as part of the Markdown answer.
- **API**: Import documents via `POST /api/documents` and ask questions via `POST /api/ask` or `POST /api/ask-streaming`.
### How it works
1. Documents are uploaded through the API and processed by the `EmbeddingWorkflow`.
2. The workflow converts the uploaded file into text, chunks it, generates embeddings, and stores documents, chunks, and VECTOR embeddings in Azure SQL Database or SQL Server 2025.
3. When a question is asked, the `ReformulationAgent` can rewrite it using the current conversation context.
4. The `RagAgent` receives relevant SQL vector-search results through a `TextSearchProvider` and answers using only the provided context.
5. Sources are not returned as a separate JSON collection. They are formatted directly in the Markdown answer.
#### Example API Request
```
POST /api/ask
Content-Type: application/json
{
"conversationId": "3d0bd178-499d-433a-b2bc-c35e488d9e2c"
"conversationId": "3d0bd178-499d-433a-b2bc-c35e488d9e2c",
"text": "Why is Mars called the red planet?"
}
```
@@ -101,227 +120,62 @@ Content-Type: application/json
```json
{
"conversationId": "3d0bd178-499d-433a-b2bc-c35e488d9e2c",
"originalQuestion": "why is mars called the red planet?",
"reformulatedQuestion": "Why is the planet Mars called the red planet?",
"answer": "Mars is called the Red Planet because its surface has an orange-red color due to being covered in iron(III) oxide dust, also known as rust. This iron oxide gives Mars its distinctive reddish appearance when observed from Earth and is the origin of its well-known nickname",
"streamState": "End",
"answer": "Mars is called the Red Planet because its surface has an orange-red color caused by iron oxide dust.\n\n*Sources*\n1. **Mars.pdf**, page 1: *surface of Mars is orange-red because it is covered in iron oxide dust*",
"streamState": null,
"tokenUsage": {
"reformulation": {
"promptTokens": 812,
"completionTokens": 11,
"totalTokens": 823
"inputTokenCount": 812,
"outputTokenCount": 11,
"totalTokenCount": 823
},
"embeddingTokenCount": 10,
"question": {
"promptTokens": 31708,
"completionTokens": 227,
"totalTokens": 31935
"inputTokenCount": 31708,
"outputTokenCount": 227,
"totalTokenCount": 31935
}
},
"citations": [
{
"documentId": "b1870ad7-4685-42a3-576a-08ddb01159d5",
"chunkId": "749aba1e-0db5-4033-cfa6-08ddb0115da3",
"fileName": "Mars.pdf",
"quote": "surface of Mars is orange-red because it is covered in iron(III) oxide",
"pageNumber": 1,
"indexOnPage": 0
},
{
"documentId": "b1870ad7-4685-42a3-576a-08ddb01159d5",
"chunkId": "215e7197-513f-4fbe-cfa8-08ddb0115da3",
"fileName": "Mars.pdf",
"quote": "Martian surface is caused by ferric oxide, or rust",
"pageNumber": 3,
"indexOnPage": 0
}
]
}
}
```
### How response streaming works
When using the `/api/ask-streaming` endpoint, answers will be streamed as with the typical response from OpenAI. The format of the response is as follows:
When using the `/api/ask-streaming` endpoint, answers are streamed as [Server-Sent Events](https://developer.mozilla.org/docs/Web/API/Server-sent_events). Each event has a name matching the `streamState` value and a JSON `Response` payload in the `data` field. The format is as follows:
```json
[
{
"originalQuestion": "why is mars called the red planet?",
"reformulatedQuestion": "Why is the planet Mars known as the red planet?",
"answer": null,
"streamState": "Start",
"tokenUsage": {
"reformulation": {
"promptTokens": 541,
"completionTokens": 12,
"totalTokens": 553
},
"embeddingTokenCount": 11,
"question": null
},
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": "Mars",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " is",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " known",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " as",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " the",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " red",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " planet",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " because",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " its",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " surface",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " is",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " covered",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " in",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " iron",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
/// ...
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": null,
"streamState": "End",
"tokenUsage": {
"reformulation": null,
"embeddingTokenCount": null,
"question": {
"promptTokens": 30949,
"completionTokens": 221,
"totalTokens": 31170
}
},
"citations": [
{
"documentId": "b1870ad7-4685-42a3-576a-08ddb01159d5",
"chunkId": "749aba1e-0db5-4033-cfa6-08ddb0115da3",
"fileName": "Mars.pdf",
"quote": "surface of Mars is orange-red",
"pageNumber": 1,
"indexOnPage": 0
},
{
"documentId": "b1870ad7-4685-42a3-576a-08ddb01159d5",
"chunkId": "215e7197-513f-4fbe-cfa8-08ddb0115da3",
"fileName": "Mars.pdf",
"quote": "red-orange appearance of the Martian surface is caused by ferric oxide, or rust",
"pageNumber": 3,
"indexOnPage": 0
}
]
}
]
```text
event: Start
data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":"why is mars called the red planet?","reformulatedQuestion":"Why is the planet Mars known as the red planet?","answer":null,"streamState":"Start","tokenUsage":{"reformulation":{"inputTokenCount":541,"outputTokenCount":12,"totalTokenCount":553,"cachedInputTokenCount":0,"reasoningTokenCount":0,"inputAudioTokenCount":null,"inputTextTokenCount":null,"outputAudioTokenCount":null,"outputTextTokenCount":null,"additionalCounts":null},"question":null}}
event: Delta
data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":null,"reformulatedQuestion":null,"answer":"Mars","streamState":"Delta","tokenUsage":null}
event: Delta
data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":null,"reformulatedQuestion":null,"answer":" is known as the red planet because its surface is rich in iron oxide dust.\n\n","streamState":"Delta","tokenUsage":null}
event: Delta
data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":null,"reformulatedQuestion":null,"answer":"Sources\n1. **Mars.pdf**, page 1: *surface of Mars is orange-red because it is covered in iron oxide dust*","streamState":"Delta","tokenUsage":null}
event: End
data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":null,"reformulatedQuestion":null,"answer":null,"streamState":"End","tokenUsage":{"reformulation":null,"question":{"inputTokenCount":30949,"outputTokenCount":221,"totalTokenCount":31170,"cachedInputTokenCount":3840,"reasoningTokenCount":0,"inputAudioTokenCount":null,"inputTextTokenCount":null,"outputAudioTokenCount":null,"outputTextTokenCount":null,"additionalCounts":null}}}
```
- The first piece of the response has the following characteristics:
- The first event has the following characteristics:
- The SSE event name is `Start`.
- The *streamState* property is set to `Start`.
- It contains the question and its reformulation (if not requested, *reformulatedQuestion* will be equal to *originalQuestion*).
- The *tokenUsage* section holds information about tokens used for reformulation (if done) and for the embedding of the question.
- Then, there are as many elements for the actual answer as necessary:
- Each one contains a token.
- The *streamState* property is set to `Append`.
- *originalQuestion*, *reformulatedQuestion*, *tokenUsage* and *citations* are always `null`.
- The stream ends when an element with *streamState* equals `End` is received. This element contains token usage information for the question and the whole answer, and the list of citations.
- The *tokenUsage* section holds information about tokens used for reformulation, if done.
- Then, there are as many `Delta` events as necessary for the actual answer:
- Each event contains a token or chunk of generated text in the *answer* property.
- The *streamState* property is set to `Delta`.
- *originalQuestion*, *reformulatedQuestion* and *tokenUsage* are always `null`.
- The stream ends when an `End` event is received. This event contains token usage information for the final answer.
- Sources are included in the Markdown answer text.
## Limitations & FAQ
- **Database**: Azure SQL Database or SQL Server 2025 (or later). Both provide the native `VECTOR` type used by this sample.
- **VECTOR column size**: Maximum allowed is 1998. For `text-embedding-3-large`, set `Dimensions` <= 1998.
- **Supported file types**: PDF, DOCX, TXT, MD.
- **Known Issues**: See [Issues](https://github.com/marcominerva/SqlDatabaseVectorSearch/issues)
SqlDatabaseVectorSearch/Components/App.razor
+2 -2
View File
@@ -14,11 +14,11 @@
<link rel="stylesheet" href="@Assets["SqlDatabaseVectorSearch.styles.css"]" />
<ImportMap />
<link rel="icon" type="image/png" href="favicon.png" />
<HeadOutlet @rendermode="InteractiveServer" />
<HeadOutlet @rendermode="new InteractiveServerRenderMode(prerender: false)" />
</head>
<body>
<Routes @rendermode="InteractiveServer" />
<Routes @rendermode="new InteractiveServerRenderMode(prerender: false)" />
<ReconnectModal />
<script src="_framework/blazor.web.js"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js" integrity="sha384-YvpcrYf0tY3lHB60NNkmXc5s9fDVZLESaAA55NDzOxhy9GkcIdslK1eN7N6jIeHz" crossorigin="anonymous"></script>
SqlDatabaseVectorSearch/Components/Layout/MainLayout.razor
+4 -4
View File
@@ -5,10 +5,10 @@
<BlazorBootstrapLayout StickyHeader="true">
<HeaderSection>
<a href="/swagger" target="_blank" class="text-decoration-none" title="OpenAPI documentation">
<Icon Name="IconName.FileTypeJson" Class="ps-3 ps-lg-2" Size="IconSize.x2" Color="IconColor.Muted"></Icon>
<Icon Name="IconName.FileEarmarkCodeFill" Class="ps-3 ps-lg-2" Size="IconSize.x2" Color="IconColor.Info"></Icon>
</a>
<a href="https://github.com/marcominerva/SqlDatabaseVectorSearch" target="_blank" class="text-decoration-none" title="View on GitHub">
<Icon Name="IconName.Github" Class="ps-4 ps-lg-4" Size="IconSize.x2" Color="IconColor.Muted"></Icon>
<Icon Name="IconName.Github" Class="ps-4 ps-lg-4" Size="IconSize.x2" Color="IconColor.Dark"></Icon>
</a>
</HeaderSection>
@@ -44,8 +44,8 @@
{
navItems = [
new() { Id = "1", Href = "/", IconName = IconName.HouseDoorFill, Text = "Home", Match = NavLinkMatch.All},
new() { Id = "2", Href= "/documents", IconName = IconName.FileText, Text = "Documents" },
new() { Id = "3", Href = "/ask", IconName = IconName.ChatDots, Text = "Ask"}
new() { Id = "2", Href= "/documents", IconName = IconName.FileEarmarkTextFill, Text = "Documents" },
new() { Id = "3", Href = "/ask", IconName = IconName.ChatDotsFill, Text = "Ask"}
];
return navItems;
SqlDatabaseVectorSearch/Components/Layout/ReconnectModal.razor
+5 -5
View File
@@ -10,7 +10,7 @@
Rejoining the server...
</p>
<p class="components-reconnect-repeated-attempt-visible">
Rejoin failed... Trying again in <span id="components-seconds-to-next-attempt"></span> seconds.
Rejoin failed... trying again in <span id="components-seconds-to-next-attempt"></span> seconds.
</p>
<p class="components-reconnect-failed-visible">
Failed to rejoin.<br />Please retry or reload the page.
@@ -21,11 +21,11 @@
<p class="components-pause-visible">
The session has been paused by the server.
</p>
<button id="components-resume-button" class="components-pause-visible">
<p class="components-resume-failed-visible">
Failed to resume the session.<br />Please retry or reload the page.
</p>
<button id="components-resume-button" class="components-pause-visible components-resume-failed-visible">
Resume
</button>
<p class="components-resume-failed-visible">
Failed to resume the session.<br />Please reload the page.
</p>
</div>
</dialog>
SqlDatabaseVectorSearch/Components/Pages/Ask.razor
+82 -115
View File
@@ -1,7 +1,7 @@
@page "/ask"
@using System.Text.RegularExpressions
@using Microsoft.Extensions.AI
@inject IServiceProvider ServiceProvider
@inject IServiceScopeFactory ServiceScopeFactory
@inject IJSRuntime JSRuntime
<PageTitle>Chat with your data</PageTitle>
@@ -9,6 +9,15 @@
<div class="card mx-auto mt-2">
<div class="card-body">
@if (messages.Count == 0)
{
<div class="h-100 d-flex flex-column justify-content-center align-items-center text-body-secondary">
<Icon Name="IconName.ChatSquareQuoteFill" Color="IconColor.Primary" Size="IconSize.x4" />
<p class="mt-3 mb-1 fw-semibold">Chat with your documents</p>
<p class="small mb-0">Ask a question about the documents you have uploaded. Press the up arrow key to recall your previous question.</p>
</div>
}
@foreach (var message in messages)
{
if (message.Role == "user")
@@ -37,10 +46,10 @@
@if (message.Text is null)
{
<div class="card card-text d-inline-block p-3 px-3 m-1">
<div class="progress-chat" role="progressbar" aria-label="I'm thinking" aria-valuenow="0" aria-valuemin="0" aria-valuemax="100">
<div class="progress-bar-chat">
<div class="progress-bar-indeterminate"></div>
</div>
<div class="typing-indicator" role="status" aria-label="The assistant is typing">
<span class="typing-dot"></span>
<span class="typing-dot"></span>
<span class="typing-dot"></span>
</div>
</div>
}
@@ -65,13 +74,13 @@
<div class="d-flex justify-content-between">
<div class="text-start bg-transparent mt-3">
<Tooltip Title="@message.TokenUsage" IsHtml="true" Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
<Icon Class="d-flex text-body-secondary" Name="IconName.InfoCircle"></Icon>
<Icon Class="d-flex" Name="IconName.CashCoin"></Icon>
</Tooltip>
</div>
<div class="text-end bg-transparent">
<Tooltip Title="@toolTipText" Color="TooltipColor.Dark" Placement="TooltipPlacement.Bottom">
<Button Type="ButtonType.Button" Outline="false" @onclick="@(async () => await CopyToClipboardAsync(message.Text))">
@if (showCopyConfirmation)
<div class="text-end bg-transparent copy-button">
<Tooltip Title="@(copiedMessage == message ? "Copied!" : "Copy to clipboard")" Color="TooltipColor.Dark" Placement="TooltipPlacement.Bottom">
<Button Type="ButtonType.Button" Outline="false" @onclick="@(async () => await CopyToClipboardAsync(message))">
@if (copiedMessage == message)
{
<Icon Name="IconName.Check" Class="text-success" />
}
@@ -83,23 +92,6 @@
</Tooltip>
</div>
</div>
@if (message.Citations is not null && message.Citations.Count() > 0)
{
<div class="mt-3 d-flex flex-wrap">
@foreach (var citation in message.Citations)
{
<div class="border rounded p-2 me-2 mb-2 citation-box small">
<div>
<strong>@citation.FileName</strong> @if (citation.PageNumber.GetValueOrDefault() > 0)
{
<span class="ms-2">pag. @citation.PageNumber</span>
}
</div>
<div class="text-secondary small mt-1">@citation.Quote</div>
</div>
}
</div>
}
}
</div>
}
@@ -115,18 +107,20 @@
<div class="card-footer bg-white w-100 bottom-0 m-0 p-1">
<div class="input-group">
<span class="input-group-text bg-transparent border-0">
<Tooltip Title="Messages aren't stored in any way on either the client or the server." Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
<Icon Class="d-flex text-body-secondary" Name="IconName.InfoCircle"></Icon>
<Tooltip Title="Messages aren't stored in any way, either on the client or on the server." Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
<Icon Class="d-flex" Color="IconColor.Success" Name="IconName.ShieldLockFill"></Icon>
</Tooltip>
</span>
<input @ref="askInput" type="text" @bind="@question" @bind:event="oninput" placeholder="Ask me anything..." class="form-control border-0" maxlength="2000" @onkeydown="HandleKeyDown" />
<input @ref="askInput" type="text" @bind="@question" @bind:event="oninput" placeholder="Ask a question about your documents..." class="form-control border-0" maxlength="2000" @onkeydown="HandleKeyDown" />
<div class="input-group-text bg-transparent border-0">
<Button Type="ButtonType.Submit" @ref="askButton" Color="ButtonColor.Primary" Disabled="@(isAsking || string.IsNullOrWhiteSpace(question))" @onclick="AskQuestion">
<Button Type="ButtonType.Submit" Color="ButtonColor.Primary" Disabled="@(isAsking || string.IsNullOrWhiteSpace(question))" @onclick="AskQuestion">
<Icon Name="IconName.Send" />
</Button>
<Button Type="ButtonType.Reset" @ref="resetButton" Class="ms-2" Color="ButtonColor.Secondary" Disabled="@isAsking" @onclick="Reset">
<Icon CustomIconName="bi bi-x-lg" />
</Button>
<Tooltip Title="Start a new conversation" Color="TooltipColor.Secondary" Placement="TooltipPlacement.Bottom">
<Button Type="ButtonType.Reset" Class="ms-2" Color="ButtonColor.Secondary" Disabled="@isAsking" @onclick="Reset">
<Icon CustomIconName="bi bi-x-lg" />
</Button>
</Tooltip>
</div>
</div>
</div>
@@ -134,8 +128,6 @@
@code
{
private Button askButton = default!;
private Button resetButton = default!;
private ElementReference askInput = default!;
private ElementReference chat = default!;
@@ -143,13 +135,18 @@
private string? question;
private Guid conversationId = Guid.NewGuid();
private bool isAsking = false;
private bool isAsking;
private bool showCopyConfirmation = false;
private string toolTipText = "Copy to Clipboard";
private Message? copiedMessage;
protected override async Task OnAfterRenderAsync(bool firstRender)
{
// Scrolling here guarantees the browser DOM already contains the latest messages.
if (messages.Count > 0)
{
await EnsureMessageIsVisibleAsync();
}
if (!firstRender)
{
return;
@@ -187,62 +184,47 @@
messages.Add(assistantMessage);
question = null;
await InvokeAsync(StateHasChanged);
await Task.Yield();
await EnsureMessageIsVisibleAsync();
try
{
await using var scope = ServiceProvider.CreateAsyncScope();
await using var scope = ServiceScopeFactory.CreateAsyncScope();
var vectorSearchService = scope.ServiceProvider.GetRequiredService<VectorSearchService>();
var response = vectorSearchService.AskStreamingAsync(userQuestion);
await foreach (var delta in response)
await foreach (var update in response)
{
if (delta.StreamState == StreamState.Start)
if (update.StreamState == StreamState.Start)
{
userMessage.Text = delta.ReformulatedQuestion;
assistantMessage.TokenUsage = FormatTokenUsage(delta.TokenUsage);
userMessage.Text = update.ReformulatedQuestion;
assistantMessage.TokenUsage = FormatTokenUsage(update.TokenUsage);
assistantMessage.Status = MessageStatus.Streaming;
}
else if (delta.StreamState == StreamState.Append)
else if (update.StreamState == StreamState.Delta)
{
// Adds tokens to the assistant message as they are received.
assistantMessage.Text += delta.Answer;
assistantMessage.Text += update.Answer;
}
else if (delta.StreamState == StreamState.End)
else if (update.StreamState == StreamState.End)
{
// Get citations from the response.
assistantMessage.Citations = delta.Citations?.Select(c => new Citation
{
DocumentId = c.DocumentId,
ChunkId = c.ChunkId,
FileName = c.FileName,
Quote = c.Quote,
PageNumber = c.PageNumber,
IndexOnPage = c.IndexOnPage
});
assistantMessage.Status = MessageStatus.Completed;
assistantMessage.TokenUsage += FormatTokenUsage(delta.TokenUsage);
assistantMessage.TokenUsage += FormatTokenUsage(update.TokenUsage);
}
await Task.Yield();
StateHasChanged();
await InvokeAsync(StateHasChanged);
await EnsureMessageIsVisibleAsync();
}
}
catch (Exception ex)
{
assistantMessage.Text = $"There was an error while processing the question: {ex.Message}";
assistantMessage.Text = $"There was an error while processing your question: {ex.Message}";
assistantMessage.Status = MessageStatus.Completed;
}
finally
{
await EnsureMessageIsVisibleAsync();
isAsking = false;
await InvokeAsync(StateHasChanged);
}
}
@@ -250,25 +232,29 @@
{
question = null;
conversationId = Guid.NewGuid();
copiedMessage = null;
messages.Clear();
}
private async Task CopyToClipboardAsync(string text)
private async Task CopyToClipboardAsync(Message message)
{
if (text is null)
if (string.IsNullOrEmpty(message.Text))
{
return;
}
await JSRuntime.InvokeVoidAsync("navigator.clipboard.writeText", text);
await JSRuntime.InvokeVoidAsync("navigator.clipboard.writeText", message.Text);
showCopyConfirmation = true;
toolTipText = "Copied!";
StateHasChanged();
copiedMessage = message;
await InvokeAsync(StateHasChanged);
await Task.Delay(3000); // Shows the checkmark for 3 seconds
toolTipText = "Copy to Clipboard";
showCopyConfirmation = false;
StateHasChanged();
if (copiedMessage == message)
{
copiedMessage = null;
await InvokeAsync(StateHasChanged);
}
}
private static string FormatTokenUsage(TokenUsageResponse? tokenUsageResponse)
@@ -278,30 +264,29 @@
return string.Empty;
}
var reformulation = tokenUsageResponse.Reformulation is not null
? $"<p><strong>Reformulation:</strong><br />{FormatTokenUsageDetails(tokenUsageResponse.Reformulation)}</p>"
: string.Empty;
(string Label, UsageDetails? Usage)[] sections =
[
("Reformulation", tokenUsageResponse.Reformulation),
("Question", tokenUsageResponse.Question)
];
var embeddingTokenCount = tokenUsageResponse.EmbeddingTokenCount.HasValue
? $"<p><strong>Embedding Token Count:</strong> {tokenUsageResponse.EmbeddingTokenCount}</p>"
: string.Empty;
return string.Concat(sections
.Where(section => section.Usage is not null)
.Select(section => $"<p><strong>{section.Label}:</strong><br />{FormatTokenUsageDetails(section.Usage!)}</p>"));
var question = tokenUsageResponse.Question is not null
? $"<p><strong>Question:</strong><br />{FormatTokenUsageDetails(tokenUsageResponse.Question)}</p>"
: string.Empty;
return $"{reformulation}{embeddingTokenCount}{question}";
static string FormatTokenUsageDetails(TokenUsage? tokenUsage)
static string FormatTokenUsageDetails(UsageDetails tokenUsage)
{
if (tokenUsage is null)
{
return string.Empty;
}
// Reasoning tokens are part of the output tokens, so they're shown as an indented detail of that value.
var outputTokens = tokenUsage.ReasoningTokenCount is > 0
? $"Output tokens: {tokenUsage.OutputTokenCount}<br /><span class=\"ms-3 small\">&bull; Reasoning: {tokenUsage.ReasoningTokenCount}</span>"
: $"Output tokens: {tokenUsage.OutputTokenCount}";
return $"Prompt tokens: {tokenUsage.PromptTokens}<br />" +
$"Completion tokens: {tokenUsage.CompletionTokens}<br />" +
$"Total tokens: {tokenUsage.TotalTokens}";
return string.Join("<br />",
[
$"Input tokens: {tokenUsage.InputTokenCount}",
outputTokens,
$"Total tokens: {tokenUsage.TotalTokenCount}"
]);
}
}
@@ -326,23 +311,5 @@
public MessageStatus Status { get; set; } = MessageStatus.New;
public string? TokenUsage { get; set; }
// List of citations extracted from the answer.
public IEnumerable<Citation>? Citations { get; set; }
}
public class Citation
{
public Guid DocumentId { get; set; }
public Guid ChunkId { get; set; }
public string FileName { get; set; } = null!;
public string Quote { get; set; } = null!;
public int? PageNumber { get; set; }
public int IndexOnPage { get; set; }
}
}
SqlDatabaseVectorSearch/Components/Pages/Ask.razor.css
+62 -21
View File
@@ -43,37 +43,55 @@ input[type="checkbox"] + label {
border-radius: 8px;
}
.progress-chat {
width: 200px;
height: 4px;
.typing-indicator {
display: flex;
align-items: center;
gap: 5px;
height: 8px;
}
.progress-bar-chat {
height: 4px;
background-color: rgba(5, 114, 206, 0.2);
width: 100%;
overflow: hidden;
}
.progress-bar-indeterminate {
width: 100%;
height: 100%;
.typing-dot {
width: 8px;
height: 8px;
border-radius: 50%;
background-color: rgb(5, 114, 206);
animation: indeterminate-animation 1s infinite linear;
transform-origin: 0% 50%;
animation: typing-bounce 1.2s infinite ease-in-out both;
}
@keyframes indeterminate-animation {
0% {
transform: translateX(0) scaleX(0);
.typing-dot:nth-child(1) {
animation-delay: -0.24s;
}
.typing-dot:nth-child(2) {
animation-delay: -0.12s;
}
@keyframes typing-bounce {
0%, 80%, 100% {
transform: translateY(0);
opacity: .4;
}
40% {
transform: translateX(0) scaleX(0.4);
transform: translateY(-5px);
opacity: 1;
}
}
@media (prefers-reduced-motion: reduce) {
.typing-dot {
animation-name: typing-fade;
animation-duration: 2s;
}
100% {
transform: translateX(100%) scaleX(0.5);
@keyframes typing-fade {
0%, 80%, 100% {
opacity: .4;
}
40% {
opacity: 1;
}
}
}
@@ -101,6 +119,29 @@ input[type="checkbox"] + label {
z-index: 10;
}
/* The copy button only shows an icon, so the default button chrome is removed and just the pointer cursor is kept. */
.copy-button ::deep .btn {
background-color: transparent;
border-color: transparent;
box-shadow: none;
color: var(--bs-secondary-color);
}
.copy-button ::deep .btn:hover {
background-color: transparent;
border-color: transparent;
color: var(--bs-body-color);
}
.copy-button ::deep .btn:focus,
.copy-button ::deep .btn:focus-visible,
.copy-button ::deep .btn:active {
background-color: transparent;
border-color: transparent;
box-shadow: none;
outline: none;
}
.btn-clipboard {
line-height: 1;
color: var(--bs-body-color);
SqlDatabaseVectorSearch/Components/Pages/Documents.razor
+142 -86
View File
@@ -1,50 +1,54 @@
@page "/documents"
@using MimeMapping
@inject IServiceProvider ServiceProvider
@inject IServiceScopeFactory ServiceScopeFactory
@inject IJSRuntime JSRuntime
<ConfirmDialog @ref="dialog" />
<PageTitle>Documents</PageTitle>
<h4 class="mb-4">
<Icon Name="IconName.Upload" class="me-2" />
Upload new document
<h4 class="mb-1">
<Icon Name="IconName.CloudArrowUpFill" Color="IconColor.Primary" class="me-2" />
Upload a new document
</h4>
<p class="text-body-secondary small mb-4">The document is split into chunks, and an embedding is generated and stored for each chunk.</p>
<EditForm Model="Model" Enhance OnValidSubmit="UploadFile">
<DataAnnotationsValidator />
<div class="row">
<div class="col-md-5 col-sm-4 col-5">
<div class="input-group">
<span class="input-group-text">
<Tooltip Title="PDF, DOCX, TXT and MD files are supported" Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
<Icon Class="d-flex text-body-secondary" Name="IconName.InfoCircle"></Icon>
</Tooltip>
</span>
<InputFile class="form-control" OnChange="@((e) => Model.File = e.File)" accept=".pdf,.docx,.txt,.md" id="fileInput" />
<fieldset disabled="@isUploading">
<div class="row">
<div class="col-md-5 col-sm-4 col-5">
<div class="input-group">
<span class="input-group-text">
<Tooltip Title="@($"{UploadDocument.SupportedExtensionsDescription} files are supported, up to {UploadDocument.MaxFileSizeInMegaBytes} MB")" Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
<Icon Class="d-flex" Color="IconColor.Info" Name="IconName.InfoCircle"></Icon>
</Tooltip>
</span>
<InputFile class="form-control" OnChange="@((e) => Model.File = e.File)" accept="@UploadDocument.AcceptedFileTypes" id="fileInput" />
</div>
<ValidationMessage For="@(() => Model.File)" />
</div>
<div class="col-md-5 col-sm-5 col-5">
<div class="input-group">
<span class="input-group-text">
<Tooltip Title="The unique identifier (GUID) of the document. If not provided, a new one will be generated. If you specify an existing Document ID, the corresponding document will be overwritten." Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
<Icon Class="d-flex me-2" Color="IconColor.Warning" Name="IconName.KeyFill"></Icon>
</Tooltip>
Document ID
</span>
<TextInput Placeholder="Enter a valid GUID or leave empty for auto-generation" @bind-Value="@Model.DocumentId" />
</div>
<ValidationMessage For="@(() => Model.DocumentId)" />
</div>
<div class="col-md-2 col-sm-3 col-2">
<div class="d-grid gap-2">
<Button @ref="uploadButton" Type="ButtonType.Submit" Color="ButtonColor.Primary" To="#" Disabled="@(Model.File is null)" Class="w-100 py-2 fw-semibold shadow-sm"><Icon Name="IconName.Upload" /><span class="d-none d-lg-inline ps-3">Upload</span></Button>
</div>
</div>
</div>
<div class="col-md-5 col-sm-5 col-5">
<div class="input-group">
<span class="input-group-text">
<Tooltip Title="The unique identifier (GUID) of the document. If not provided, a new one will be generated. If you specify an existing Document ID, the corresponding document will be overwritten." Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
<Icon Class="d-flex text-body-secondary me-2" Name="IconName.InfoCircle"></Icon>
</Tooltip>
Document ID
</span>
<TextInput Placeholder="Enter a valid GUID or leave empty for auto-generation" @bind-Value="@Model.DocumentId" />
</div>
<ValidationMessage For="@(() => Model.DocumentId)" />
</div>
<div class="col-md-2 col-sm-3 col-2">
<div class="d-grid gap-2">
<Button @ref="uploadButton" Type="ButtonType.Submit" Color="ButtonColor.Primary" To="#" Disabled="@(Model.File is null)" Class="w-100 py-2 fw-semibold shadow-sm"><Icon Name="IconName.Upload" /><span class="d-none d-lg-inline ps-3">Upload</span></Button>
</div>
</div>
</div>
</fieldset>
</EditForm>
@if (isLoading && documents.Count == 0)
@@ -56,55 +60,66 @@
else
{
<h4 class="mt-4 mb-4">
<Icon Name="IconName.Files" class="me-2" />
<Icon Name="IconName.FileEarmarkTextFill" Color="IconColor.Info" class="me-2" />
Available documents
<span class="badge bg-secondary-subtle text-secondary-emphasis ms-2 align-middle">@documents.Count</span>
</h4>
<div class="table-responsive">
<table class="table table-hover align-middle mb-0 border rounded overflow-hidden">
<thead class="table-light sticky-top">
<tr>
<th style="width:40px;"></th>
<th class="text-secondary">ID</th>
<th class="text-secondary">Name</th>
<th class="text-secondary">Content type</th>
<th class="text-secondary text-center">Chunks</th>
<th class="text-secondary">Created</th>
</tr>
</thead>
<tbody>
@foreach (var document in documents)
{
<tr class="@((document.IsSelected ? "table-primary" : null))">
<td>
<div class="d-flex justify-content-center align-items-center">
<CheckboxInput @bind-Value="document.IsSelected" />
</div>
</td>
<td class="text-break small">@document.Id</td>
<td class="fw-medium">@document.Name</td>
<td>
<span class="badge content-type-badge px-2 py-1 rounded-pill small">
@document.ContentType
</span>
</td>
<td class="text-center">@document.ChunkCount</td>
<td class="small text-secondary">@document.LocalCreationDateString</td>
@if (documents.Count == 0)
{
<div class="text-center border rounded py-5 text-body-secondary">
<Icon Name="IconName.Inbox" Color="IconColor.Secondary" Size="IconSize.x3" />
<p class="mt-3 mb-0">No documents have been indexed yet. Upload one to get started.</p>
</div>
}
else
{
<div class="table-responsive">
<table class="table table-hover align-middle mb-0 border rounded overflow-hidden">
<thead class="table-light sticky-top">
<tr>
<th style="width:40px;"></th>
<th class="text-secondary">ID</th>
<th class="text-secondary">Name</th>
<th class="text-secondary">Content type</th>
<th class="text-secondary text-center">Chunks</th>
<th class="text-secondary">Created</th>
</tr>
}
</tbody>
</table>
</div>
<div class="my-4"></div>
<div class="row">
<div class="col-md-2 col-sm-3 col-2">
<div class="d-grid gap-2">
<Button @ref="deleteButton" Color="ButtonColor.Danger" Disabled="@(!documents.Any(d => d.IsSelected))" @onclick="DeleteSelectedDocuments" Class="w-100 py-2 fw-semibold shadow-sm">
<Icon Name="IconName.Trash" /><span class="d-none d-lg-inline ps-3">Delete</span>
</Button>
</thead>
<tbody>
@foreach (var document in documents)
{
<tr class="@((document.IsSelected ? "table-primary" : null))">
<td>
<div class="d-flex justify-content-center align-items-center">
<CheckboxInput @bind-Value="document.IsSelected" />
</div>
</td>
<td class="text-break small">@document.Id</td>
<td class="fw-medium">@document.Name</td>
<td>
<span class="badge content-type-badge px-2 py-1 rounded-pill small">
@document.ContentType
</span>
</td>
<td class="text-center">@document.ChunkCount</td>
<td class="small text-secondary">@document.LocalCreationDateString</td>
</tr>
}
</tbody>
</table>
</div>
<div class="my-4"></div>
<div class="row">
<div class="col-md-2 col-sm-3 col-2">
<div class="d-grid gap-2">
<Button @ref="deleteButton" Color="ButtonColor.Danger" Disabled="@(!documents.Any(d => d.IsSelected))" @onclick="DeleteSelectedDocuments" Class="w-100 py-2 fw-semibold shadow-sm">
<Icon Name="IconName.Trash" /><span class="d-none d-lg-inline ps-3">Delete</span>
</Button>
</div>
</div>
</div>
</div>
}
}
@code {
@@ -113,6 +128,7 @@ else
private Button deleteButton = default!;
private bool isLoading = true;
private bool isUploading;
private IList<SelectableDocument> documents = [];
private UploadDocument Model { get; set; } = new();
@@ -127,10 +143,10 @@ else
return;
}
await using var scope = ServiceProvider.CreateAsyncScope();
await using var scope = ServiceScopeFactory.CreateAsyncScope();
await LoadDocumentsAsync(scope.ServiceProvider);
StateHasChanged();
await InvokeAsync(StateHasChanged);
}
private async Task LoadDocumentsAsync(IServiceProvider services)
@@ -166,20 +182,22 @@ else
uploadButton.ShowLoading();
isUploading = true;
await InvokeAsync(StateHasChanged);
var fileName = Model.File.Name;
try
{
await using var inputStream = Model.File.OpenReadStream(20 * 1024 * 1024); // 20 MB
await using var inputStream = Model.File.OpenReadStream(UploadDocument.MaxFileSize);
await using var stream = await inputStream.GetMemoryStreamAsync();
await using var scope = ServiceProvider.CreateAsyncScope();
await using var scope = ServiceScopeFactory.CreateAsyncScope();
var vectorSearchService = scope.ServiceProvider.GetRequiredService<VectorSearchService>();
var documentId = string.IsNullOrWhiteSpace(Model.DocumentId) ? null : (Guid?)Guid.Parse(Model.DocumentId);
await vectorSearchService.ImportAsync(stream, fileName, MimeUtility.GetMimeMapping(fileName), documentId);
await vectorSearchService.ImportAsync(EmbeddingRequest.Create(stream, fileName, Model.GetDocumentId()));
ToastService.Notify(await CreateToastMessageAsync(ToastType.Success, "Upload document", $"The document {fileName} has been successfully uploaded and indexed."));
ToastService.Notify(await CreateToastMessageAsync(ToastType.Success, "Upload document", $"The document '{fileName}' has been successfully uploaded and indexed."));
Model = new UploadDocument();
await JSRuntime.InvokeVoidAsync("resetFileInput", "fileInput");
@@ -188,17 +206,20 @@ else
}
catch (Exception ex)
{
ToastService.Notify(await CreateToastMessageAsync(ToastType.Danger, "Upload error", $"There was an error while uploading the document {fileName}: {ex.Message}"));
ToastService.Notify(await CreateToastMessageAsync(ToastType.Danger, "Upload error", $"There was an error while uploading the document '{fileName}': {ex.Message}"));
}
finally
{
uploadButton.HideLoading();
isUploading = false;
await InvokeAsync(StateHasChanged);
}
}
private async Task DeleteSelectedDocuments()
{
var selectedDocumentIds = documents?.Where(d => d.IsSelected).Select(d => d.Id) ?? [];
var selectedDocumentIds = documents.Where(d => d.IsSelected).Select(d => d.Id);
var options = new ConfirmDialogOptions
{
@@ -223,7 +244,7 @@ else
{
deleteButton.ShowLoading();
await using var scope = ServiceProvider.CreateAsyncScope();
await using var scope = ServiceScopeFactory.CreateAsyncScope();
var documentService = scope.ServiceProvider.GetRequiredService<DocumentService>();
await documentService.DeleteAsync(selectedDocumentIds);
@@ -268,11 +289,46 @@ else
public string LocalCreationDateString { get; set; } = string.Empty;
}
public class UploadDocument
public class UploadDocument : IValidatableObject
{
public const int MaxFileSizeInMegaBytes = 20;
public const long MaxFileSize = MaxFileSizeInMegaBytes * 1024 * 1024;
public static readonly string[] SupportedExtensions = [".pdf", ".docx", ".txt", ".md"];
public static string AcceptedFileTypes { get; } = string.Join(',', SupportedExtensions);
public static string SupportedExtensionsDescription { get; } = string.Join(", ", SupportedExtensions.Select(e => e.TrimStart('.').ToUpperInvariant()));
public IBrowserFile? File { get; set; }
[RegularExpression(@"^(\{|\()?[0-9a-fA-F]{8}(-?)[0-9a-fA-F]{4}(-?)[0-9a-fA-F]{4}(-?)[0-9a-fA-F]{4}(-?)[0-9a-fA-F]{12}(\}|\))?$", ErrorMessage = "Invalid GUID format.")]
public string? DocumentId { get; set; }
public Guid? GetDocumentId()
=> Guid.TryParse(DocumentId, out var documentId) ? documentId : null;
public IEnumerable<ValidationResult> Validate(ValidationContext validationContext)
{
if (!string.IsNullOrWhiteSpace(DocumentId) && !Guid.TryParse(DocumentId, out _))
{
yield return new ValidationResult("Invalid GUID format.", [nameof(DocumentId)]);
}
if (File is null)
{
yield break;
}
if (!SupportedExtensions.Contains(Path.GetExtension(File.Name), StringComparer.OrdinalIgnoreCase))
{
yield return new ValidationResult($"Only {SupportedExtensionsDescription} files are supported.", [nameof(File)]);
}
if (File.Size > MaxFileSize)
{
yield return new ValidationResult($"The file exceeds the maximum allowed size of {MaxFileSizeInMegaBytes} MB.", [nameof(File)]);
}
}
}
}
SqlDatabaseVectorSearch/Components/Pages/Error.razor
+4 -6
View File
@@ -1,6 +1,4 @@
@page "/Error"
@using System.Diagnostics
@rendermode @(new InteractiveServerRenderMode(prerender: false))
<div class="d-flex align-items-center justify-content-center">
<div class="text-center">
@@ -9,9 +7,9 @@
<PageTitle>Page Not Found</PageTitle>
<h1 class="display-1 fw-bold">404</h1>
<p class="fs-3"><span class="text-danger">Ops!</span> Page Not Found.</p>
<p class="fs-3"><span class="text-danger">Oops!</span> Page Not Found.</p>
<p class="lead">
The page you're looking for does not exists.
The page you're looking for does not exist.
</p>
}
else if (Code > 0)
@@ -19,9 +17,9 @@
<PageTitle>Unexpected Error</PageTitle>
<h1 class="display-1 fw-bold">500</h1>
<p class="fs-3"><span class="text-danger">Ops!</span> Unexpected error.</p>
<p class="fs-3"><span class="text-danger">Oops!</span> Unexpected error.</p>
<p class="lead">
An unexpected error occurred while loading the page. Please, wait a minute and try again.
An unexpected error occurred while loading the page. Please wait a moment and try again.
</p>
}
SqlDatabaseVectorSearch/Components/Pages/Home.razor
+104 -26
View File
@@ -3,35 +3,113 @@
<PageTitle>SQL Database Vector Search</PageTitle>
<h1>SQL Database Vector Search</h1>
<div class="hero p-4 p-md-5 mb-4 rounded-3 shadow-sm">
<h1 class="display-6 fw-semibold mb-3">
<Icon Name="IconName.Search" Color="IconColor.Primary" Class="me-2" />
SQL Database Vector Search
</h1>
<p class="lead mb-4">
A Blazor Web App and Minimal API for Retrieval Augmented Generation (RAG) and vector search using the native
<code>VECTOR</code> type in
<img src="/images/sqldatabase.svg" class="inline-logo" alt="Azure SQL Database" /> Azure SQL Database or
SQL Server 2025, with
<img src="/images/openai.svg" class="inline-logo" alt="Azure OpenAI" /> Azure OpenAI and
<a href="https://github.com/microsoft/agent-framework" target="_blank" rel="noopener">Microsoft Agent Framework</a>.
</p>
<div class="d-flex flex-wrap gap-2">
<a href="/documents" class="btn btn-primary px-4 py-2 fw-semibold shadow-sm">
<Icon Name="IconName.CloudArrowUpFill" Class="me-2" />Upload a document
</a>
<a href="/ask" class="btn btn-outline-primary px-4 py-2 fw-semibold">
<Icon Name="IconName.ChatDotsFill" Class="me-2" />Ask a question
</a>
</div>
</div>
<p class="mt-3 p-3 rounded bg-light text-dark shadow-sm">
A Blazor Web App and Minimal API for Retrieval Augmented Generation (RAG) and vector search using the native VECTOR type in <img src="/images/sqldatabase.svg" style="height:1.5em;vertical-align:middle;" /> Azure SQL Database with <img src="/images/openai.svg" style="height:1.5em;vertical-align:middle;" /> Azure OpenAI.
</p>
<h2 class="h4 mb-3">
<Icon Name="IconName.Diagram3Fill" Color="IconColor.Secondary" Class="me-2" />
How it works
</h2>
<p>
This application allows you to:
<ul>
<li>Load documents (PDF, DOCX, TXT, MD)</li>
<li>Generate embeddings and save them as vectors in Azure SQL Database</li>
<li>Perform semantic search and RAG using Azure OpenAI</li>
<li>Interact via a Blazor Web App or programmatically via Minimal API</li>
</ul>
Embeddings and chat completion are powered by <a href="https://github.com/microsoft/semantic-kernel" target="_blank">Semantic Kernel</a>. Vectors are managed with <a href="https://github.com/efcore/EfCore.SqlServer.VectorSearch" target="_blank">EFCore.SqlServer.VectorSearch</a>.
</p>
<div class="row g-3 mb-4">
<div class="col-md-4">
<div class="card h-100 border-0 shadow-sm">
<div class="card-body">
<Icon Name="IconName.FileEarmarkArrowUpFill" Color="IconColor.Primary" Size="IconSize.x3" />
<h3 class="h6 mt-3 mb-2">1. Import</h3>
<p class="text-body-secondary mb-0 small">
Upload a PDF, DOCX, TXT or MD file. Its content is split into chunks, and each chunk is turned into
an embedding by an Agent Framework workflow.
</p>
</div>
</div>
</div>
<div class="col-md-4">
<div class="card h-100 border-0 shadow-sm">
<div class="card-body">
<Icon Name="IconName.DatabaseFillCheck" Color="IconColor.Success" Size="IconSize.x3" />
<h3 class="h6 mt-3 mb-2">2. Store</h3>
<p class="text-body-secondary mb-0 small">
Embeddings are persisted in Azure SQL Database or SQL Server 2025 using the native
<code>VECTOR</code> type, so no external vector store is required.
</p>
</div>
</div>
</div>
<div class="col-md-4">
<div class="card h-100 border-0 shadow-sm">
<div class="card-body">
<Icon Name="IconName.ChatSquareQuoteFill" Color="IconColor.Info" Size="IconSize.x3" />
<h3 class="h6 mt-3 mb-2">3. Ask</h3>
<p class="text-body-secondary mb-0 small">
Your question is reformulated with the conversation context, matched against the stored vectors, and
answered with citations.
</p>
</div>
</div>
</div>
</div>
<h3>Supported Features</h3>
<ul>
<li><strong>Conversation History with Question Reformulation</strong>: View and reformulate your conversation history for better clarity and understanding.</li>
<li><strong>Information about Token Usage</strong>: Access detailed information about token usage for transparency and management.</li>
<li><strong>Response Streaming</strong>: Receive real-time streaming of responses for a seamless and efficient user experience.</li>
<li><strong>Citations</strong>: Get citations for the sources used to justify each answer, allowing you to verify and understand the origin of the content.</li>
<h2 class="h4 mb-3">
<Icon Name="IconName.ListStars" Color="IconColor.Warning" Class="me-2" />
Supported features
</h2>
<ul class="list-group list-group-flush mb-4 shadow-sm rounded overflow-hidden border">
<li class="list-group-item">
<Icon Name="IconName.Diagram3Fill" Color="IconColor.Primary" Class="me-2" />
<strong>Microsoft Agent Framework orchestration</strong>: documents are imported through an embedding workflow,
and questions are answered by dedicated reformulation and RAG agents.
</li>
<li class="list-group-item">
<Icon Name="IconName.ArrowRepeat" Color="IconColor.Secondary" Class="me-2" />
<strong>Conversation history with question reformulation</strong>: follow-up questions are rewritten with the
current conversation context before the vector search is performed.
</li>
<li class="list-group-item">
<Icon Name="IconName.DatabaseFillCheck" Color="IconColor.Success" Class="me-2" />
<strong>SQL vector search</strong>: the most relevant chunks are retrieved from Azure SQL Database or
SQL Server 2025 through native <code>VECTOR</code> cosine-distance search.
</li>
<li class="list-group-item">
<Icon Name="IconName.CashCoin" Color="IconColor.Warning" Class="me-2" />
<strong>Token usage details</strong>: input, output and total tokens are reported for both question
reformulation and answer generation.
</li>
<li class="list-group-item">
<Icon Name="IconName.LightningChargeFill" Color="IconColor.Danger" Class="me-2" />
<strong>Response streaming</strong>: answers are streamed token by token in the chat page and through the
Server-Sent Events API.
</li>
<li class="list-group-item">
<Icon Name="IconName.ChatQuoteFill" Color="IconColor.Info" Class="me-2" />
<strong>Markdown source citations</strong>: citations are embedded in the answer with the source name, the page
number when available, and a short supporting excerpt.
</li>
</ul>
<p class="mt-3 p-3 rounded bg-light text-dark shadow-sm">
Try <a href="/documents">uploading a document</a> or <a href="/ask">ask a question</a> to get started!
</p>
<p class="mt-4">
<em>For API usage and more details, see the <a href="https://github.com/marcominerva/SqlDatabaseVectorSearch#how-to-use" target="_blank">README</a>.</em>
<p class="text-body-secondary small">
<Icon Name="IconName.InfoCircleFill" Color="IconColor.Info" Class="me-2" />
For API usage and more details, see the
<a href="https://github.com/marcominerva/SqlDatabaseVectorSearch#how-to-use" target="_blank" rel="noopener">README</a>.
</p>
SqlDatabaseVectorSearch/Components/Pages/Home.razor.css
+14
View File
@@ -0,0 +1,14 @@
.hero {
background: linear-gradient(135deg, #eef4ff 0%, #f8f9fa 100%);
border: 1px solid #e3e8f0;
}
.inline-logo {
height: 1.5em;
vertical-align: middle;
}
.card:hover {
transform: translateY(-2px);
transition: transform 0.15s ease-in-out;
}
SqlDatabaseVectorSearch/Components/_Imports.razor
+1
View File
@@ -13,4 +13,5 @@
@using SqlDatabaseVectorSearch.Extensions
@using SqlDatabaseVectorSearch.Models
@using SqlDatabaseVectorSearch.Services
@using SqlDatabaseVectorSearch.Workflows
@using BlazorBootstrap
SqlDatabaseVectorSearch/Endpoints/AskEndpoints.cs
+6 -4
View File
@@ -1,4 +1,6 @@
using System.ComponentModel;
using System.Net.ServerSentEvents;
using System.Runtime.CompilerServices;
using MinimalHelpers.FluentValidation;
using SqlDatabaseVectorSearch.Models;
using SqlDatabaseVectorSearch.Services;
@@ -23,18 +25,18 @@ public class AskEndpoints : IEndpointRouteHandlerBuilder
endpoints.MapPost("/api/ask-streaming", (Question question, VectorSearchService vectorSearchService, CancellationToken cancellationToken,
[Description("If true, the question will be reformulated taking into account the context of the chat identified by the given ConversationId.")] bool reformulate = true) =>
{
async IAsyncEnumerable<Response> Stream()
async IAsyncEnumerable<SseItem<Response>> StreamAsync([EnumeratorCancellation] CancellationToken innerCancellationToken)
{
// Requests a streaming response.
var responseStream = vectorSearchService.AskStreamingAsync(question, reformulate, cancellationToken);
var responseStream = vectorSearchService.AskStreamingAsync(question, reformulate, innerCancellationToken);
await foreach (var delta in responseStream)
{
yield return delta;
yield return new(delta, delta.StreamState?.ToString());
}
}
return Stream();
return TypedResults.ServerSentEvents(StreamAsync(cancellationToken));
})
.WithValidation<Question>()
.WithSummary("Asks a question and gets the response as streaming")
SqlDatabaseVectorSearch/Endpoints/DocumentEndpoints.cs
+3 -7
View File
@@ -1,9 +1,9 @@
using System.ComponentModel;
using Microsoft.AspNetCore.Http.HttpResults;
using MimeMapping;
using SqlDatabaseVectorSearch.Models;
using SqlDatabaseVectorSearch.Services;
using SqlDatabaseVectorSearch.Workflows;
namespace SqlDatabaseVectorSearch.Endpoints;
@@ -23,12 +23,8 @@ public class DocumentEndpoints : IEndpointRouteHandlerBuilder
documentsApiGroup.MapPost(string.Empty, async (IFormFile file, VectorSearchService vectorSearchService, CancellationToken cancellationToken,
[Description("The unique identifier of the document. If not provided, a new one will be generated. If you specify an existing documentId, the corresponding document will be overwritten.")] Guid? documentId = null) =>
{
using var stream = file.OpenReadStream();
// Note: file.ContentType is not 100% reliable (for example, for markdown file).
var response = await vectorSearchService.ImportAsync(stream, file.FileName, MimeUtility.GetMimeMapping(file.FileName), documentId, cancellationToken);
return TypedResults.Ok(response);
var result = await vectorSearchService.ImportAsync(EmbeddingRequest.FromFormFile(file, documentId), cancellationToken);
return TypedResults.Ok(result);
})
.DisableAntiforgery()
.ProducesProblem(StatusCodes.Status400BadRequest)
SqlDatabaseVectorSearch/Models/ChatResponse.cs
-3
View File
@@ -1,3 +0,0 @@
namespace SqlDatabaseVectorSearch.Models;
public record class ChatResponse(string? Text, TokenUsage? TokenUsage = null);
SqlDatabaseVectorSearch/Models/Citation.cs
-16
View File
@@ -1,16 +0,0 @@
namespace SqlDatabaseVectorSearch.Models;
public class Citation
{
public Guid DocumentId { get; set; }
public Guid ChunkId { get; set; }
public string FileName { get; set; } = null!;
public string Quote { get; set; } = null!;
public int? PageNumber { get; set; }
public int IndexOnPage { get; set; }
}
SqlDatabaseVectorSearch/Models/ImportDocumentResponse.cs
-3
View File
@@ -1,3 +0,0 @@
namespace SqlDatabaseVectorSearch.Models;
public record class ImportDocumentResponse(Guid DocumentId, int EmbeddingTokenCount);
SqlDatabaseVectorSearch/Models/Response.cs
+8 -3
View File
@@ -1,10 +1,15 @@
namespace SqlDatabaseVectorSearch.Models;
// Question and Answer can be null when using response streaming.
public record class Response(string? OriginalQuestion, string? ReformulatedQuestion, string? Answer, StreamState? StreamState = null, TokenUsageResponse? TokenUsage = null, IEnumerable<Citation>? Citations = null)
public record class Response(Guid ConversationId, string? OriginalQuestion, string? ReformulatedQuestion, string? Answer, StreamState? StreamState = null, TokenUsageResponse? TokenUsage = null)
{
public Response(string? token, StreamState streamState, TokenUsageResponse? tokenUsageResponse = null, IEnumerable<Citation>? citations = null)
: this(null, null, token, streamState, tokenUsageResponse, citations)
public Response(Guid conversationId, string? token, StreamState streamState, TokenUsageResponse? tokenUsageResponse = null)
: this(conversationId, null, null, token, streamState, tokenUsageResponse)
{
}
public Response(Guid conversationId, StreamState streamState, TokenUsageResponse? tokenUsageResponse = null)
: this(conversationId, null, null, null, streamState, tokenUsageResponse)
{
}
}
SqlDatabaseVectorSearch/Models/StreamState.cs
+1 -1
View File
@@ -3,6 +3,6 @@
public enum StreamState
{
Start,
Append,
Delta,
End
}
SqlDatabaseVectorSearch/Models/TokenUsage.cs
-6
View File
@@ -1,6 +0,0 @@
namespace SqlDatabaseVectorSearch.Models;
public record class TokenUsage(int PromptTokens, int CompletionTokens)
{
public int TotalTokens => PromptTokens + CompletionTokens;
}
SqlDatabaseVectorSearch/Models/TokenUsageResponse.cs
+4 -8
View File
@@ -1,9 +1,5 @@
namespace SqlDatabaseVectorSearch.Models;
using Microsoft.Extensions.AI;
public record class TokenUsageResponse(TokenUsage? Reformulation, int? EmbeddingTokenCount, TokenUsage? Question)
{
public TokenUsageResponse(TokenUsage? question)
: this(null, null, question)
{
}
}
namespace SqlDatabaseVectorSearch.Models;
public record class TokenUsageResponse(UsageDetails? Reformulation, UsageDetails? Question);
SqlDatabaseVectorSearch/Program.cs
+202 -15
View File
@@ -1,8 +1,15 @@
using System.ClientModel;
using System.Net.Mime;
using System.Text;
using System.Text.Json.Serialization;
using FluentValidation;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Hosting;
using Microsoft.Agents.AI.Workflows;
using Microsoft.EntityFrameworkCore;
using Microsoft.SemanticKernel;
using Microsoft.Extensions.AI;
using OpenAI;
using OpenAI.Responses;
using SqlDatabaseVectorSearch.Components;
using SqlDatabaseVectorSearch.ContentDecoders;
using SqlDatabaseVectorSearch.Data;
@@ -10,6 +17,7 @@ using SqlDatabaseVectorSearch.Extensions;
using SqlDatabaseVectorSearch.Services;
using SqlDatabaseVectorSearch.Settings;
using SqlDatabaseVectorSearch.TextChunkers;
using SqlDatabaseVectorSearch.Workflows;
using TinyHelpers.AspNetCore.Extensions;
using TinyHelpers.AspNetCore.OpenApi;
@@ -32,17 +40,23 @@ builder.Services.ConfigureHttpJsonOptions(options =>
builder.Services.AddSingleton(TimeProvider.System);
builder.Services.AddSqlServer<ApplicationDbContext>(builder.Configuration.GetConnectionString("SqlConnection"), optionsAction: options =>
builder.Services.AddDbContext<ApplicationDbContext>(options =>
{
options.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
});
var connectionString = builder.Configuration.GetConnectionString("SqlConnection")!;
builder.Services.AddHybridCache(options =>
{
options.DefaultEntryOptions = new()
if (connectionString.Contains("database.windows.net"))
{
LocalCacheExpiration = appSettings.MessageExpiration
};
options.UseAzureSql(connectionString);
}
else
{
options.UseSqlServer(connectionString, sqlOptions =>
{
sqlOptions.EnableRetryOnFailure(maxRetryCount: 5, maxRetryDelay: TimeSpan.FromSeconds(10), errorNumbersToAdd: null);
});
}
options.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
});
builder.Services.ConfigureHttpClientDefaults(configure =>
@@ -54,11 +68,25 @@ builder.Services.ConfigureHttpClientDefaults(configure =>
});
});
// Semantic Kernel is used to generate embeddings and to reformulate questions taking into account all the previous interactions,
// so that embeddings themselves can be generated more accurately.
builder.Services.AddKernel()
.AddAzureOpenAIEmbeddingGenerator(aiSettings.Embedding.Deployment, aiSettings.Embedding.Endpoint, aiSettings.Embedding.ApiKey, modelId: aiSettings.Embedding.ModelId, dimensions: aiSettings.Embedding.Dimensions)
.AddAzureOpenAIChatCompletion(aiSettings.ChatCompletion.Deployment, aiSettings.ChatCompletion.Endpoint, aiSettings.ChatCompletion.ApiKey, modelId: aiSettings.ChatCompletion.ModelId);
builder.Services.AddSingleton(_ =>
{
var embeddingClient = new OpenAIClient(new ApiKeyCredential(aiSettings.Embedding.ApiKey), new()
{
Endpoint = new(aiSettings.Embedding.Endpoint),
}).GetEmbeddingClient(aiSettings.Embedding.Deployment).AsIEmbeddingGenerator(aiSettings.Embedding.Dimensions);
return embeddingClient;
});
builder.Services.AddChatClient(_ =>
{
var chatClient = new OpenAIClient(new ApiKeyCredential(aiSettings.ChatCompletion.ApiKey), new()
{
Endpoint = new(aiSettings.ChatCompletion.Endpoint)
}).GetResponsesClient().AsIChatClientWithStoredOutputDisabled(aiSettings.ChatCompletion.Deployment);
return chatClient;
});
builder.Services.AddKeyedSingleton<IContentDecoder, PdfContentDecoder>(MediaTypeNames.Application.Pdf);
builder.Services.AddKeyedSingleton<IContentDecoder, DocxContentDecoder>("application/vnd.openxmlformats-officedocument.wordprocessingml.document");
@@ -69,10 +97,168 @@ builder.Services.AddKeyedSingleton<ITextChunker, DefaultTextChunker>(KeyedServic
builder.Services.AddKeyedSingleton<ITextChunker, MarkdownTextChunker>(MediaTypeNames.Text.Markdown);
builder.Services.AddSingleton<TokenizerService>();
builder.Services.AddSingleton<ChatService>();
builder.Services.AddScoped<DocumentService>();
builder.Services.AddScoped<VectorSearchService>();
builder.Services.AddScoped<DocumentContextProviderService>();
builder.Services.AddSingleton<ExtractChunksExecutor>();
builder.Services.AddSingleton<GenerateEmbeddingExecutor>();
builder.Services.AddScoped<StoreEmbeddingExecutor>(); // This executor is registered as scoped because it uses the DbContext, which is also scoped.
builder.AddWorkflow("EmbeddingWorkflow", (services, key) =>
{
var extractChunksExecutor = services.GetRequiredService<ExtractChunksExecutor>();
var generateEmbeddingExecutor = services.GetRequiredService<GenerateEmbeddingExecutor>();
var storeEmbeddingExecutor = services.GetRequiredService<StoreEmbeddingExecutor>();
var workflow = new WorkflowBuilder(extractChunksExecutor).WithName(key)
.AddEdge(extractChunksExecutor, generateEmbeddingExecutor)
.AddEdge(generateEmbeddingExecutor, storeEmbeddingExecutor)
.WithOutputFrom(storeEmbeddingExecutor)
.Build(validateOrphans: true);
return workflow;
}, ServiceLifetime.Scoped);
builder.Services.AddAIAgent("ReformulationAgent", (services, key) =>
{
var chatClient = services.GetRequiredService<IChatClient>();
return chatClient.AsAIAgent(new()
{
Id = key.ToLowerInvariant(),
Name = key,
ChatOptions = new()
{
Instructions = """
You are a helpful assistant that reformulates questions to perform embeddings search.
Your task is to reformulate the question taking into account the context of the chat.
The reformulated question must always explicitly contain the subject of the question.
You MUST reformulate the question in the SAME language as the user's question.
For example, if the user asks a question in English, the reformulated question MUST be in English. If the user asks in Italian, the reformulated question MUST be in Italian.
Never add "in this chat", "in the context of this chat", "in the context of our conversation", "search for" or something like that in your answer.
Your answer must contain only the reformulated question and nothing else.
Never add follow-up messages, clarifications, notes, disclaimers, or requests for more information such as "if you give me more information, I can be more precise".
""",
Reasoning = new()
{
Effort = ReasoningEffort.None,
Output = ReasoningOutput.None
}
},
ChatHistoryProvider = new InMemoryChatHistoryProvider(new()
{
// The reformulation agent reads the conversation only to get the context it needs, but its own questions and answers
// must not pollute the session: the history is kept clean for the RAG agent, so nothing is stored back.
StorageInputRequestMessageFilter = _ => [],
StorageInputResponseMessageFilter = _ => []
})
},
loggerFactory: services.GetRequiredService<ILoggerFactory>(),
services: services);
});
var textSearchOptions = new TextSearchProviderOptions()
{
ContextFormatter = results =>
{
var sb = new StringBuilder();
sb.AppendLine("## Additional Context");
sb.AppendLine("Use the excerpts below to answer the user.");
sb.AppendLine("Citation rules:");
sb.AppendLine("- Do NOT add inline citations.");
sb.AppendLine("- At the END of your answer, add a sources section that follows this template exactly, where the sources label and the page label are localized in the same language as the user's question:");
sb.AppendLine(" *localized-sources-label*");
sb.AppendLine(" 1. **SourceName**, localized-page-label PageNumber: *supporting excerpt of about 20-30 words*");
sb.AppendLine("- Omit the page label and the page number when the page number is not available.");
sb.AppendLine("- Do NOT use headings or links in the sources section.");
sb.AppendLine("- Include ONLY sources you actually used. No duplicates.");
sb.AppendLine();
sb.AppendLine("### Sources");
foreach (var (i, r) in results.Index())
{
sb.AppendLine($"[{i + 1}] {GetSourceName(r, i)}");
sb.AppendLine(r.Text);
sb.AppendLine("---");
}
return sb.ToString();
static string GetSourceName(TextSearchProvider.TextSearchResult result, int index)
{
var name = string.IsNullOrWhiteSpace(result.SourceName) ? $"Source {index + 1}" : result.SourceName;
var pageNumber = result.RawRepresentation is int number ? number : (int?)null;
var pageText = pageNumber.HasValue ? $", page {pageNumber}" : string.Empty;
return $"{name}{pageText}";
}
}
};
builder.Services.AddHybridCache(options =>
{
options.DefaultEntryOptions = new()
{
LocalCacheExpiration = appSettings.MessageExpiration
};
});
builder.Services.AddSingleton<HybridCacheSessionStoreService>();
builder.Services.AddAIAgent("RagAgent", (services, key) =>
{
var chatClient = services.GetRequiredService<IChatClient>();
return chatClient.AsAIAgent(new()
{
Id = key.ToLowerInvariant(),
Name = key,
ChatOptions = new()
{
Instructions = """
You are a helpful assistant. Answer questions using the provided context and cite the source document when available.
You can use only the information provided in this chat to answer questions. If you don't know the answer, reply suggesting to refine the question.
For example, if the user asks "What is the capital of Italy?" and in this chat there isn't information about Italy, you should reply something like:
- This information isn't available in the given context.
- I'm sorry, I don't know the answer to that question.
- I don't have that information.
- I don't know.
- Given the context, I can't answer that question.
- I'm sorry, I don't have enough information to answer that question.
Never answer questions that are not related to this chat.
""",
Reasoning = new()
{
Effort = ReasoningEffort.Low,
Output = ReasoningOutput.None
}
},
ChatHistoryProvider = new InMemoryChatHistoryProvider(new()
{
ChatReducer = new MessageCountingChatReducer(appSettings.MessageLimit),
ReducerTriggerEvent = InMemoryChatHistoryProviderOptions.ChatReducerTriggerEvent.AfterMessageAdded,
StorageInputRequestMessageFilter = messages =>
{
return messages.Where(m => m.GetAgentRequestMessageSourceType() != AgentRequestMessageSourceType.ChatHistory
&& m.GetAgentRequestMessageSourceType() != AgentRequestMessageSourceType.AIContextProvider);
}
}),
AIContextProviders = [new TextSearchProvider(services.GetRequiredService<DocumentContextProviderService>().SearchAsync, textSearchOptions)]
},
loggerFactory: services.GetRequiredService<ILoggerFactory>(),
services: services);
}, ServiceLifetime.Scoped)
.WithSessionStore((services, _) =>
{
var sessionStore = services.GetRequiredService<HybridCacheSessionStoreService>();
return sessionStore;
}, withIsolation: false);
builder.Services.AddOpenApi(options =>
{
@@ -109,6 +295,7 @@ app.UseWhen(context => context.IsApiRequest(), builder =>
{
app.UseExceptionHandler(new ExceptionHandlerOptions
{
SuppressDiagnosticsCallback = _ => false,
StatusCodeSelector = exception => exception switch
{
NotSupportedException => StatusCodes.Status501NotImplemented,
SqlDatabaseVectorSearch/Services/ChatService.cs
-247
View File
@@ -1,247 +0,0 @@
using System.Runtime.CompilerServices;
using System.Text;
using Microsoft.Extensions.Caching.Hybrid;
using Microsoft.Extensions.Options;
using Microsoft.SemanticKernel.ChatCompletion;
using Microsoft.SemanticKernel.Connectors.AzureOpenAI;
using OpenAI.Chat;
using SqlDatabaseVectorSearch.Models;
using SqlDatabaseVectorSearch.Settings;
using Entities = SqlDatabaseVectorSearch.Data.Entities;
namespace SqlDatabaseVectorSearch.Services;
public class ChatService(IChatCompletionService chatCompletionService, TokenizerService tokenizerService, HybridCache cache, IOptions<AppSettings> appSettingsOptions, ILogger<ChatService> logger)
{
private readonly AppSettings appSettings = appSettingsOptions.Value;
private static readonly string systemPromptForReformulation = """
You are a helpful assistant that reformulates questions to perform embeddings search.
Your task is to reformulate the question taking into account the context of the chat.
The reformulated question must always explicitly contain the subject of the question.
You MUST reformulate the question in the SAME language as the user's question. For example, if the user asks a question in English, the reformulated question MUST be in English. If the user asks in Italian, the reformulated question MUST be in Italian.
If asking a clarifying question to the user would help, ask the question.
Never add "in this chat", "in the context of this chat", "in the context of our conversation", "search for" or something like that in your answer.
""";
private static readonly string systemPromptForAnswering = """
You can use only the information provided in this chat to answer questions. If you don't know the answer, reply suggesting to refine the question.
For example, if the user asks "What is the capital of Italy?" and in this chat there isn't information about Italy, you should reply something like:
- This information isn't available in the given context.
- I'm sorry, I don't know the answer to that question.
- I don't have that information.
- I don't know.
- Given the context, I can't answer that question.
- I'm sorry, I don't have enough information to answer that question.
Never answer questions that are not related to this chat.
LANGUAGE RULE: You MUST ALWAYS answer in the SAME language as the user's question. For example, if the user asks a question in English, the answer MUST be in English. If the user asks in Italian, the answer MUST be in Italian. This rule applies NO MATTER what language the documents are written in. The language of your response must match the language of the question, NOT the language of the documents.
FORMATTING REQUIREMENT: Your answer MUST ALWAYS end with a period followed by a space before the citations block.
If your answer doesn't naturally end with a period, you MUST add one followed by a space.
After the answer, you need to include citations following the XML format below ONLY IF you know the answer and are providing information from the context. If you do NOT know the answer, DO NOT include the citations section at all.
【<citation document-id="document_id" chunk-id="chunk_id" filename="string" page-number="page_number" index-on-page="index_on_page">exact quote here</citation>
<citation document-id="document_id" chunk-id="chunk_id" filename="string" page-number="page_number" index-on-page="index_on_page">exact quote here</citation>】
The entire list of XML citations MUST be enclosed between 【 and 】 (U+3010 and U+3011) and must exactly match the above format.
The quote in each <citation> MUST be MAXIMUM 5 words, taken word-for-word from the search result.
IMPORTANT CITATION RULES:
1. NEVER put citations inside your answer text.
2. ALWAYS provide your complete answer FIRST.
3. ONLY AFTER completing your answer, add ALL citations in a block at the very end.
4. The citations block MUST be the last thing in your response, with absolutely nothing (no text, no spaces, no newlines, no punctuation, no comments) after it.
5. NEVER reference citations by number or mention them in your answer text.
6. The citations MUST ALWAYS follow the XML format exactly as shown below. Any other format is NOT ACCEPTED.
7. If you add anything after the citations block, your answer will be considered invalid.
8. If you do NOT know the answer, DO NOT include the citations block at all.
9. ALWAYS check that your answer ends with a period followed by a space before adding citations.
---
Example of a correct answer:
The capital of Italy is Rome.
【<citation document-id="123" chunk-id="456" filename="italy.pdf" page-number="1" index-on-page="1">capital of Italy is Rome</citation>】
Example of a correct answer when you do NOT know the answer:
I'm sorry, I don't know the answer to that question.
Example of an incorrect answer (NOT ACCEPTED):
The capital of Italy is Rome
【<citation document-id="123" chunk-id="456" filename="italy.pdf" page-number="1" index-on-page="1">capital of Italy is Rome</citation>】
Thank you for your question.
Another incorrect example (NOT ACCEPTED):
The capital of Italy is Rome.
【<citation document-id="123" chunk-id="456" filename="italy.pdf" page-number="1" index-on-page="1">capital of Italy is Rome</citation>】
[1] italy.pdf, page 1
---
Only the correct format is accepted. If you do not follow the XML format exactly, or if you add anything after the citations block, your answer will be considered invalid.
If you do NOT know the answer, DO NOT include the citations block at all.
Remember to ALWAYS end your answer with a period followed by a space before adding citations.
""";
public async Task<ChatResponse> CreateReformulateQuestionAsync(Guid conversationId, string question, CancellationToken cancellationToken = default)
{
var chat = await GetChatHistoryAsync(conversationId, cancellationToken);
var settings = new AzureOpenAIPromptExecutionSettings
{
ChatSystemPrompt = systemPromptForReformulation
};
var embeddingQuestion = $"""
Reformulate the following question:
---
{question}
""";
chat.AddUserMessage(embeddingQuestion);
var reformulatedQuestion = await chatCompletionService.GetChatMessageContentAsync(chat, settings, cancellationToken: cancellationToken);
chat.AddAssistantMessage(reformulatedQuestion.Content!);
await UpdateCacheAsync(conversationId, chat, cancellationToken);
var tokenUsage = GetTokenUsage(reformulatedQuestion);
logger.LogDebug("Reformulation: {TokenUsage}", tokenUsage);
return new(reformulatedQuestion.Content!, tokenUsage);
}
public async Task<ChatResponse> AskQuestionAsync(Guid conversationId, IEnumerable<Entities.DocumentChunk> chunks, string question, CancellationToken cancellationToken = default)
{
var (chat, settings) = CreateChatAsync(chunks, question);
var answer = await chatCompletionService.GetChatMessageContentAsync(chat, settings, cancellationToken: cancellationToken);
// Add question and answer to the chat history.
await SetChatHistoryAsync(conversationId, question, answer.Content!, cancellationToken);
var tokenUsage = GetTokenUsage(answer);
logger.LogDebug("Ask question: {TokenUsage}", tokenUsage);
return new(answer.Content!, tokenUsage);
}
public async IAsyncEnumerable<ChatResponse> AskStreamingAsync(Guid conversationId, IEnumerable<Entities.DocumentChunk> chunks, string question, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
var (chat, settings) = CreateChatAsync(chunks, question);
var answer = new StringBuilder();
await foreach (var token in chatCompletionService.GetStreamingChatMessageContentsAsync(chat, settings, cancellationToken: cancellationToken))
{
if (!string.IsNullOrEmpty(token.Content))
{
yield return new(token.Content);
answer.Append(token.Content);
}
else if (token.Content is null)
{
// Token usage is returned in the last message, when the Content is null.
var tokenUsage = GetTokenUsage(token);
if (tokenUsage is not null)
{
logger.LogDebug("Ask streaming: {TokenUsage}", tokenUsage);
yield return new(null, tokenUsage);
}
}
}
// Add question and answer to the chat history.
await SetChatHistoryAsync(conversationId, question, answer.ToString(), cancellationToken).ConfigureAwait(false);
}
private static TokenUsage? GetTokenUsage(Microsoft.SemanticKernel.ChatMessageContent message) =>
message.InnerContent is ChatCompletion content && content.Usage is not null
? new(content.Usage.InputTokenCount, content.Usage.OutputTokenCount) : null;
private static TokenUsage? GetTokenUsage(Microsoft.SemanticKernel.StreamingChatMessageContent message) =>
message.InnerContent is StreamingChatCompletionUpdate content && content.Usage is not null
? new(content.Usage.InputTokenCount, content.Usage.OutputTokenCount) : null;
private (ChatHistory Chat, AzureOpenAIPromptExecutionSettings Settings) CreateChatAsync(IEnumerable<Entities.DocumentChunk> chunks, string question)
{
var settings = new AzureOpenAIPromptExecutionSettings
{
MaxTokens = appSettings.MaxOutputTokens,
ChatSystemPrompt = systemPromptForAnswering
};
var prompt = new StringBuilder($"""
Answer the following question:
---
{question}
=====
Using the following information:
""");
var availableTokens = appSettings.MaxInputTokens
- tokenizerService.CountChatCompletionTokens(systemPromptForAnswering) // System prompt.
- tokenizerService.CountChatCompletionTokens(prompt.ToString()) // Initial user prompt.
- appSettings.MaxOutputTokens; // To ensure there is enough space for the answer.
foreach (var chunk in chunks)
{
var text = $"--- {chunk.Document.Name} (Document ID: {chunk.Document.Id} | Chunk ID: {chunk.Id} | Page Number: {chunk.PageNumber} | Index on Page: {chunk.IndexOnPage}) {Environment.NewLine}{chunk.Content}{Environment.NewLine}";
var tokenCount = tokenizerService.CountChatCompletionTokens(text);
if (tokenCount > availableTokens)
{
// There isn't enough space to add the current chunk.
break;
}
prompt.Append(text);
availableTokens -= tokenCount;
if (availableTokens <= 0)
{
// There isn't enough space to add more chunks.
break;
}
}
var chat = new ChatHistory();
chat.AddUserMessage(prompt.ToString());
return (chat, settings);
}
private async Task UpdateCacheAsync(Guid conversationId, ChatHistory chat, CancellationToken cancellationToken)
{
if (chat.Count > appSettings.MessageLimit)
{
chat.RemoveRange(0, chat.Count - appSettings.MessageLimit);
}
await cache.SetAsync(conversationId.ToString(), chat, cancellationToken: cancellationToken);
}
private async Task<ChatHistory> GetChatHistoryAsync(Guid conversationId, CancellationToken cancellationToken)
{
var chat = await cache.GetOrCreateAsync(conversationId.ToString(), (cancellationToken) =>
{
return ValueTask.FromResult<ChatHistory>([]);
}, cancellationToken: cancellationToken);
return chat;
}
private async Task SetChatHistoryAsync(Guid conversationId, string question, string answer, CancellationToken cancellationToken)
{
var chat = await GetChatHistoryAsync(conversationId, cancellationToken);
chat.AddUserMessage(question);
chat.AddAssistantMessage(answer);
await UpdateCacheAsync(conversationId, chat, cancellationToken);
}
}
SqlDatabaseVectorSearch/Services/DocumentContextProviderService.cs
+35
View File
@@ -0,0 +1,35 @@
using System.Data;
using Microsoft.Agents.AI;
using Microsoft.Data.SqlTypes;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Options;
using SqlDatabaseVectorSearch.Data;
using SqlDatabaseVectorSearch.Settings;
namespace SqlDatabaseVectorSearch.Services;
public class DocumentContextProviderService(ApplicationDbContext dbContext, IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator, IOptions<AppSettings> appSettingsOptions)
{
private readonly AppSettings appSettings = appSettingsOptions.Value;
public async Task<IEnumerable<TextSearchProvider.TextSearchResult>> SearchAsync(string query, CancellationToken cancellationToken)
{
// Perform Vector Search on SQL Database.
var questionEmbedding = await embeddingGenerator.GenerateVectorAsync(query, cancellationToken: cancellationToken);
var embeddingVector = new SqlVector<float>(questionEmbedding);
var chunks = await dbContext.DocumentChunks.Include(c => c.Document)
.OrderBy(c => EF.Functions.VectorDistance("cosine", c.Embedding, embeddingVector))
.Take(appSettings.MaxRelevantChunks).Select(c => new TextSearchProvider.TextSearchResult
{
SourceLink = c.Id.ToString().ToLowerInvariant(),
SourceName = c.Document.Name,
Text = c.Content,
RawRepresentation = c.PageNumber
})
.ToListAsync(cancellationToken);
return chunks;
}
}
SqlDatabaseVectorSearch/Services/HybridCacheSessionStoreService.cs
+37
View File
@@ -0,0 +1,37 @@
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Hosting;
using Microsoft.Extensions.Caching.Hybrid;
namespace SqlDatabaseVectorSearch.Services;
public class HybridCacheSessionStoreService(HybridCache cache) : AgentSessionStore
{
public override async ValueTask<AgentSession> GetSessionAsync(AIAgent agent, string conversationId, CancellationToken cancellationToken = default)
{
var key = GetKey(agent, conversationId);
var sessionContent = await cache.GetOrCreateAsync(key, async ct =>
{
var session = await agent.CreateSessionAsync(ct);
return await agent.SerializeSessionAsync(session, cancellationToken: ct);
}, cancellationToken: cancellationToken);
return await agent.DeserializeSessionAsync(sessionContent, cancellationToken: cancellationToken);
}
public override async ValueTask SaveSessionAsync(AIAgent agent, string conversationId, AgentSession session, CancellationToken cancellationToken = default)
{
var key = GetKey(agent, conversationId);
var sessionContent = await agent.SerializeSessionAsync(session, cancellationToken: cancellationToken);
await cache.SetAsync(key, sessionContent, cancellationToken: cancellationToken);
}
public override async ValueTask DeleteSessionAsync(AIAgent agent, string conversationId, CancellationToken cancellationToken = default)
{
var key = GetKey(agent, conversationId);
await cache.RemoveAsync(key, cancellationToken);
}
private static string GetKey(AIAgent agent, string conversationId)
=> $"{agent.Id}:{conversationId}";
}
SqlDatabaseVectorSearch/Services/VectorSearchService.cs
+50 -166
View File
@@ -1,200 +1,84 @@
using System.Data;
using System.Runtime.CompilerServices;
using System.Text;
using System.Text.RegularExpressions;
using Microsoft.Data.SqlTypes;
using Microsoft.EntityFrameworkCore;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Hosting;
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Options;
using SqlDatabaseVectorSearch.ContentDecoders;
using SqlDatabaseVectorSearch.Data;
using SqlDatabaseVectorSearch.Models;
using SqlDatabaseVectorSearch.Settings;
using ChatResponse = SqlDatabaseVectorSearch.Models.ChatResponse;
using Entities = SqlDatabaseVectorSearch.Data.Entities;
using SqlDatabaseVectorSearch.Workflows;
namespace SqlDatabaseVectorSearch.Services;
public partial class VectorSearchService(IServiceProvider serviceProvider, ApplicationDbContext dbContext, DocumentService documentService, IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator, TokenizerService tokenizerService, ChatService chatService, TimeProvider timeProvider, IOptions<AppSettings> appSettingsOptions, ILogger<VectorSearchService> logger)
public partial class VectorSearchService([FromKeyedServices("EmbeddingWorkflow")] Workflow workflow, [FromKeyedServices("ReformulationAgent")] AIAgent reformulationAgent, [FromKeyedServices("RagAgent")] AIAgent ragAgent,
[FromKeyedServices("RagAgent")] AgentSessionStore sessionStore)
{
private readonly AppSettings appSettings = appSettingsOptions.Value;
public async Task<ImportDocumentResponse> ImportAsync(Stream stream, string name, string contentType, Guid? documentId, CancellationToken cancellationToken = default)
public async Task<StoreEmbeddingResponse> ImportAsync(EmbeddingRequest request, CancellationToken cancellationToken = default)
{
// Extract the contents of the file.
var decoder = serviceProvider.GetKeyedService<IContentDecoder>(contentType) ?? throw new NotSupportedException($"Content type '{contentType}' is not supported.");
var chunks = await decoder.DecodeAsync(stream, contentType, cancellationToken);
var chunkContents = chunks.Select(p => p.Content).ToList();
await using var run = await InProcessExecution.RunAsync(workflow, request, cancellationToken: cancellationToken);
var events = run.NewEvents.ToList();
// We get the token count of the whole document because it is the total number of token used by embedding (it may be necessary, for example, for cost analysis).
var tokenCount = tokenizerService.CountEmbeddingTokens(string.Join(" ", chunkContents));
var strategy = dbContext.Database.CreateExecutionStrategy();
var document = await strategy.ExecuteAsync(async (cancellationToken) =>
var exception = events.OfType<WorkflowErrorEvent>().Select(e => e.Exception).FirstOrDefault();
if (exception is not null)
{
await dbContext.Database.BeginTransactionAsync(cancellationToken);
throw exception;
}
if (documentId.HasValue)
{
// If the user is importing a document that already exists, delete the previous one.
await documentService.DeleteAsync(documentId.Value, cancellationToken);
}
var document = new Entities.Document { Id = documentId.GetValueOrDefault(), Name = name, CreationDate = timeProvider.GetUtcNow() };
dbContext.Documents.Add(document);
// Process paragraphs in batches.
var embeddings = new List<Embedding<float>>();
foreach (var batch in chunkContents.Chunk(appSettings.EmbeddingBatchSize))
{
logger.LogDebug("Processing batch of {Count} chunks for embedding generation...", batch.Length);
// Generate embeddings for this batch.
var batchEmbeddings = await embeddingGenerator.GenerateAsync(batch, cancellationToken: cancellationToken);
embeddings.AddRange(batchEmbeddings);
}
// Save the document chunks and the corresponding embedding in the database.
foreach (var (index, embedding) in embeddings.Index())
{
var chunk = chunks.ElementAt(index);
logger.LogDebug("Storing a chunk of {TokenCount} tokens.", tokenizerService.CountEmbeddingTokens(chunk.Content));
var documentChunk = new Entities.DocumentChunk
{
Document = document,
Index = index,
PageNumber = chunk.PageNumber,
IndexOnPage = chunk.IndexOnPage,
Content = chunk.Content,
Embedding = new SqlVector<float>(embedding.Vector)
};
dbContext.DocumentChunks.Add(documentChunk);
}
await dbContext.SaveChangesAsync(cancellationToken);
await dbContext.Database.CommitTransactionAsync(cancellationToken);
return document;
}, cancellationToken);
return new(document.Id, tokenCount);
var result = events.OfType<WorkflowOutputEvent>().Select(e => e.Data).OfType<StoreEmbeddingResponse>().First();
return result;
}
public async Task<Response> AskQuestionAsync(Question question, bool reformulate = true, CancellationToken cancellationToken = default)
{
// It the user doesn't want to reforulate the question, CreateContextAsync returns the original one.
var (reformulatedQuestion, embeddingTokenCount, chunks) = await CreateContextAsync(question, reformulate, cancellationToken);
UsageDetails? reformulationUsage = null;
var reformulatedQuestion = question.Text;
var session = await sessionStore.GetSessionAsync(ragAgent, question.ConversationId.ToString(), cancellationToken);
var (fullAnswer, tokenUsage) = await chatService.AskQuestionAsync(question.ConversationId, chunks, reformulatedQuestion.Text!, cancellationToken);
if (reformulate)
{
// Reformulates the question taking into account the context of the chat to perform keyword search and embeddings.
var reformulationResponse = await reformulationAgent.RunAsync(question.Text, session, cancellationToken: cancellationToken);
reformulatedQuestion = reformulationResponse.Text;
reformulationUsage = reformulationResponse.Usage;
}
// Extract citations from the answer.
var (answer, citations) = ExtractCitations(fullAnswer);
var response = await ragAgent.RunAsync(reformulatedQuestion, session, cancellationToken: cancellationToken);
return new(question.Text, reformulatedQuestion.Text!, answer, StreamState.End, new(reformulatedQuestion.TokenUsage, embeddingTokenCount, tokenUsage), citations);
await sessionStore.SaveSessionAsync(ragAgent, question.ConversationId.ToString(), session, cancellationToken);
return new(question.ConversationId, question.Text, reformulatedQuestion, response.Text, null, new TokenUsageResponse(reformulationUsage, response.Usage));
}
public async IAsyncEnumerable<Response> AskStreamingAsync(Question question, bool reformulate = true, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
// It the user doesn't want to reforulate the question, CreateContextAsync returns the original one.
var (reformulatedQuestion, embeddingTokenCount, chunks) = await CreateContextAsync(question, reformulate, cancellationToken);
UsageDetails? reformulationUsage = null;
var reformulatedQuestion = question.Text;
var session = await sessionStore.GetSessionAsync(ragAgent, question.ConversationId.ToString(), cancellationToken);
var answerStream = chatService.AskStreamingAsync(question.ConversationId, chunks, reformulatedQuestion.Text!, cancellationToken: cancellationToken);
if (reformulate)
{
// Reformulates the question taking into account the context of the chat to perform keyword search and embeddings.
var reformulationResponse = await reformulationAgent.RunAsync(question.Text, session, cancellationToken: cancellationToken);
reformulatedQuestion = reformulationResponse.Text;
reformulationUsage = reformulationResponse.Usage;
}
// The first message contains the question and the corresponding token usage (if reformulated).
yield return new(question.Text, reformulatedQuestion.Text!, null, StreamState.Start, new(reformulatedQuestion.TokenUsage, embeddingTokenCount, null));
yield return new(question.ConversationId, question.Text, reformulatedQuestion, null, StreamState.Start, new(reformulationUsage, null));
TokenUsageResponse? tokenUsageResponse = null;
var fullAnswer = new StringBuilder();
var citationsStarted = false;
var updates = new List<AgentResponseUpdate>();
// Returns each token as a partial response.
await foreach (var (token, tokenUsage) in answerStream)
await foreach (var update in ragAgent.RunStreamingAsync(reformulatedQuestion, session, cancellationToken: cancellationToken))
{
if (token is not null) // token can be null when the stream ends.
updates.Add(update);
if (!string.IsNullOrEmpty(update.Text))
{
fullAnswer.Append(token);
if (token.Contains('【'))
{
// Citations start when we encounter a token containing a 【 character.
// We need to track it because we don't want to return the citations in the actual response.
citationsStarted = true;
}
if (!citationsStarted)
{
yield return new(token, StreamState.Append);
}
}
else
{
// Token usage is expected in the last message, when token is null.
tokenUsageResponse ??= tokenUsage is not null ? new(tokenUsage) : null;
yield return new(question.ConversationId, update.Text, StreamState.Delta);
}
}
// Extract citations at the end of streaming.
var (_, citations) = ExtractCitations(fullAnswer.ToString());
yield return new(null, StreamState.End, tokenUsageResponse, citations);
await sessionStore.SaveSessionAsync(ragAgent, question.ConversationId.ToString(), session, cancellationToken);
var response = updates.ToAgentResponse();
yield return new(question.ConversationId, StreamState.End, new TokenUsageResponse(null, response.Usage));
}
private async Task<(ChatResponse ReformulatedQuestion, int EmbeddingTokenCount, IEnumerable<Entities.DocumentChunk> Chunks)> CreateContextAsync(Question question, bool reformulate, CancellationToken cancellationToken)
{
// Reformulate the question taking into account the context of the chat to perform keyword search and embeddings.
var reformulatedQuestion = reformulate ? await chatService.CreateReformulateQuestionAsync(question.ConversationId, question.Text, cancellationToken) : new(question.Text);
var embeddingTokenCount = tokenizerService.CountEmbeddingTokens(reformulatedQuestion.Text!);
logger.LogDebug("Embedding Token Count: {EmbeddingTokenCount}", embeddingTokenCount);
// Perform Vector Search on SQL Database.
var questionEmbedding = await embeddingGenerator.GenerateVectorAsync(reformulatedQuestion.Text!, cancellationToken: cancellationToken);
var embeddingVector = new SqlVector<float>(questionEmbedding);
var chunks = await dbContext.DocumentChunks.Include(c => c.Document)
.OrderBy(c => EF.Functions.VectorDistance("cosine", c.Embedding, embeddingVector))
.Take(appSettings.MaxRelevantChunks)
.ToListAsync(cancellationToken);
return (reformulatedQuestion, embeddingTokenCount, chunks);
}
private static (string, IEnumerable<Citation>) ExtractCitations(string? text)
{
var citations = new List<Citation>();
if (string.IsNullOrEmpty(text))
{
return (text ?? string.Empty, citations);
}
var matches = CitationRegEx.Matches(text);
foreach (Match match in matches)
{
if (match.Success)
{
citations.Add(new Citation
{
DocumentId = Guid.Parse(match.Groups["documentId"].Value),
ChunkId = Guid.Parse(match.Groups["chunkId"].Value),
FileName = match.Groups["filename"].Value,
PageNumber = int.TryParse(match.Groups["pageNumber"].Value, out var pageNumber) && pageNumber > 0 ? pageNumber : null,
IndexOnPage = int.TryParse(match.Groups["indexOnPage"].Value, out var indexOnPage) ? indexOnPage : 0,
Quote = match.Groups["quote"].Value
});
}
}
// Remove all content between 【 and 】.
var cleanText = RemoveCitationsRegEx.Replace(text, string.Empty).TrimEnd();
return (cleanText, citations.OrderBy(c => c.FileName).ThenBy(c => c.PageNumber));
}
[GeneratedRegex(@"<citation\s+document-id=(?:""|'|)(?<documentId>[^""']*)(?:""|'|)\s+chunk-id=(?:""|'|)(?<chunkId>[^""']*)(?:""|'|)\s+filename=(?:""|'|)(?<filename>[^""']*)(?:""|'|)\s+page-number=(?:""|'|)(?<pageNumber>[^""']*)(?:""|'|)\s+index-on-page=(?:""|'|)(?<indexOnPage>[^""']*)(?:""|'|)>\s*(?<quote>.*?)\s*</citation>", RegexOptions.Singleline)]
private static partial Regex CitationRegEx { get; }
[GeneratedRegex(@"【.*?】", RegexOptions.Singleline)]
private static partial Regex RemoveCitationsRegEx { get; }
}
}
SqlDatabaseVectorSearch/Settings/AppSettings.cs
-4
View File
@@ -12,10 +12,6 @@ public class AppSettings
public int MaxRelevantChunks { get; init; } = 5;
public int MaxInputTokens { get; init; } = 16385;
public int MaxOutputTokens { get; init; } = 800;
public TimeSpan MessageExpiration { get; init; }
public int MessageLimit { get; set; } = 20;
SqlDatabaseVectorSearch/SqlDatabaseVectorSearch.csproj
+20 -10
View File
@@ -4,7 +4,7 @@
<TargetFramework>net10.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<NoWarn>$(NoWarn);SKEXP0010;SKEXP0050</NoWarn>
<NoWarn>$(NoWarn);SKEXP0010;SKEXP0050;OPENAI001;MAAI001;MEAI001</NoWarn>
</PropertyGroup>
<ItemGroup>
@@ -12,24 +12,34 @@
<PackageReference Include="DocumentFormat.OpenXml" Version="3.5.1" />
<PackageReference Include="EntityFrameworkCore.Exceptions.SqlServer" Version="10.0.1" />
<PackageReference Include="FluentValidation.DependencyInjectionExtensions" Version="12.1.1" />
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="10.0.9" />
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.9" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="10.0.9">
<PackageReference Include="Microsoft.Agents.AI.Hosting" Version="1.15.0-preview.260722.1" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.15.0" />
<PackageReference Include="Microsoft.Agents.AI.Workflows" Version="1.15.0" />
<PackageReference Include="Microsoft.Agents.AI.Workflows.Generators" Version="1.15.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="10.0.10" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Design" Version="10.0.10">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="10.0.10" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="10.0.10">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.Extensions.Caching.Hybrid" Version="10.7.0" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="10.7.0" />
<PackageReference Include="Microsoft.Extensions.Caching.Hybrid" Version="10.8.0" />
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="10.8.0" />
<PackageReference Include="Microsoft.ML.Tokenizers" Version="2.0.0" />
<PackageReference Include="Microsoft.ML.Tokenizers.Data.Cl100kBase" Version="2.0.0" />
<PackageReference Include="Microsoft.ML.Tokenizers.Data.O200kBase" Version="2.0.0" />
<PackageReference Include="Microsoft.SemanticKernel" Version="1.77.0" />
<PackageReference Include="MimeMapping" Version="4.0.0" />
<PackageReference Include="MinimalHelpers.FluentValidation" Version="1.1.8" />
<PackageReference Include="MinimalHelpers.Routing.Analyzers" Version="1.2.2" />
<PackageReference Include="PdfPig" Version="0.1.14" />
<PackageReference Include="Swashbuckle.AspNetCore.SwaggerUI" Version="10.2.1" />
<PackageReference Include="TinyHelpers.AspNetCore" Version="4.2.12" />
<PackageReference Include="PdfPig" Version="0.1.15" />
<PackageReference Include="Swashbuckle.AspNetCore.SwaggerUI" Version="10.2.3" />
<PackageReference Include="TinyHelpers.AspNetCore" Version="4.2.17" />
</ItemGroup>
</Project>
SqlDatabaseVectorSearch/TextChunkers/DefaultTextChunker.cs
+3 -3
View File
@@ -1,7 +1,7 @@
using Microsoft.Extensions.Options;
using Microsoft.SemanticKernel.Text;
using SqlDatabaseVectorSearch.Services;
using SqlDatabaseVectorSearch.Settings;
using SqlDatabaseVectorSearch.TextChunkers.Implementations;
namespace SqlDatabaseVectorSearch.TextChunkers;
@@ -11,8 +11,8 @@ public class DefaultTextChunker(TokenizerService tokenizerService, IOptions<AppS
public IList<string> Split(string text)
{
var lines = TextChunker.SplitPlainTextLines(text, appSettings.MaxTokensPerLine, tokenizerService.CountEmbeddingTokens);
var paragraphs = TextChunker.SplitPlainTextParagraphs(lines, appSettings.MaxTokensPerParagraph, appSettings.OverlapTokens, tokenCounter: tokenizerService.CountEmbeddingTokens);
var lines = PlainTextChunker.SplitPlainTextLines(text, appSettings.MaxTokensPerLine, tokenizerService.CountEmbeddingTokens);
var paragraphs = PlainTextChunker.SplitPlainTextParagraphs(lines, appSettings.MaxTokensPerParagraph, appSettings.OverlapTokens, tokenCounter: tokenizerService.CountEmbeddingTokens);
return paragraphs;
}
SqlDatabaseVectorSearch/TextChunkers/Implementations/PlainTextChunker.cs
+372
View File
@@ -0,0 +1,372 @@
using System.Diagnostics;
using System.Text;
namespace SqlDatabaseVectorSearch.TextChunkers.Implementations;
/// <summary>
/// Split text in chunks, attempting to leave meaning intact.
/// For plain text, split looking at new lines first, then periods, and so on.
/// For markdown, split looking at punctuation first, and so on.
/// </summary>
internal static class PlainTextChunker
{
/// <summary>
/// Represents a list of strings with token count.
/// Used to reduce the number of calls to the tokenizer.
/// </summary>
private sealed class StringListWithTokenCount(TokenCounter? tokenCounter)
{
private readonly TokenCounter? tokenCounter = tokenCounter;
public void Add(string value) => Values.Add((value, tokenCounter is null ? GetDefaultTokenCount(value.Length) : tokenCounter(value)));
public void Add(string value, int tokenCount) => Values.Add((value, tokenCount));
public void AddRange(StringListWithTokenCount range) => Values.AddRange(range.Values);
public void RemoveRange(int index, int count) => Values.RemoveRange(index, count);
public int Count => Values.Count;
public List<string> ToStringList() => Values.Select(v => v.Value).ToList();
private List<(string Value, int TokenCount)> Values { get; } = [];
public string ValueAt(int i) => Values[i].Value;
public int TokenCountAt(int i) => Values[i].TokenCount;
}
/// <summary>
/// Delegate for counting tokens in a string.
/// </summary>
/// <param name="input">The input string to count tokens in.</param>
/// <returns>The number of tokens in the input string.</returns>
public delegate int TokenCounter(string input);
private static readonly string?[] plainTextSplitOptions = ["\n", ".。．", "?!", ";", ":", ",，、", ")]}", " ", "-", null];
private static readonly string?[] markdownSplitOptions = [".\u3002\uFF0E", "?!", ";", ":", ",\uFF0C\u3001", ")]}", " ", "-", "\n\r", null];
/// <summary>
/// Split plain text into lines.
/// </summary>
/// <param name="text">Text to split</param>
/// <param name="maxTokensPerLine">Maximum number of tokens per line.</param>
/// <param name="tokenCounter">Function to count tokens in a string. If not supplied, the default counter will be used.</param>
/// <returns>List of lines.</returns>
public static List<string> SplitPlainTextLines(string text, int maxTokensPerLine, TokenCounter? tokenCounter = null)
{
ArgumentNullException.ThrowIfNull(text);
ValidateMaxTokens(maxTokensPerLine, nameof(maxTokensPerLine));
return InternalSplitLines(text, maxTokensPerLine, trim: true, plainTextSplitOptions, tokenCounter);
}
/// <summary>
/// Split markdown text into lines.
/// </summary>
/// <param name="text">Text to split</param>
/// <param name="maxTokensPerLine">Maximum number of tokens per line.</param>
/// <param name="tokenCounter">Function to count tokens in a string. If not supplied, the default counter will be used.</param>
/// <returns>List of lines.</returns>
public static List<string> SplitMarkdownLines(string text, int maxTokensPerLine, TokenCounter? tokenCounter = null)
{
ArgumentNullException.ThrowIfNull(text);
ValidateMaxTokens(maxTokensPerLine, nameof(maxTokensPerLine));
return InternalSplitLines(text, maxTokensPerLine, trim: true, markdownSplitOptions, tokenCounter);
}
/// <summary>
/// Split plain text into paragraphs.
/// </summary>
/// <param name="lines">Lines of text.</param>
/// <param name="maxTokensPerParagraph">Maximum number of tokens per paragraph.</param>
/// <param name="overlapTokens">Number of tokens to overlap between paragraphs.</param>
/// <param name="chunkHeader">Text to be prepended to each individual chunk.</param>
/// <param name="tokenCounter">Function to count tokens in a string. If not supplied, the default counter will be used.</param>
/// <returns>List of paragraphs.</returns>
public static List<string> SplitPlainTextParagraphs(IEnumerable<string> lines, int maxTokensPerParagraph, int overlapTokens = 0, string? chunkHeader = null, TokenCounter? tokenCounter = null)
=> InternalSplitTextParagraphs(lines, maxTokensPerParagraph, overlapTokens, chunkHeader,
static (text, maxTokens, tokenCounter) => InternalSplitLines(text, maxTokens, trim: false, plainTextSplitOptions, tokenCounter), tokenCounter);
/// <summary>
/// Split markdown text into paragraphs.
/// </summary>
/// <param name="lines">Lines of text.</param>
/// <param name="maxTokensPerParagraph">Maximum number of tokens per paragraph.</param>
/// <param name="overlapTokens">Number of tokens to overlap between paragraphs.</param>
/// <param name="chunkHeader">Text to be prepended to each individual chunk.</param>
/// <param name="tokenCounter">Function to count tokens in a string. If not supplied, the default counter will be used.</param>
/// <returns>List of paragraphs.</returns>
public static List<string> SplitMarkdownParagraphs(IEnumerable<string> lines, int maxTokensPerParagraph, int overlapTokens = 0, string? chunkHeader = null, TokenCounter? tokenCounter = null)
=> InternalSplitTextParagraphs(lines, maxTokensPerParagraph, overlapTokens, chunkHeader,
static (text, maxTokens, tokenCounter) => InternalSplitLines(text, maxTokens, trim: false, markdownSplitOptions, tokenCounter), tokenCounter);
private static List<string> InternalSplitTextParagraphs(IEnumerable<string> lines, int maxTokensPerParagraph, int overlapTokens, string? chunkHeader, Func<string, int, TokenCounter?, List<string>> longLinesSplitter, TokenCounter? tokenCounter)
{
ArgumentNullException.ThrowIfNull(lines);
ValidateMaxTokens(maxTokensPerParagraph, nameof(maxTokensPerParagraph));
if (overlapTokens < 0)
{
throw new ArgumentOutOfRangeException(nameof(overlapTokens), "overlapTokens cannot be negative.");
}
if (maxTokensPerParagraph <= overlapTokens)
{
throw new ArgumentException("overlapTokens cannot be larger than or equal to maxTokensPerParagraph.", nameof(overlapTokens));
}
// Optimize empty inputs if we can efficiently determine they're empty.
if (lines is ICollection<string> c && c.Count == 0)
{
return [];
}
var chunkHeaderTokens = chunkHeader is { Length: > 0 } ? GetTokenCount(chunkHeader, tokenCounter) : 0;
var adjustedMaxTokensPerParagraph = maxTokensPerParagraph - overlapTokens - chunkHeaderTokens;
if (adjustedMaxTokensPerParagraph <= 0)
{
throw new ArgumentException("chunkHeader and overlapTokens must leave room for paragraph content.", nameof(chunkHeader));
}
// Split long lines first
var truncatedLines = lines.SelectMany(line => longLinesSplitter(NormalizeLineEndings(line), adjustedMaxTokensPerParagraph, tokenCounter));
var paragraphs = BuildParagraph(truncatedLines, adjustedMaxTokensPerParagraph, tokenCounter);
var processedParagraphs = ProcessParagraphs(paragraphs, adjustedMaxTokensPerParagraph, overlapTokens, chunkHeader, longLinesSplitter, tokenCounter);
return processedParagraphs;
}
private static List<string> BuildParagraph(IEnumerable<string> truncatedLines, int maxTokensPerParagraph, TokenCounter? tokenCounter)
{
StringBuilder paragraphBuilder = new();
List<string> paragraphs = [];
foreach (var line in truncatedLines)
{
if (paragraphBuilder.Length > 0)
{
string? paragraph = null;
var currentCount = GetTokenCount(line, tokenCounter) + 1;
if (currentCount < maxTokensPerParagraph)
{
currentCount += tokenCounter is null ?
GetDefaultTokenCount(paragraphBuilder.Length) :
tokenCounter(paragraph = paragraphBuilder.ToString());
}
if (currentCount >= maxTokensPerParagraph)
{
// Complete the paragraph and prepare for the next
paragraph ??= paragraphBuilder.ToString();
paragraphs.Add(paragraph.Trim());
paragraphBuilder.Clear();
}
}
paragraphBuilder.AppendLine(line);
}
if (paragraphBuilder.Length > 0)
{
// Add the final paragraph if there's anything remaining
paragraphs.Add(paragraphBuilder.ToString().Trim());
}
return paragraphs;
}
private static List<string> ProcessParagraphs(List<string> paragraphs, int adjustedMaxTokensPerParagraph, int overlapTokens, string? chunkHeader, Func<string, int, TokenCounter?, List<string>> longLinesSplitter, TokenCounter? tokenCounter)
{
// distribute text more evenly in the last paragraphs when the last paragraph is too short.
if (paragraphs.Count > 1)
{
var lastParagraph = paragraphs[^1];
var secondLastParagraph = paragraphs[^2];
if (GetTokenCount(lastParagraph, tokenCounter) < adjustedMaxTokensPerParagraph / 4)
{
var mergedParagraph = $"{secondLastParagraph} {lastParagraph}";
if (GetTokenCount(mergedParagraph, tokenCounter) <= adjustedMaxTokensPerParagraph)
{
paragraphs[^2] = mergedParagraph;
paragraphs.RemoveAt(paragraphs.Count - 1);
}
}
}
var processedParagraphs = new List<string>();
var paragraphStringBuilder = new StringBuilder();
for (var i = 0; i < paragraphs.Count; i++)
{
paragraphStringBuilder.Clear();
if (chunkHeader is not null)
{
paragraphStringBuilder.Append(chunkHeader);
}
var paragraph = paragraphs[i];
if (overlapTokens > 0 && i < paragraphs.Count - 1)
{
var nextParagraph = paragraphs[i + 1];
var split = longLinesSplitter(nextParagraph, overlapTokens, tokenCounter);
paragraphStringBuilder.Append(paragraph);
if (split.Count != 0)
{
paragraphStringBuilder.Append(' ').Append(split[0]);
}
}
else
{
paragraphStringBuilder.Append(paragraph);
}
processedParagraphs.Add(paragraphStringBuilder.ToString());
}
return processedParagraphs;
}
private static List<string> InternalSplitLines(string text, int maxTokensPerLine, bool trim, string?[] splitOptions, TokenCounter? tokenCounter)
{
var result = new StringListWithTokenCount(tokenCounter);
text = NormalizeLineEndings(text);
result.Add(text);
for (var i = 0; i < splitOptions.Length; i++)
{
var count = result.Count; // track where the original input left off
var (splits2, inputWasSplit2) = Split(result, maxTokensPerLine, splitOptions[i].AsSpan(), trim, tokenCounter);
result.AddRange(splits2);
result.RemoveRange(0, count); // remove the original input
if (!inputWasSplit2)
{
break;
}
}
return result.ToStringList();
}
private static (StringListWithTokenCount, bool) Split(StringListWithTokenCount input, int maxTokens, ReadOnlySpan<char> separators, bool trim, TokenCounter? tokenCounter)
{
var inputWasSplit = false;
StringListWithTokenCount result = new(tokenCounter);
var count = input.Count;
for (var i = 0; i < count; i++)
{
var (splits, split) = Split(input.ValueAt(i).AsSpan(), input.ValueAt(i), maxTokens, separators, trim, tokenCounter, input.TokenCountAt(i));
result.AddRange(splits);
inputWasSplit |= split;
}
return (result, inputWasSplit);
}
private static (StringListWithTokenCount, bool) Split(ReadOnlySpan<char> input, string? inputString, int maxTokens, ReadOnlySpan<char> separators, bool trim, TokenCounter? tokenCounter, int inputTokenCount)
{
Debug.Assert(inputString is null || input.SequenceEqual(inputString.AsSpan()));
StringListWithTokenCount result = new(tokenCounter);
var inputWasSplit = false;
if (inputTokenCount > maxTokens)
{
inputWasSplit = true;
var half = input.Length / 2;
var cutPoint = -1;
if (separators.IsEmpty)
{
cutPoint = half;
}
else if (input.Length > 2)
{
var pos = 0;
while (true)
{
var index = input[pos..^1].IndexOfAny(separators);
if (index < 0)
{
break;
}
index += pos;
if (Math.Abs(half - index) < Math.Abs(half - cutPoint))
{
cutPoint = index + 1;
}
pos = index + 1;
}
}
if (cutPoint > 0)
{
var firstHalf = input[..cutPoint];
var secondHalf = input[cutPoint..];
if (trim)
{
firstHalf = firstHalf.Trim();
secondHalf = secondHalf.Trim();
}
// Recursion
var (splits1, split1) = Split(firstHalf, null, maxTokens, separators, trim, tokenCounter, GetTokenCount(firstHalf, tokenCounter));
result.AddRange(splits1);
var (splits2, split2) = Split(secondHalf, null, maxTokens, separators, trim, tokenCounter, GetTokenCount(secondHalf, tokenCounter));
result.AddRange(splits2);
inputWasSplit = split1 || split2;
return (result, inputWasSplit);
}
}
var resultString = inputString ?? input.ToString();
var resultTokenCount = inputTokenCount;
if (trim)
{
var trimmedResult = resultString.Trim();
if (!trimmedResult.Equals(resultString, StringComparison.Ordinal))
{
resultString = trimmedResult;
resultTokenCount = GetTokenCount(resultString, tokenCounter);
}
}
result.Add(resultString, resultTokenCount);
return (result, inputWasSplit);
}
private static int GetTokenCount(string input, TokenCounter? tokenCounter) => tokenCounter is null ? GetDefaultTokenCount(input.Length) : tokenCounter(input);
private static int GetTokenCount(ReadOnlySpan<char> input, TokenCounter? tokenCounter) => tokenCounter is null ? GetDefaultTokenCount(input.Length) : tokenCounter(input.ToString());
private static string NormalizeLineEndings(string text) => text.Replace("\r\n", "\n").Replace('\r', '\n');
private static void ValidateMaxTokens(int maxTokens, string parameterName)
{
if (maxTokens <= 0)
{
throw new ArgumentOutOfRangeException(parameterName, "The maximum token count must be a positive number.");
}
}
private static int GetDefaultTokenCount(int length)
{
Debug.Assert(length >= 0);
return length == 0 ? 0 : Math.Max(1, length >> 2);
}
}
SqlDatabaseVectorSearch/TextChunkers/MarkdownTextChunker.cs
+3 -3
View File
@@ -1,7 +1,7 @@
using Microsoft.Extensions.Options;
using Microsoft.SemanticKernel.Text;
using SqlDatabaseVectorSearch.Services;
using SqlDatabaseVectorSearch.Settings;
using SqlDatabaseVectorSearch.TextChunkers.Implementations;
namespace SqlDatabaseVectorSearch.TextChunkers;
@@ -11,8 +11,8 @@ public class MarkdownTextChunker(TokenizerService tokenizerService, IOptions<App
public IList<string> Split(string text)
{
var lines = TextChunker.SplitMarkDownLines(text, appSettings.MaxTokensPerLine, tokenizerService.CountEmbeddingTokens);
var paragraphs = TextChunker.SplitMarkdownParagraphs(lines, appSettings.MaxTokensPerParagraph, appSettings.OverlapTokens, tokenCounter: tokenizerService.CountEmbeddingTokens);
var lines = PlainTextChunker.SplitMarkdownLines(text, appSettings.MaxTokensPerLine, tokenizerService.CountEmbeddingTokens);
var paragraphs = PlainTextChunker.SplitMarkdownParagraphs(lines, appSettings.MaxTokensPerParagraph, appSettings.OverlapTokens, tokenCounter: tokenizerService.CountEmbeddingTokens);
return paragraphs;
}
SqlDatabaseVectorSearch/Workflows/EmbeddingRequest.cs
+26
View File
@@ -0,0 +1,26 @@
namespace SqlDatabaseVectorSearch.Workflows;
public record class EmbeddingRequest(Stream Content, string FileName, string ContentType, Guid? DocumentId)
{
/// <summary>
/// Creates an <see cref="EmbeddingRequest"/> from an uploaded <see cref="IFormFile"/>.
/// </summary>
/// <param name="file">The uploaded file.</param>
/// <param name="documentId">The optional identifier of the document to overwrite.</param>
public static EmbeddingRequest FromFormFile(IFormFile file, Guid? documentId = null) => Create(file.OpenReadStream(), Path.GetFileName(file.FileName), documentId);
/// <summary>
/// Creates an <see cref="EmbeddingRequest"/> from a content stream, inferring the content type from the file name.
/// </summary>
/// <param name="content">The stream that contains the document content.</param>
/// <param name="fileName">The name of the document.</param>
/// <param name="documentId">The optional identifier of the document to overwrite.</param>
/// <remarks>
/// The content type is inferred from the file name because the content type declared by the client is not always reliable (for example, for Markdown files).
/// </remarks>
public static EmbeddingRequest Create(Stream content, string fileName, Guid? documentId = null)
{
var name = Path.GetFileName(fileName);
return new EmbeddingRequest(content, name, MimeMapping.MimeUtility.GetMimeMapping(name), documentId);
}
}
SqlDatabaseVectorSearch/Workflows/ExtractChunksExecutor.cs
+21
View File
@@ -0,0 +1,21 @@
using Microsoft.Agents.AI.Workflows;
using SqlDatabaseVectorSearch.ContentDecoders;
namespace SqlDatabaseVectorSearch.Workflows;
public partial class ExtractChunksExecutor(IServiceProvider serviceProvider, ILogger<ExtractChunksExecutor> logger) : Executor(nameof(ExtractChunksExecutor))
{
[MessageHandler]
private async ValueTask<ExtractChunksResponse> HandleAsync(EmbeddingRequest request, IWorkflowContext context, CancellationToken cancellationToken)
{
// Extract the contents of the file.
var decoder = serviceProvider.GetKeyedService<IContentDecoder>(request.ContentType) ?? throw new NotSupportedException($"Content type '{request.ContentType}' is not supported.");
var chunks = await decoder.DecodeAsync(request.Content, request.ContentType, cancellationToken);
logger.LogDebug("Extracted {Count} chunks from '{FileName}'.", chunks.Count(), request.FileName);
return new(request, chunks);
}
}
public record class ExtractChunksResponse(EmbeddingRequest Request, IEnumerable<Chunk> Chunks);
SqlDatabaseVectorSearch/Workflows/GenerateEmbeddingExecutor.cs
+37
View File
@@ -0,0 +1,37 @@
using Microsoft.Agents.AI.Workflows;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Options;
using SqlDatabaseVectorSearch.ContentDecoders;
using SqlDatabaseVectorSearch.Services;
using SqlDatabaseVectorSearch.Settings;
namespace SqlDatabaseVectorSearch.Workflows;
public partial class GenerateEmbeddingExecutor(IEmbeddingGenerator<string, Embedding<float>> embeddingGenerator, TokenizerService tokenizerService, IOptions<AppSettings> appSettingsOptions, ILogger<GenerateEmbeddingExecutor> logger) : Executor(nameof(GenerateEmbeddingExecutor))
{
private readonly AppSettings appSettings = appSettingsOptions.Value;
[MessageHandler]
private async ValueTask<EmbeddingResponse> HandleAsync(ExtractChunksResponse chunks, IWorkflowContext context, CancellationToken cancellationToken)
{
var chunkContents = chunks.Chunks.Select(p => p.Content).ToList();
// We get the token count of the whole document because it is the total number of tokens used by the embedding (it may be necessary, for example, for cost analysis).
var tokenCount = tokenizerService.CountEmbeddingTokens(string.Join(" ", chunkContents));
// Process paragraphs in batches.
var embeddings = new List<Embedding<float>>();
foreach (var batch in chunkContents.Chunk(appSettings.EmbeddingBatchSize))
{
logger.LogDebug("Processing batch of {Count} chunks for embedding generation...", batch.Length);
// Generate embeddings for this batch.
var batchEmbeddings = await embeddingGenerator.GenerateAsync(batch, cancellationToken: cancellationToken);
embeddings.AddRange(batchEmbeddings);
}
return new EmbeddingResponse(chunks.Request, chunks.Chunks, embeddings, tokenCount);
}
}
public record class EmbeddingResponse(EmbeddingRequest Request, IEnumerable<Chunk> Chunks, IEnumerable<Embedding<float>> Embeddings, int TokenCount);
SqlDatabaseVectorSearch/Workflows/StoreEmbeddingExecutor.cs
+58
View File
@@ -0,0 +1,58 @@
using Microsoft.Agents.AI.Workflows;
using Microsoft.Data.SqlTypes;
using Microsoft.EntityFrameworkCore;
using SqlDatabaseVectorSearch.Data;
using SqlDatabaseVectorSearch.Services;
using Entities = SqlDatabaseVectorSearch.Data.Entities;
namespace SqlDatabaseVectorSearch.Workflows;
public partial class StoreEmbeddingExecutor(ApplicationDbContext dbContext, DocumentService documentService, TokenizerService tokenizerService, TimeProvider timeProvider, ILogger<StoreEmbeddingExecutor> logger) : Executor(nameof(StoreEmbeddingExecutor))
{
[MessageHandler]
private async ValueTask<StoreEmbeddingResponse> HandleAsync(EmbeddingResponse embeddingData, IWorkflowContext context, CancellationToken cancellationToken)
{
var strategy = dbContext.Database.CreateExecutionStrategy();
var document = await strategy.ExecuteAsync(async (cancellationToken) =>
{
await dbContext.Database.BeginTransactionAsync(cancellationToken);
if (embeddingData.Request.DocumentId.HasValue)
{
// If the user is importing a document that already exists, delete the previous one.
await documentService.DeleteAsync(embeddingData.Request.DocumentId.Value, cancellationToken);
}
var document = new Entities.Document { Id = embeddingData.Request.DocumentId.GetValueOrDefault(), Name = embeddingData.Request.FileName, CreationDate = timeProvider.GetUtcNow() };
dbContext.Documents.Add(document);
// Save the document chunks and the corresponding embedding in the database.
foreach (var (index, embedding) in embeddingData.Embeddings.Index())
{
var chunk = embeddingData.Chunks.ElementAt(index);
logger.LogDebug("Storing a chunk of {TokenCount} tokens.", tokenizerService.CountEmbeddingTokens(chunk.Content));
var documentChunk = new Entities.DocumentChunk
{
Document = document,
Index = index,
PageNumber = chunk.PageNumber,
IndexOnPage = chunk.IndexOnPage,
Content = chunk.Content,
Embedding = new SqlVector<float>(embedding.Vector)
};
dbContext.DocumentChunks.Add(documentChunk);
}
await dbContext.SaveChangesAsync(cancellationToken);
await dbContext.Database.CommitTransactionAsync(cancellationToken);
return document;
}, cancellationToken);
return new(document.Id, embeddingData.TokenCount);
}
}
public record class StoreEmbeddingResponse(Guid DocumentId, int TokenCount);
SqlDatabaseVectorSearch/appsettings.json
+2 -4
View File
@@ -6,7 +6,7 @@
"ChatCompletion": {
"Endpoint": "",
"Deployment": "",
"ModelId": "", // gpt-4o, gpt-4, gpt-3.5, etc. Note that for gpt-4.1 and gpt-5 models, the ModelId must be set to gpt-4o.
"ModelId": "", // Used only for token counting: gpt-5, gpt-4.1, gpt-4o, gpt-4, gpt-3.5-turbo, etc. If the model isn't supported by Microsoft.ML.Tokenizers, use the closest one (for example gpt-4o).
"ApiKey": ""
},
"Embedding": {
@@ -24,9 +24,7 @@
"MaxTokensPerLine": 300,
"MaxTokensPerParagraph": 1000,
"OverlapTokens": 100,
"MaxRelevantChunks": 50,
"MaxInputTokens": 32768,
"MaxOutputTokens": 800,
"MaxRelevantChunks": 30,
"MessageExpiration": "00:05:00",
"MessageLimit": 20
},
SqlDatabaseVectorSearch/wwwroot/css/app.css
+51 -20
View File
@@ -7,34 +7,65 @@ body, html {
:root {
--bb-sidebar2-width: 270px;
--bb-sidebar2-collapsed-width: 50px;
--bb-sidebar2-background-color: rgba(234, 234, 234, 1);
--bb-sidebar2-top-row-background-color: rgba(0,0,0,0.08);
--bb-sidebar2-top-row-border-color: rgb(194,192,192);
--bb-sidebar2-title-text-color: rgb(0,0,0);
--bb-sidebar2-brand-icon-color: rgb(0,0,0);
--bb-sidebar2-background-color: transparent;
--bb-sidebar2-top-row-background-color: transparent;
--bb-sidebar2-top-row-border-color: rgba(13,110,253,0.12);
--bb-sidebar2-title-text-color: #1b2436;
--bb-sidebar2-brand-icon-color: #0d6efd;
--bb-sidebar2-brand-image-width: 24px;
--bb-sidebar2-brand-image-height: 24px;
--bb-sidebar2-title-badge-text-color: rgb(255,255,255);
--bb-sidebar2-title-badge-background-color: rgba(25,135,84,var(--bs-bg-opacity,1));
--bb-sidebar2-navbar-toggler-icon-color: rgb(0,0,0);
--bb-sidebar2-navbar-toggler-background-color: rgba(0,0,0,0.08);
--bb-sidebar2-content-border-color: rgb(194,192,192);
--bb-sidebar2-nav-item-text-color: rgba(0,0,0,0.9);
--bb-sidebar2-nav-item-text-active-color-rgb: 0,0,0;
--bb-sidebar2-nav-item-text-hover-color: rgba(var(--bb-sidebar-nav-item-text-active-color-rgb),0.9);
--bb-sidebar2-nav-item-text-active-color: rgba(var(--bb-sidebar-nav-item-text-active-color-rgb),0.9);
--bb-sidebar2-nav-item-background-hover-color: rgba(var(--bb-sidebar-nav-item-text-active-color-rgb),0.08);
--bb-sidebar2-nav-item-group-background-color: rgba(var(--bb-sidebar-nav-item-text-active-color-rgb),0.08);
--bb-sidebar2-navbar-toggler-icon-color: #1b2436;
--bb-sidebar2-navbar-toggler-background-color: rgba(13,110,253,0.08);
--bb-sidebar2-content-border-color: rgba(13,110,253,0.12);
--bb-sidebar2-nav-item-text-color: #495867;
--bb-sidebar2-nav-item-text-active-color-rgb: 27,36,54;
--bb-sidebar2-nav-item-text-hover-color: #0d6efd;
--bb-sidebar2-nav-item-text-active-color: #0d6efd;
--bb-sidebar2-nav-item-background-hover-color: rgba(13,110,253,0.08);
--bb-sidebar2-nav-item-group-background-color: rgba(13,110,253,0.06);
}
.bb-sidebar2 nav .nav-item a:hover {
background-color: rgba(0,0,0,0.08) !important;
color: rgba(0,0,0,0.9) !important;
/* A soft light gradient gives the sidebar a modern look while keeping the content area in focus. */
.bb-sidebar2 {
background: linear-gradient(180deg, #ffffff 0%, #eef2f8 100%);
border-right: 1px solid rgba(13,110,253,0.1);
box-shadow: 2px 0 12px rgba(27,36,54,0.06);
}
.bb-sidebar2 nav .nav-item a.active {
background-color: rgb(194,192,192) !important;
color: rgba(0,0,0,0.9) !important;
.bb-sidebar2 nav .nav-item {
margin: 0.125rem 0.5rem;
}
.bb-sidebar2 nav .nav-item a {
border-radius: 0.5rem;
transition: background-color 0.15s ease-in-out, color 0.15s ease-in-out;
}
.bb-sidebar2 nav .nav-item a:hover {
background-color: rgba(13,110,253,0.08) !important;
color: #0d6efd !important;
}
.bb-sidebar2 nav .nav-item a.active {
background: linear-gradient(90deg, rgba(13,110,253,0.16) 0%, rgba(13,110,253,0.04) 100%) !important;
color: #0d6efd !important;
font-weight: 600;
box-shadow: inset 3px 0 0 #0d6efd;
}
/* The nav item icons inherit the link color, so they are colored individually. */
.bb-sidebar2 nav .nav-item a .bi-house-door-fill {
color: #0d6efd;
}
.bb-sidebar2 nav .nav-item a .bi-file-earmark-text-fill {
color: #0aa2c0;
}
.bb-sidebar2 nav .nav-item a .bi-chat-dots-fill {
color: #198754;
}
h1:focus {
assets/SqlDatabaseVectorSearch.mp4
BIN
View File
Binary file not shown.
assets/SqlDatabaseVectorSearch_WebApp.png
BIN
View File
Binary file not shown.
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 119 KiB

After

Width:  |  Height:  |  Size: 111 KiB