Mirrors/SqlDatabaseVectorSearch
1
0
Fork 0
mirror of https://github.com/marcominerva/SqlDatabaseVectorSearch.git synced 2026-08-04 09:48:57 +00:00
Code Issues Packages Projects Releases Wiki Activity

Switch to Agent Framework, update SSE protocol and docs

Browse Source 
Updated README.md and Home.razor to document migration from Semantic Kernel to Microsoft Agent Framework, detailing new orchestration, document import, question reformulation, and RAG agent workflows. Revised API and streaming response examples to use Server-Sent Events (SSE) with event names Start, Delta, and End. Refactored AskEndpoints.cs to use .NET 8 ServerSentEvents API and yield SseItem<Response> with correct event names. Updated Ask.razor, Documents.razor, and VectorSearchService.cs to use StreamState.Delta instead of Append, and replaced StateHasChanged() with await InvokeAsync(StateHasChanged) for async UI updates. Removed MaxInputTokens and MaxOutputTokens from AppSettings.cs and appsettings.json.
This commit is contained in:
Marco Minerva
2026-07-27 17:44:23 +02:00
parent d5b0e1606f
commit c7138d571c
9 changed files with 84 additions and 242 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+53 -208
View File
@@ -4,7 +4,7 @@
[![Minimal API](https://img.shields.io/badge/Minimal%20API-Available-green)](https://dotnet.microsoft.com/apps/aspnet/apis) [![Minimal API](https://img.shields.io/badge/Minimal%20API-Available-green)](https://dotnet.microsoft.com/apps/aspnet/apis)
[![Blazor](https://img.shields.io/badge/Blazor-WebApp-purple)](https://dotnet.microsoft.com/apps/aspnet/web-apps/blazor) [![Blazor](https://img.shields.io/badge/Blazor-WebApp-purple)](https://dotnet.microsoft.com/apps/aspnet/web-apps/blazor)
A Blazor Web App and Minimal API for performing RAG (Retrieval Augmented Generation) and vector search using the native VECTOR type in Azure SQL Database and Azure OpenAI. A Blazor Web App and Minimal API for performing RAG (Retrieval Augmented Generation) and vector search using the native VECTOR type in Azure SQL Database, Azure OpenAI, and [Microsoft Agent Framework](https://github.com/microsoft/agent-framework).
## Table of Contents ## Table of Contents
- [Overview](#overview) - [Overview](#overview)
@@ -24,10 +24,10 @@ A Blazor Web App and Minimal API for performing RAG (Retrieval Augmented Generat
This application allows you to: This application allows you to:
- Load documents (PDF, DOCX, TXT, MD) - Load documents (PDF, DOCX, TXT, MD)
- Generate embeddings and save them as vectors in Azure SQL Database - Generate embeddings and save them as vectors in Azure SQL Database
- Perform semantic search and RAG using Azure OpenAI - Perform semantic search and RAG using Azure OpenAI and Microsoft Agent Framework agents
- Interact via a Blazor Web App or programmatically via Minimal API - Interact via a Blazor Web App or programmatically via Minimal API
Embeddings and chat completion are powered by [Semantic Kernel](https://github.com/microsoft/semantic-kernel). Embeddings and chat completion are orchestrated with [Microsoft Agent Framework](https://github.com/microsoft/agent-framework). The application uses an embedding workflow to import documents, a reformulation agent to rewrite follow-up questions with conversation context, and a RAG agent connected to a SQL vector-search context provider.
## Screenshots ## Screenshots
@@ -49,6 +49,7 @@ Embeddings and chat completion are powered by [Semantic Kernel](https://github.c
- `Endpoints/` - Minimal API endpoints - `Endpoints/` - Minimal API endpoints
- `Services/` - Business logic and integration services - `Services/` - Business logic and integration services
- `TextChunkers/` - Text splitting utilities - `TextChunkers/` - Text splitting utilities
- `Workflows/` - Microsoft Agent Framework workflow executors for document import and embedding generation
- `Settings/` - Configuration classes - `Settings/` - Configuration classes
## Setup ## Setup
@@ -76,16 +77,26 @@ Embeddings and chat completion are powered by [Semantic Kernel](https://github.c
## Supported features ## Supported features
- **Conversation History with Question Reformulation**: This feature allows users to view the history of their conversations, including the ability to reformulate questions for better clarity and understanding. This ensures that users can track their interactions and refine their queries as needed. - **Microsoft Agent Framework orchestration**: Document import is implemented as a workflow, while question reformulation and RAG are implemented as agents.
- **Information about Token Usage**: Users can access detailed information about token usage, which helps in understanding the consumption of tokens during interactions. This feature provides transparency and helps users manage their token usage effectively. - **Conversation history with question reformulation**: The reformulation agent rewrites each question using the conversation context before vector search is performed.
- **Response Streaming**: This feature enables real-time streaming of responses, allowing users to receive information as it is being processed. This ensures a seamless and efficient flow of information, enhancing the overall user experience. - **SQL vector-search context provider**: The RAG agent receives relevant chunks from Azure SQL Database through a `TextSearchProvider` backed by native VECTOR search.
- **Citations**: The application provides citations for the sources used to justify each answer. This allows users to verify the information and understand the origin of the content provided by the system. - **Information about token usage**: The Blazor chat page and API responses expose token usage for reformulation and final answer generation.
- **Response streaming**: The Blazor chat page uses streaming responses, appending answer tokens as they arrive.
- **Markdown source citations**: Citations are included directly in the generated Markdown answer as a localized sources section with source name, page number when available, and a short supporting excerpt.
## How to Use ## How to Use
- **Web App**: Use the Blazor interface to upload documents, search, and chat with RAG. - **Web App**: Use the Blazor interface to manage documents and chat with your indexed content. The chat page streams answers, shows token usage, supports conversation reset, and renders source citations as part of the Markdown answer.
- **API**: Import documents via `POST /api/documents` and ask questions via `POST /api/ask` or `POST /api/ask-streaming`. - **API**: Import documents via `POST /api/documents` and ask questions via `POST /api/ask` or `POST /api/ask-streaming`.
### How it works
1. Documents are uploaded through the API and processed by the `EmbeddingWorkflow`.
2. The workflow converts the uploaded file into text, chunks it, generates embeddings, and stores documents, chunks, and VECTOR embeddings in Azure SQL Database.
3. When a question is asked, the `ReformulationAgent` can rewrite it using the current conversation context.
4. The `RagAgent` receives relevant SQL vector-search results through a `TextSearchProvider` and answers using only the provided context.
5. Sources are not returned as a separate JSON collection. They are formatted directly in the Markdown answer.
#### Example API Request #### Example API Request
``` ```
POST /api/ask POST /api/ask
@@ -101,224 +112,58 @@ Content-Type: application/json
```json ```json
{ {
"conversationId": "3d0bd178-499d-433a-b2bc-c35e488d9e2c",
"originalQuestion": "why is mars called the red planet?", "originalQuestion": "why is mars called the red planet?",
"reformulatedQuestion": "Why is the planet Mars called the red planet?", "reformulatedQuestion": "Why is the planet Mars called the red planet?",
"answer": "Mars is called the Red Planet because its surface has an orange-red color due to being covered in iron(III) oxide dust, also known as rust. This iron oxide gives Mars its distinctive reddish appearance when observed from Earth and is the origin of its well-known nickname", "answer": "Mars is called the Red Planet because its surface has an orange-red color caused by iron oxide dust.\n\n*Sources*\n1. **Mars.pdf**, page 1: *surface of Mars is orange-red because it is covered in iron oxide dust*",
"streamState": "End", "streamState": null,
"tokenUsage": { "tokenUsage": {
"reformulation": { "reformulation": {
"promptTokens": 812, "inputTokenCount": 812,
"completionTokens": 11, "outputTokenCount": 11,
"totalTokens": 823 "totalTokenCount": 823
}, },
"embeddingTokenCount": 10,
"question": { "question": {
"promptTokens": 31708, "inputTokenCount": 31708,
"completionTokens": 227, "outputTokenCount": 227,
"totalTokens": 31935 "totalTokenCount": 31935
} }
},
"citations": [
{
"documentId": "b1870ad7-4685-42a3-576a-08ddb01159d5",
"chunkId": "749aba1e-0db5-4033-cfa6-08ddb0115da3",
"fileName": "Mars.pdf",
"quote": "surface of Mars is orange-red because it is covered in iron(III) oxide",
"pageNumber": 1,
"indexOnPage": 0
},
{
"documentId": "b1870ad7-4685-42a3-576a-08ddb01159d5",
"chunkId": "215e7197-513f-4fbe-cfa8-08ddb0115da3",
"fileName": "Mars.pdf",
"quote": "Martian surface is caused by ferric oxide, or rust",
"pageNumber": 3,
"indexOnPage": 0
} }
]
} }
``` ```
### How response streaming works ### How response streaming works
When using the `/api/ask-streaming` endpoint, answers will be streamed as with the typical response from OpenAI. The format of the response is as follows: When using the `/api/ask-streaming` endpoint, answers are streamed as [Server-Sent Events](https://developer.mozilla.org/docs/Web/API/Server-sent_events). Each event has a name matching the `streamState` value and a JSON `Response` payload in the `data` field. The format is as follows:
```json ```text
[ event: Start
{ data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":"why is mars called the red planet?","reformulatedQuestion":"Why is the planet Mars known as the red planet?","answer":null,"streamState":"Start","tokenUsage":{"reformulation":{"inputTokenCount":541,"outputTokenCount":12,"totalTokenCount":553,"cachedInputTokenCount":0,"reasoningTokenCount":0,"inputAudioTokenCount":null,"inputTextTokenCount":null,"outputAudioTokenCount":null,"outputTextTokenCount":null,"additionalCounts":null},"question":null}}
"originalQuestion": "why is mars called the red planet?",
"reformulatedQuestion": "Why is the planet Mars known as the red planet?", event: Delta
"answer": null, data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":null,"reformulatedQuestion":null,"answer":"Mars","streamState":"Delta","tokenUsage":null}
"streamState": "Start",
"tokenUsage": { event: Delta
"reformulation": { data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":null,"reformulatedQuestion":null,"answer":" is known as the red planet because its surface is rich in iron oxide dust.\n\n","streamState":"Delta","tokenUsage":null}
"promptTokens": 541,
"completionTokens": 12, event: Delta
"totalTokens": 553 data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":null,"reformulatedQuestion":null,"answer":"Sources\n1. **Mars.pdf**, page 1: *surface of Mars is orange-red because it is covered in iron oxide dust*","streamState":"Delta","tokenUsage":null}
},
"embeddingTokenCount": 11, event: End
"question": null data: {"conversationId":"3d0bd178-499d-433a-b2bc-c35e488d9e2c","originalQuestion":null,"reformulatedQuestion":null,"answer":null,"streamState":"End","tokenUsage":{"reformulation":null,"question":{"inputTokenCount":30949,"outputTokenCount":221,"totalTokenCount":31170,"cachedInputTokenCount":3840,"reasoningTokenCount":0,"inputAudioTokenCount":null,"inputTextTokenCount":null,"outputAudioTokenCount":null,"outputTextTokenCount":null,"additionalCounts":null}}}
},
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": "Mars",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " is",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " known",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " as",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " the",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " red",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " planet",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " because",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " its",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " surface",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " is",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " covered",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " in",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " iron",
"streamState": "Append",
"tokenUsage": null,
"citations": null
},
/// ...
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": null,
"streamState": "End",
"tokenUsage": {
"reformulation": null,
"embeddingTokenCount": null,
"question": {
"promptTokens": 30949,
"completionTokens": 221,
"totalTokens": 31170
}
},
"citations": [
{
"documentId": "b1870ad7-4685-42a3-576a-08ddb01159d5",
"chunkId": "749aba1e-0db5-4033-cfa6-08ddb0115da3",
"fileName": "Mars.pdf",
"quote": "surface of Mars is orange-red",
"pageNumber": 1,
"indexOnPage": 0
},
{
"documentId": "b1870ad7-4685-42a3-576a-08ddb01159d5",
"chunkId": "215e7197-513f-4fbe-cfa8-08ddb0115da3",
"fileName": "Mars.pdf",
"quote": "red-orange appearance of the Martian surface is caused by ferric oxide, or rust",
"pageNumber": 3,
"indexOnPage": 0
}
]
}
]
``` ```
- The first piece of the response has the following characteristics: - The first event has the following characteristics:
- The SSE event name is `Start`.
- The *streamState* property is set to `Start`. - The *streamState* property is set to `Start`.
- It contains the question and its reformulation (if not requested, *reformulatedQuestion* will be equal to *originalQuestion*). - It contains the question and its reformulation (if not requested, *reformulatedQuestion* will be equal to *originalQuestion*).
- The *tokenUsage* section holds information about tokens used for reformulation (if done) and for the embedding of the question. - The *tokenUsage* section holds information about tokens used for reformulation, if done.
- Then, there are as many elements for the actual answer as necessary: - Then, there are as many `Delta` events as necessary for the actual answer:
- Each one contains a token. - Each event contains a token or chunk of generated text in the *answer* property.
- The *streamState* property is set to `Append`. - The *streamState* property is set to `Delta`.
- *originalQuestion*, *reformulatedQuestion*, *tokenUsage* and *citations* are always `null`. - *originalQuestion*, *reformulatedQuestion* and *tokenUsage* are always `null`.
- The stream ends when an element with *streamState* equals `End` is received. This element contains token usage information for the question and the whole answer, and the list of citations. - The stream ends when an `End` event is received. This event contains token usage information for the final answer.
- Sources are included in the Markdown answer text.
## Limitations & FAQ ## Limitations & FAQ
SqlDatabaseVectorSearch/Components/Pages/Ask.razor
+12 -13
View File
@@ -170,7 +170,7 @@
messages.Add(assistantMessage); messages.Add(assistantMessage);
question = null; question = null;
await Task.Yield(); await InvokeAsync(StateHasChanged);
await EnsureMessageIsVisibleAsync(); await EnsureMessageIsVisibleAsync();
@@ -180,27 +180,26 @@
var vectorSearchService = scope.ServiceProvider.GetRequiredService<VectorSearchService>(); var vectorSearchService = scope.ServiceProvider.GetRequiredService<VectorSearchService>();
var response = vectorSearchService.AskStreamingAsync(userQuestion); var response = vectorSearchService.AskStreamingAsync(userQuestion);
await foreach (var delta in response) await foreach (var update in response)
{ {
if (delta.StreamState == StreamState.Start) if (update.StreamState == StreamState.Start)
{ {
userMessage.Text = delta.ReformulatedQuestion; userMessage.Text = update.ReformulatedQuestion;
assistantMessage.TokenUsage = FormatTokenUsage(delta.TokenUsage); assistantMessage.TokenUsage = FormatTokenUsage(update.TokenUsage);
assistantMessage.Status = MessageStatus.Streaming; assistantMessage.Status = MessageStatus.Streaming;
} }
else if (delta.StreamState == StreamState.Append) else if (update.StreamState == StreamState.Delta)
{ {
// Adds tokens to the assistant message as they are received. // Adds tokens to the assistant message as they are received.
assistantMessage.Text += delta.Answer; assistantMessage.Text += update.Answer;
} }
else if (delta.StreamState == StreamState.End) else if (update.StreamState == StreamState.End)
{ {
assistantMessage.Status = MessageStatus.Completed; assistantMessage.Status = MessageStatus.Completed;
assistantMessage.TokenUsage += FormatTokenUsage(delta.TokenUsage); assistantMessage.TokenUsage += FormatTokenUsage(update.TokenUsage);
} }
await Task.Yield(); await InvokeAsync(StateHasChanged);
StateHasChanged();
await EnsureMessageIsVisibleAsync(); await EnsureMessageIsVisibleAsync();
} }
@@ -234,13 +233,13 @@
showCopyConfirmation = true; showCopyConfirmation = true;
toolTipText = "Copied!"; toolTipText = "Copied!";
StateHasChanged(); await InvokeAsync(StateHasChanged);
await Task.Delay(3000); // Shows the checkmark for 3 seconds await Task.Delay(3000); // Shows the checkmark for 3 seconds
toolTipText = "Copy to Clipboard"; toolTipText = "Copy to Clipboard";
showCopyConfirmation = false; showCopyConfirmation = false;
StateHasChanged(); await InvokeAsync(StateHasChanged);
} }
private static string FormatTokenUsage(TokenUsageResponse? tokenUsageResponse) private static string FormatTokenUsage(TokenUsageResponse? tokenUsageResponse)
SqlDatabaseVectorSearch/Components/Pages/Documents.razor
+1 -1
View File
@@ -130,7 +130,7 @@ else
await using var scope = ServiceScopeFactory.CreateAsyncScope(); await using var scope = ServiceScopeFactory.CreateAsyncScope();
await LoadDocumentsAsync(scope.ServiceProvider); await LoadDocumentsAsync(scope.ServiceProvider);
StateHasChanged(); await InvokeAsync(StateHasChanged);
} }
private async Task LoadDocumentsAsync(IServiceProvider services) private async Task LoadDocumentsAsync(IServiceProvider services)
SqlDatabaseVectorSearch/Components/Pages/Home.razor
+9 -7
View File
@@ -6,7 +6,7 @@
<h1>SQL Database Vector Search</h1> <h1>SQL Database Vector Search</h1>
<p class="mt-3 p-3 rounded bg-light text-dark shadow-sm"> <p class="mt-3 p-3 rounded bg-light text-dark shadow-sm">
A Blazor Web App and Minimal API for Retrieval Augmented Generation (RAG) and vector search using the native VECTOR type in <img src="/images/sqldatabase.svg" style="height:1.5em;vertical-align:middle;" /> Azure SQL Database with <img src="/images/openai.svg" style="height:1.5em;vertical-align:middle;" /> Azure OpenAI. A Blazor Web App and Minimal API for Retrieval Augmented Generation (RAG) and vector search using the native VECTOR type in <img src="/images/sqldatabase.svg" style="height:1.5em;vertical-align:middle;" /> Azure SQL Database with <img src="/images/openai.svg" style="height:1.5em;vertical-align:middle;" /> Azure OpenAI and <a href="https://github.com/microsoft/agent-framework" target="_blank">Microsoft Agent Framework</a>.
</p> </p>
<p> <p>
@@ -14,18 +14,20 @@
<ul> <ul>
<li>Load documents (PDF, DOCX, TXT, MD)</li> <li>Load documents (PDF, DOCX, TXT, MD)</li>
<li>Generate embeddings and save them as vectors in Azure SQL Database</li> <li>Generate embeddings and save them as vectors in Azure SQL Database</li>
<li>Perform semantic search and RAG using Azure OpenAI</li> <li>Perform semantic search and RAG using Azure OpenAI agents</li>
<li>Interact via a Blazor Web App or programmatically via Minimal API</li> <li>Interact via a Blazor Web App or programmatically via Minimal API</li>
</ul> </ul>
Embeddings and chat completion are powered by <a href="https://github.com/microsoft/semantic-kernel" target="_blank">Semantic Kernel</a>. Vectors are managed with <a href="https://github.com/efcore/EfCore.SqlServer.VectorSearch" target="_blank">EFCore.SqlServer.VectorSearch</a>. Document import, question reformulation, and RAG are orchestrated with <a href="https://github.com/microsoft/agent-framework" target="_blank">Microsoft Agent Framework</a>. Vectors are managed with <a href="https://github.com/efcore/EfCore.SqlServer.VectorSearch" target="_blank">EFCore.SqlServer.VectorSearch</a>.
</p> </p>
<h3>Supported Features</h3> <h3>Supported Features</h3>
<ul> <ul>
<li><strong>Conversation History with Question Reformulation</strong>: View and reformulate your conversation history for better clarity and understanding.</li> <li><strong>Microsoft Agent Framework orchestration</strong>: Import documents through an embedding workflow and answer questions with dedicated reformulation and RAG agents.</li>
<li><strong>Information about Token Usage</strong>: Access detailed information about token usage for transparency and management.</li> <li><strong>Conversation history with question reformulation</strong>: Rewrite follow-up questions with the current conversation context before vector search.</li>
<li><strong>Response Streaming</strong>: Receive real-time streaming of responses for a seamless and efficient user experience.</li> <li><strong>SQL vector-search context</strong>: Retrieve relevant chunks from Azure SQL Database through native VECTOR search.</li>
<li><strong>Citations</strong>: Get citations for the sources used to justify each answer, allowing you to verify and understand the origin of the content.</li> <li><strong>Information about token usage</strong>: Access token usage for reformulation and final answer generation.</li>
<li><strong>Response streaming</strong>: Receive answer tokens in real time in the chat page and streaming API.</li>
<li><strong>Markdown source citations</strong>: Get citations directly in the answer text with source name, page number when available, and a short supporting excerpt.</li>
</ul> </ul>
<p class="mt-3 p-3 rounded bg-light text-dark shadow-sm"> <p class="mt-3 p-3 rounded bg-light text-dark shadow-sm">
SqlDatabaseVectorSearch/Endpoints/AskEndpoints.cs
+6 -4
View File
@@ -1,4 +1,6 @@
using System.ComponentModel; using System.ComponentModel;
using System.Net.ServerSentEvents;
using System.Runtime.CompilerServices;
using MinimalHelpers.FluentValidation; using MinimalHelpers.FluentValidation;
using SqlDatabaseVectorSearch.Models; using SqlDatabaseVectorSearch.Models;
using SqlDatabaseVectorSearch.Services; using SqlDatabaseVectorSearch.Services;
@@ -23,18 +25,18 @@ public class AskEndpoints : IEndpointRouteHandlerBuilder
endpoints.MapPost("/api/ask-streaming", (Question question, VectorSearchService vectorSearchService, CancellationToken cancellationToken, endpoints.MapPost("/api/ask-streaming", (Question question, VectorSearchService vectorSearchService, CancellationToken cancellationToken,
[Description("If true, the question will be reformulated taking into account the context of the chat identified by the given ConversationId.")] bool reformulate = true) => [Description("If true, the question will be reformulated taking into account the context of the chat identified by the given ConversationId.")] bool reformulate = true) =>
{ {
async IAsyncEnumerable<Response> Stream() async IAsyncEnumerable<SseItem<Response>> StreamAsync([EnumeratorCancellation] CancellationToken innerCancellationToken)
{ {
// Requests a streaming response. // Requests a streaming response.
var responseStream = vectorSearchService.AskStreamingAsync(question, reformulate, cancellationToken); var responseStream = vectorSearchService.AskStreamingAsync(question, reformulate, innerCancellationToken);
await foreach (var delta in responseStream) await foreach (var delta in responseStream)
{ {
yield return delta; yield return new(delta, delta.StreamState?.ToString());
} }
} }
return Stream(); return TypedResults.ServerSentEvents(StreamAsync(cancellationToken));
}) })
.WithValidation<Question>() .WithValidation<Question>()
.WithSummary("Asks a question and gets the response as streaming") .WithSummary("Asks a question and gets the response as streaming")
SqlDatabaseVectorSearch/Models/StreamState.cs
+1 -1
View File
@@ -3,6 +3,6 @@
public enum StreamState public enum StreamState
{ {
Start, Start,
Append, Delta,
End End
} }
SqlDatabaseVectorSearch/Services/VectorSearchService.cs
+1 -1
View File
@@ -77,7 +77,7 @@ public partial class VectorSearchService([FromKeyedServices("EmbeddingWorkflow")
updates.Add(update); updates.Add(update);
if (!string.IsNullOrEmpty(update.Text)) if (!string.IsNullOrEmpty(update.Text))
{ {
yield return new(question.ConversationId, update.Text, StreamState.Append); yield return new(question.ConversationId, update.Text, StreamState.Delta);
} }
} }
SqlDatabaseVectorSearch/Settings/AppSettings.cs
-4
View File
@@ -12,10 +12,6 @@ public class AppSettings
public int MaxRelevantChunks { get; init; } = 5; public int MaxRelevantChunks { get; init; } = 5;
public int MaxInputTokens { get; init; } = 16385;
public int MaxOutputTokens { get; init; } = 800;
public TimeSpan MessageExpiration { get; init; } public TimeSpan MessageExpiration { get; init; }
public int MessageLimit { get; set; } = 20; public int MessageLimit { get; set; } = 20;
SqlDatabaseVectorSearch/appsettings.json
-2
View File
@@ -25,8 +25,6 @@
"MaxTokensPerParagraph": 1000, "MaxTokensPerParagraph": 1000,
"OverlapTokens": 100, "OverlapTokens": 100,
"MaxRelevantChunks": 30, "MaxRelevantChunks": 30,
"MaxInputTokens": 32768,
"MaxOutputTokens": 800,
"MessageExpiration": "00:05:00", "MessageExpiration": "00:05:00",
"MessageLimit": 20 "MessageLimit": 20
}, },