Mirrors/SqlDatabaseVectorSearch
1
0
Fork 0
mirror of https://github.com/marcominerva/SqlDatabaseVectorSearch.git synced 2026-06-20 12:23:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

Enhance documentation and refactor request handling

Browse Source 
Updated README.md to clarify functionality and added a link to OpenAPI documentation in MainLayout.razor. Removed unnecessary methods from RequestExtensions.cs for simplification. Streamlined docs.md by removing detailed JSON response examples while retaining key information about response streaming.
This commit is contained in:
Marco Minerva
2025-02-25 15:17:06 +01:00
parent eb368470e8
commit 9071e130de
4 changed files with 8 additions and 148 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+3 -1
View File
@@ -1,11 +1,13 @@
# SQL Database Vector Search Sample # SQL Database Vector Search Sample
A repository that showcases the native VECTOR type in Azure SQL Database to perform embeddings and RAG with Azure OpenAI. A repository that showcases the native VECTOR type in Azure SQL Database to perform embeddings and RAG with Azure OpenAI.
The application is a Minimal API that exposes endpoints to load documents, generate embeddings and save them into the database as Vectors, and perform searches using Vector Search and RAG. Currently, PDF, DOCX, TXT and MD files are supported. Vectors are saved and retrieved with Entity Framework Core using the [EFCore.SqlServer.VectorSearch](https://github.com/efcore/EfCore.SqlServer.VectorSearch) library. Embedding and Chat Completion are integrated with [Semantic Kernel](https://github.com/microsoft/semantic-kernel). The application allows to load documents, generate embeddings and save them into the database as Vectors, and perform searches using Vector Search and RAG. Currently, PDF, DOCX, TXT and MD files are supported. Vectors are saved and retrieved with Entity Framework Core using the [EFCore.SqlServer.VectorSearch](https://github.com/efcore/EfCore.SqlServer.VectorSearch) library. Embedding and Chat Completion are integrated with [Semantic Kernel](https://github.com/microsoft/semantic-kernel).
> [!NOTE] > [!NOTE]
> If you prefer to use straight SQL, check out the [sql branch](https://github.com/marcominerva/SqlDatabaseVectorSearch/tree/sql). > If you prefer to use straight SQL, check out the [sql branch](https://github.com/marcominerva/SqlDatabaseVectorSearch/tree/sql).
This repository contains a Blazor Web App as well as a Minimal API that allows to programatically interact with embeddings and RAG.
### Web App ### Web App
![SQL Database Vector Search Web App](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/docs/SqlDatabaseVectorSearch_API_.png) ![SQL Database Vector Search Web App](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/docs/SqlDatabaseVectorSearch_API_.png)
SqlDatabaseVectorSearch/Components/Layout/MainLayout.razor
+4 -1
View File
@@ -6,8 +6,11 @@
<BlazorBootstrapLayout StickyHeader="true"> <BlazorBootstrapLayout StickyHeader="true">
<HeaderSection> <HeaderSection>
<a href="/swagger" target="_blank" class="text-decoration-none" title="OpenAPI documentation">
<Icon Name="IconName.FileTypeJson" Class="ps-3 ps-lg-2" Size="IconSize.x2" Color="IconColor.Muted"></Icon>
</a>
<a href="https://github.com/marcominerva/SqlDatabaseVectorSearch" target="_blank" class="text-decoration-none" title="View on GitHub"> <a href="https://github.com/marcominerva/SqlDatabaseVectorSearch" target="_blank" class="text-decoration-none" title="View on GitHub">
<Icon Name="IconName.Github" Class="ps-3 ps-lg-2" Size="IconSize.x2" Color="IconColor.Muted"></Icon> <Icon Name="IconName.Github" Class="ps-4 ps-lg-4" Size="IconSize.x2" Color="IconColor.Muted"></Icon>
</a> </a>
</HeaderSection> </HeaderSection>
SqlDatabaseVectorSearch/Extensions/RequestExtensions.cs
-24
View File
@@ -1,5 +1,4 @@
using System.Text.RegularExpressions; using System.Text.RegularExpressions;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Net.Http.Headers; using Microsoft.Net.Http.Headers;
namespace SqlDatabaseVectorSearch.Extensions; namespace SqlDatabaseVectorSearch.Extensions;
@@ -30,18 +29,6 @@ public static partial class RequestExtensions
return isMobileBrowser; return isMobileBrowser;
} }
public static string GetCultureFromRoute(this HttpContext httpContext)
=> RouteCultureRegex.Match(httpContext.Request.Path).Groups["culture"].Value.ToLowerInvariant();
public static string GetPathWithCulture(this HttpContext httpContext, string culture)
{
var request = httpContext.Request;
var path = RouteCultureRegex.Replace(request.Path.ToString(), "/");
var newPath = $"/{culture}{path}{request.QueryString}";
return newPath;
}
public static bool IsApiRequest(this HttpContext httpContext) public static bool IsApiRequest(this HttpContext httpContext)
=> httpContext.Request.Path.StartsWithSegments("/api"); => httpContext.Request.Path.StartsWithSegments("/api");
@@ -50,15 +37,4 @@ public static partial class RequestExtensions
public static bool IsWebRequest(this HttpContext httpContext) public static bool IsWebRequest(this HttpContext httpContext)
=> !httpContext.IsApiRequest() && !httpContext.IsSwaggerRequest(); => !httpContext.IsApiRequest() && !httpContext.IsSwaggerRequest();
public static string Content(this IUrlHelper urlHelper, HttpContext httpContext, string contentPath)
=> urlHelper.Content(httpContext.Request, contentPath);
public static string Content(this IUrlHelper urlHelper, HttpRequest request, string contentPath)
{
var path = urlHelper.Content(contentPath);
var url = $"{request.Scheme}://{request.Host}{request.PathBase}{path}";
return url;
}
} }
SqlDatabaseVectorSearch/wwwroot/docs.md
+1 -122
View File
@@ -12,125 +12,4 @@
- Conversation history with question reformulation - Conversation history with question reformulation
- Information about token usage - Information about token usage
- Response streaming - Response streaming
```json
{
"originalQuestion": "why is mars called the red planet?",
"reformulatedQuestion": "Why is Mars referred to as the Red Planet?",
"answer": "Mars is referred to as the Red Planet due to its characteristic reddish color, which is caused by the abundance of iron oxide (rust) on its surface. This distinctive coloration has also been a significant factor in the cultural and mythological associations of Mars across different civilizations.",
"streamState": null,
"tokenUsage": {
"reformulation": {
"inputTokenCount": 107,
"outputTokenCount": 10,
"totalTokenCount": 117
},
"embeddingTokenCount": 10,
"question": {
"inputTokenCount": 9142,
"outputTokenCount": 53,
"totalTokenCount": 9195
}
}
}
```
### How response streaming works
When using the `/api/ask-streaming` endpoint, answers will be streamed as happens with the typical response from OpenAI. The format of the response is the following:
```json
[
{
"originalQuestion": "why is mars called the red planet?",
"reformulatedQuestion": "Why is Mars referred to as the Red Planet?",
"answer": null,
"streamState": "Start",
"tokenUsage": {
"reformulation": {
"inputTokenCount": 107,
"outputTokenCount": 10,
"totalTokenCount": 117
},
"embeddingTokenCount": 10,
"question": null
}
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": "Mars",
"streamState": "Append",
"tokenUsage": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " is",
"streamState": "Append",
"tokenUsage": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " called",
"streamState": "Append",
"tokenUsage": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " the",
"streamState": "Append",
"tokenUsage": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " Red",
"streamState": "Append",
"tokenUsage": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": " Planet",
"streamState": "Append",
"tokenUsage": null
},
//...
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": ".",
"streamState": "Append",
"tokenUsage": null
},
{
"originalQuestion": null,
"reformulatedQuestion": null,
"answer": null,
"streamState": "End",
"tokenUsage": {
"reformulation": null,
"embeddingTokenCount": null,
"question": {
"inputTokenCount": 8986,
"outputTokenCount": 31,
"totalTokenCount": 9017
}
}
}
]
```
- The first piece of the response has the following characteristics:
- the *streamState* property is set to `Start`,
- it contains the question and its reformulation (if not requested, *reformulatedQuestion* will be equals to *originalQuestion*)
- the *tokenUsage* section holds information about token used for reformulation (if done) and for the embedding of the question
- Then, there are as many elements for the actual answer as necessary:
- each one contains a token
- The *streamState* property is set to `Append`
- *origianlQuestion*, *reformulatedQuestion* and *tokenUsage* are always `null`
- The stream ends when an element with *streamState* equals to `End` is received. This element contains token usage information for the question and the whole answer.