@@ -1,22 +1,29 @@
|
|||||||
# SQL Database Vector Search Sample
|
# SQL Database Vector Search Sample
|
||||||
A repository that showcases the native VECTOR type in Azure SQL Database to perform embeddings and RAG with Azure OpenAI.
|
A repository that showcases the native VECTOR type in Azure SQL Database to perform embeddings and RAG with Azure OpenAI.
|
||||||
|
|
||||||
The application is a Minimal API that exposes endpoints to load documents, generate embeddings and save them into the database as Vectors, and perform searches using Vector Search and RAG. Currently, PDF, DOCX, TXT and MD files are supported. Vectors are saved and retrieved with Entity Framework Core using the [EFCore.SqlServer.VectorSearch](https://github.com/efcore/EfCore.SqlServer.VectorSearch) library. Embedding and Chat Completion are integrated with [Semantic Kernel](https://github.com/microsoft/semantic-kernel).
|
The application allows to load documents, generate embeddings and save them into the database as Vectors, and perform searches using Vector Search and RAG. Currently, PDF, DOCX, TXT and MD files are supported. Vectors are saved and retrieved with Entity Framework Core using the [EFCore.SqlServer.VectorSearch](https://github.com/efcore/EfCore.SqlServer.VectorSearch) library. Embedding and Chat Completion are integrated with [Semantic Kernel](https://github.com/microsoft/semantic-kernel).
|
||||||
|
|
||||||
> [!NOTE]
|
> [!NOTE]
|
||||||
> If you prefer to use straight SQL, check out the [sql branch](https://github.com/marcominerva/SqlDatabaseVectorSearch/tree/sql).
|
> If you prefer to use straight SQL, check out the [sql branch](https://github.com/marcominerva/SqlDatabaseVectorSearch/tree/sql).
|
||||||
|
|
||||||

|
This repository contains a Blazor Web App as well as a Minimal API that allows to programatically interact with embeddings and RAG.
|
||||||
|
|
||||||
|
### Web App
|
||||||
|

|
||||||
|
|
||||||
|
### Web API
|
||||||
|

|
||||||
|
|
||||||
## Setup
|
## Setup
|
||||||
|
|
||||||
- [Create an Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/single-database-create-quickstart) on a server that has the Vector Support feature enabled
|
- [Create an Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database/single-database-create-quickstart)
|
||||||
- Execute the [Scripts.sql](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/Scripts.sql) file to create the tables needed by the application
|
|
||||||
- You may need to update the size of the [`VECTOR`](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/Scripts.sql#L17) column to match the size of the embedding model. Currently, the maximum allowed value is 1998.
|
|
||||||
- Open the [appsettings.json](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/SqlDatabaseVectorSearch/appsettings.json) file and set the connection string to the database and the other settings required by Azure OpenAI
|
- Open the [appsettings.json](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/SqlDatabaseVectorSearch/appsettings.json) file and set the connection string to the database and the other settings required by Azure OpenAI
|
||||||
- If your embedding model supports shortening, like **text-embedding-3-small** and **text-embedding-3-large**, and you want to use this feature, you need to set the [`Dimensions`](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/SqlDatabaseVectorSearch/appsettings.json#L17) property to match the value you have used in the SQL script. If your model doesn't provide this feature, or do you want to use the default size, just leave the [`Dimensions`](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/SqlDatabaseVectorSearch/appsettings.json#L17) property to NULL. Keep in mind that **text-embedding-3-small** has a dimension of 1536, while **text-embedding-3-large** uses vectors with 3072 elements, so with this latter model it is mandatory to specify a value (that, as said, must be less or equal to 1998).
|
- If your embedding model supports shortening, like **text-embedding-3-small** and **text-embedding-3-large**, and you want to use this feature, you need to set the [`Dimensions`](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/SqlDatabaseVectorSearch/appsettings.json#L17) property to the corresponding value. If your model doesn't provide this feature, or do you want to use the default size, just leave the [`Dimensions`](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/SqlDatabaseVectorSearch/appsettings.json#L17) property to NULL. Keep in mind that **text-embedding-3-small** has a dimension of 1536, while **text-embedding-3-large** uses vectors with 3072 elements, so with this latter model it is mandatory to specify a value (that, as said, must be less or equal to 1998).
|
||||||
- Run the application and start importing your documents with `/api/documents` endpoint.
|
- You may need to update the size of the [`VECTOR`](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/SqlDatabaseVectorSearch/DataAccessLayer/ApplicationDbContext.cs?plain=1#L42C1-L42C47) column to match the size of the embedding model. The default value is 1536. Currently, the maximum allowed value is 1998. If you change it, remember to update also the [Database Migration](https://github.com/marcominerva/SqlDatabaseVectorSearch/blob/master/SqlDatabaseVectorSearch/DataAccessLayer/Migrations/00000000000000_Initial.cs?plain=1#L35C1-L35C92).
|
||||||
- Ask questions using `/api/ask` or `/api/ask-streaming` endpoints.
|
- Run the application and start importing your documents
|
||||||
|
- If you want to directly use the APIs:
|
||||||
|
- import your documents with the `/api/documents` endpoint.
|
||||||
|
- Ask questions using `/api/ask` or `/api/ask-streaming` endpoints.
|
||||||
|
|
||||||
## Supported features
|
## Supported features
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,33 @@
|
|||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
|
<base href="/" />
|
||||||
|
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-QWTKZyjpPEjISv5WaRU9OFeRpok6YctnYmDr5pNlyT2bRjXh0JMhjY6hW+ALEwIH" crossorigin="anonymous">
|
||||||
|
<link href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.11.3/font/bootstrap-icons.css" rel="stylesheet" />
|
||||||
|
<link href="_content/Blazor.Bootstrap/blazor.bootstrap.css" rel="stylesheet" />
|
||||||
|
<script src="https://kit.fontawesome.com/f7a7b34f96.js" crossorigin="anonymous"></script>
|
||||||
|
<link rel="stylesheet" href="@Assets["css/app.css"]" />
|
||||||
|
<link rel="stylesheet" href="@Assets["SqlDatabaseVectorSearch.styles.css"]" />
|
||||||
|
<ImportMap />
|
||||||
|
<link rel="icon" type="image/png" href="favicon.png" />
|
||||||
|
<HeadOutlet @rendermode="InteractiveServer" />
|
||||||
|
</head>
|
||||||
|
|
||||||
|
<body>
|
||||||
|
<Routes @rendermode="InteractiveServer" />
|
||||||
|
<script src="_framework/blazor.web.js"></script>
|
||||||
|
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js" integrity="sha384-YvpcrYf0tY3lHB60NNkmXc5s9fDVZLESaAA55NDzOxhy9GkcIdslK1eN7N6jIeHz" crossorigin="anonymous"></script>
|
||||||
|
<!-- Add chart.js reference if chart components are used in your application. -->
|
||||||
|
<script src="https://cdnjs.cloudflare.com/ajax/libs/Chart.js/4.0.1/chart.umd.js" integrity="sha512-gQhCDsnnnUfaRzD8k1L5llCCV6O9HN09zClIzzeJ8OJ9MpGmIlCxm+pdCkqTwqJ4JcjbojFr79rl2F1mzcoLMQ==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
|
||||||
|
<!-- Add chartjs-plugin-datalabels.min.js reference if chart components with data label feature is used in your application. -->
|
||||||
|
<script src="https://cdnjs.cloudflare.com/ajax/libs/chartjs-plugin-datalabels/2.2.0/chartjs-plugin-datalabels.min.js" integrity="sha512-JPcRR8yFa8mmCsfrw4TNte1ZvF1e3+1SdGMslZvmrzDYxS69J7J49vkFL8u6u8PlPJK+H3voElBtUCzaXj+6ig==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
|
||||||
|
<!-- Add sortable.js reference if SortableList component is used in your application. -->
|
||||||
|
<script src="https://cdn.jsdelivr.net/npm/sortablejs@latest/Sortable.min.js"></script>
|
||||||
|
<script src="_content/Blazor.Bootstrap/blazor.bootstrap.js" asp-append-version="true"></script>
|
||||||
|
<script src="js/functions.js" asp-append-version="true"></script>
|
||||||
|
</body>
|
||||||
|
|
||||||
|
</html>
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
@using System.Runtime.InteropServices
|
||||||
|
|
||||||
|
@inherits LayoutComponentBase
|
||||||
|
|
||||||
|
<Toasts class="p-3" AutoHide="true" Placement="ToastsPlacement.TopRight" />
|
||||||
|
|
||||||
|
<BlazorBootstrapLayout StickyHeader="true">
|
||||||
|
<HeaderSection>
|
||||||
|
<a href="/swagger" target="_blank" class="text-decoration-none" title="OpenAPI documentation">
|
||||||
|
<Icon Name="IconName.FileTypeJson" Class="ps-3 ps-lg-2" Size="IconSize.x2" Color="IconColor.Muted"></Icon>
|
||||||
|
</a>
|
||||||
|
<a href="https://github.com/marcominerva/SqlDatabaseVectorSearch" target="_blank" class="text-decoration-none" title="View on GitHub">
|
||||||
|
<Icon Name="IconName.Github" Class="ps-4 ps-lg-4" Size="IconSize.x2" Color="IconColor.Muted"></Icon>
|
||||||
|
</a>
|
||||||
|
</HeaderSection>
|
||||||
|
|
||||||
|
<SidebarSection>
|
||||||
|
<Sidebar2 Href="/"
|
||||||
|
IconName="IconName.Search"
|
||||||
|
Title="SQL Vector Search"
|
||||||
|
DataProvider="Sidebar2DataProvider"
|
||||||
|
WidthUnit="Unit.Px" />
|
||||||
|
</SidebarSection>
|
||||||
|
|
||||||
|
<ContentSection>
|
||||||
|
@Body
|
||||||
|
</ContentSection>
|
||||||
|
|
||||||
|
</BlazorBootstrapLayout>
|
||||||
|
|
||||||
|
@code {
|
||||||
|
private IEnumerable<NavItem> navItems = default!;
|
||||||
|
|
||||||
|
private Task<Sidebar2DataProviderResult> Sidebar2DataProvider(Sidebar2DataProviderRequest request)
|
||||||
|
{
|
||||||
|
if (navItems is null)
|
||||||
|
{
|
||||||
|
navItems = GetNavItems();
|
||||||
|
}
|
||||||
|
|
||||||
|
var result = request.ApplyTo(navItems);
|
||||||
|
return Task.FromResult(result);
|
||||||
|
}
|
||||||
|
|
||||||
|
private IEnumerable<NavItem> GetNavItems()
|
||||||
|
{
|
||||||
|
navItems = [
|
||||||
|
new() { Id = "1", Href = "/", IconName = IconName.HouseDoorFill, Text = "Home", Match = NavLinkMatch.All},
|
||||||
|
new() { Id = "2", Href= "/documents", IconName = IconName.FileText, Text = "Documents" },
|
||||||
|
new() { Id = "3", Href = "/ask", IconName = IconName.ChatDots, Text = "Ask"}
|
||||||
|
];
|
||||||
|
|
||||||
|
return navItems;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
<div id="blazor-error-ui" data-nosnippet>
|
||||||
|
An unhandled error has occurred.
|
||||||
|
<a href="." class="reload">Reload</a>
|
||||||
|
<span class="dismiss">🗙</span>
|
||||||
|
</div>
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
#blazor-error-ui {
|
||||||
|
color-scheme: light only;
|
||||||
|
background: lightyellow;
|
||||||
|
bottom: 0;
|
||||||
|
box-shadow: 0 -1px 2px rgba(0, 0, 0, 0.2);
|
||||||
|
box-sizing: border-box;
|
||||||
|
display: none;
|
||||||
|
left: 0;
|
||||||
|
padding: 0.6rem 1.25rem 0.7rem 1.25rem;
|
||||||
|
position: fixed;
|
||||||
|
width: 100%;
|
||||||
|
z-index: 1000;
|
||||||
|
}
|
||||||
|
|
||||||
|
#blazor-error-ui .dismiss {
|
||||||
|
cursor: pointer;
|
||||||
|
position: absolute;
|
||||||
|
right: 0.75rem;
|
||||||
|
top: 0.5rem;
|
||||||
|
}
|
||||||
@@ -0,0 +1,258 @@
|
|||||||
|
@page "/ask"
|
||||||
|
|
||||||
|
@inject IServiceProvider ServiceProvider
|
||||||
|
@inject IJSRuntime JSRuntime
|
||||||
|
|
||||||
|
<PageTitle>Chat with your data</PageTitle>
|
||||||
|
|
||||||
|
<div class="card mx-auto mt-2">
|
||||||
|
<div class="card-body">
|
||||||
|
|
||||||
|
@foreach (var message in messages)
|
||||||
|
{
|
||||||
|
if (message.Role == "user")
|
||||||
|
{
|
||||||
|
<div class="d-flex align-items-baseline text-end justify-content-end">
|
||||||
|
<div class="pe-2">
|
||||||
|
<div>
|
||||||
|
<div class="card card-text d-inline-block p-2 px-3 m-1">
|
||||||
|
<Markdown style="overflow-y:auto;">@message.Text</Markdown>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="position-relative avatar">
|
||||||
|
<Image src="/images/user.png" class="img-fluid rounded-circle" alt="" />
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
else if (message.Role == "assistant")
|
||||||
|
{
|
||||||
|
<div class="d-flex align-items-baseline">
|
||||||
|
<div class="position-relative avatar">
|
||||||
|
<Image src="/images/assistant.png" class="img-fluid rounded-circle" alt="" />
|
||||||
|
</div>
|
||||||
|
<div class="pe-2">
|
||||||
|
<div>
|
||||||
|
@if (message.Text is null)
|
||||||
|
{
|
||||||
|
<div class="card card-text d-inline-block p-3 px-3 m-1">
|
||||||
|
<div class="progress-chat" role="progressbar" aria-label="I'm thinking" aria-valuenow="0" aria-valuemin="0" aria-valuemax="100">
|
||||||
|
<div class="progress-bar-chat">
|
||||||
|
<div class="progress-bar-indeterminate"></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
<div class="card card-text d-inline-block p-2 px-3 m-1">
|
||||||
|
<div>
|
||||||
|
<Markdown style="overflow-y:auto;">@message.Text</Markdown>
|
||||||
|
</div>
|
||||||
|
@if (message.IsCompleted)
|
||||||
|
{
|
||||||
|
<div class="d-flex justify-content-between">
|
||||||
|
<div class="text-start bg-transparent mt-3">
|
||||||
|
<Tooltip Title="@message.TokenUsage" IsHtml="true" Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
|
||||||
|
<Icon Class="d-flex text-body-secondary" Name="IconName.InfoCircle"></Icon>
|
||||||
|
</Tooltip>
|
||||||
|
</div>
|
||||||
|
<div class="text-end bg-transparent">
|
||||||
|
<Tooltip Title="Copy to Clipboard" Color="TooltipColor.Dark" Placement="TooltipPlacement.Bottom">
|
||||||
|
<Button Type="ButtonType.Button" Outline="false" @onclick="@(async () => await CopyToClipboardAsync(message.Text))">
|
||||||
|
<Icon Name="IconName.Clipboard" />
|
||||||
|
</Button>
|
||||||
|
</Tooltip>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
<div @ref="chat"></div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="card-footer bg-white w-100 bottom-0 m-0 p-1">
|
||||||
|
<div class="input-group">
|
||||||
|
<span class="input-group-text bg-transparent border-0">
|
||||||
|
<Tooltip Title="Messages aren't stored in any way on either the client or the server." Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
|
||||||
|
<Icon Class="d-flex text-body-secondary" Name="IconName.InfoCircle"></Icon>
|
||||||
|
</Tooltip>
|
||||||
|
</span>
|
||||||
|
<input @ref="askInput" type="text" @bind="@question" @bind:event="oninput" placeholder="Ask me anything..." class="form-control border-0" maxlength="2000" @onkeydown="HandleKeyDown" />
|
||||||
|
<div class="input-group-text bg-transparent border-0">
|
||||||
|
<Button Type="ButtonType.Submit" @ref="askButton" Color="ButtonColor.Primary" Disabled="@(isAsking || string.IsNullOrWhiteSpace(question))" @onclick="AskQuestion">
|
||||||
|
<Icon Name="IconName.Send" />
|
||||||
|
</Button>
|
||||||
|
<Button Type="ButtonType.Reset" @ref="resetButton" Class="ms-2" Color="ButtonColor.Secondary" Disabled="@isAsking" @onclick="Reset">
|
||||||
|
<Icon CustomIconName="bi bi-x-lg" />
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
@code
|
||||||
|
{
|
||||||
|
private Button askButton = default!;
|
||||||
|
private Button resetButton = default!;
|
||||||
|
private ElementReference askInput = default!;
|
||||||
|
private ElementReference chat = default!;
|
||||||
|
|
||||||
|
private IList<Message> messages = [];
|
||||||
|
private string? question;
|
||||||
|
|
||||||
|
private Guid conversationId = Guid.NewGuid();
|
||||||
|
private bool isAsking = false;
|
||||||
|
|
||||||
|
protected override async Task OnAfterRenderAsync(bool firstRender)
|
||||||
|
{
|
||||||
|
if (!firstRender)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
await JSRuntime.InvokeVoidAsync("setFocus", askInput);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task HandleKeyDown(KeyboardEventArgs e)
|
||||||
|
{
|
||||||
|
if (isAsking)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (e.Key == "Enter" && !string.IsNullOrWhiteSpace(question))
|
||||||
|
{
|
||||||
|
await AskQuestion();
|
||||||
|
}
|
||||||
|
else if (e.Key == "ArrowUp" && messages.Count >= 2)
|
||||||
|
{
|
||||||
|
question = messages[^2].Text;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task AskQuestion()
|
||||||
|
{
|
||||||
|
isAsking = true;
|
||||||
|
|
||||||
|
var userQuestion = new Question(conversationId, question!);
|
||||||
|
var userMessage = new Message { Text = userQuestion.Text, Role = "user" };
|
||||||
|
messages.Add(userMessage);
|
||||||
|
|
||||||
|
var assistantMessage = new Message { Role = "assistant" };
|
||||||
|
messages.Add(assistantMessage);
|
||||||
|
|
||||||
|
question = null;
|
||||||
|
await Task.Yield();
|
||||||
|
|
||||||
|
await EnsureMessageIsVisibleAsync();
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await using var scope = ServiceProvider.CreateAsyncScope();
|
||||||
|
var vectorSearchService = scope.ServiceProvider.GetRequiredService<VectorSearchService>();
|
||||||
|
|
||||||
|
var response = vectorSearchService.AskStreamingAsync(userQuestion);
|
||||||
|
await foreach (var delta in response)
|
||||||
|
{
|
||||||
|
if (delta.StreamState == StreamState.Start)
|
||||||
|
{
|
||||||
|
assistantMessage.TokenUsage = FormatTokenUsage(delta.TokenUsage);
|
||||||
|
}
|
||||||
|
else if (delta.StreamState == StreamState.Append)
|
||||||
|
{
|
||||||
|
assistantMessage.Text += delta.Answer;
|
||||||
|
}
|
||||||
|
else if (delta.StreamState == StreamState.End)
|
||||||
|
{
|
||||||
|
assistantMessage.IsCompleted = true;
|
||||||
|
assistantMessage.TokenUsage += FormatTokenUsage(delta.TokenUsage);
|
||||||
|
}
|
||||||
|
|
||||||
|
await Task.Yield();
|
||||||
|
StateHasChanged();
|
||||||
|
|
||||||
|
await EnsureMessageIsVisibleAsync();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
assistantMessage.Text = $"There was an error while processing the question: {ex.Message}";
|
||||||
|
assistantMessage.IsCompleted = true;
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
await EnsureMessageIsVisibleAsync();
|
||||||
|
|
||||||
|
isAsking = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void Reset()
|
||||||
|
{
|
||||||
|
question = null;
|
||||||
|
conversationId = Guid.NewGuid();
|
||||||
|
messages.Clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task CopyToClipboardAsync(string text)
|
||||||
|
{
|
||||||
|
await JSRuntime.InvokeVoidAsync("navigator.clipboard.writeText", text);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static string FormatTokenUsage(TokenUsageResponse? tokenUsageResponse)
|
||||||
|
{
|
||||||
|
if (tokenUsageResponse is null)
|
||||||
|
{
|
||||||
|
return string.Empty;
|
||||||
|
}
|
||||||
|
|
||||||
|
var reformulation = tokenUsageResponse.Reformulation is not null
|
||||||
|
? $"<p><strong>Reformulation:</strong><br />{FormatTokenUsageDetails(tokenUsageResponse.Reformulation)}</p>"
|
||||||
|
: string.Empty;
|
||||||
|
|
||||||
|
var embeddingTokenCount = tokenUsageResponse.EmbeddingTokenCount.HasValue
|
||||||
|
? $"<p><strong>Embedding Token Count:</strong> {tokenUsageResponse.EmbeddingTokenCount}</p>"
|
||||||
|
: string.Empty;
|
||||||
|
|
||||||
|
var question = tokenUsageResponse.Question is not null
|
||||||
|
? $"<p><strong>Question:</strong><br />{FormatTokenUsageDetails(tokenUsageResponse.Question)}</p>"
|
||||||
|
: string.Empty;
|
||||||
|
|
||||||
|
return $"{reformulation}{embeddingTokenCount}{question}";
|
||||||
|
|
||||||
|
static string FormatTokenUsageDetails(TokenUsage? tokenUsage)
|
||||||
|
{
|
||||||
|
if (tokenUsage is null)
|
||||||
|
{
|
||||||
|
return string.Empty;
|
||||||
|
}
|
||||||
|
|
||||||
|
return $"Input Token Count: {tokenUsage.InputTokenCount}<br />" +
|
||||||
|
$"Output Token Count: {tokenUsage.OutputTokenCount}<br />" +
|
||||||
|
$"Total Token Count: {tokenUsage.TotalTokenCount}";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task EnsureMessageIsVisibleAsync()
|
||||||
|
{
|
||||||
|
await JSRuntime.InvokeVoidAsync("scrollTo", chat);
|
||||||
|
}
|
||||||
|
|
||||||
|
public class Message
|
||||||
|
{
|
||||||
|
public string? Text { get; set; }
|
||||||
|
|
||||||
|
public required string Role { get; set; }
|
||||||
|
|
||||||
|
public bool IsCompleted { get; set; }
|
||||||
|
|
||||||
|
public string? TokenUsage { get; set; }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
.tooltip-inner {
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
.avatar {
|
||||||
|
width: 50px;
|
||||||
|
height: 50px;
|
||||||
|
border-radius: 50%;
|
||||||
|
border: 2px solid #ddd;
|
||||||
|
padding: 2px;
|
||||||
|
flex: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
input:focus {
|
||||||
|
outline: 0px !important;
|
||||||
|
box-shadow: none !important;
|
||||||
|
}
|
||||||
|
|
||||||
|
input[type="checkbox"],
|
||||||
|
input[type="checkbox"] + label {
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
|
||||||
|
.card-body {
|
||||||
|
overflow: auto;
|
||||||
|
height: 560px;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (min-width: 768px) {
|
||||||
|
.card-body {
|
||||||
|
height: 665px;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (min-width: 2560px) {
|
||||||
|
.card-body {
|
||||||
|
height: 1020px;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
.card-text {
|
||||||
|
border: 2px solid #ddd;
|
||||||
|
border-radius: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.progress-chat {
|
||||||
|
width: 200px;
|
||||||
|
height: 4px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.progress-bar-chat {
|
||||||
|
height: 4px;
|
||||||
|
background-color: rgba(5, 114, 206, 0.2);
|
||||||
|
width: 100%;
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
|
||||||
|
.progress-bar-indeterminate {
|
||||||
|
width: 100%;
|
||||||
|
height: 100%;
|
||||||
|
background-color: rgb(5, 114, 206);
|
||||||
|
animation: indeterminate-animation 1s infinite linear;
|
||||||
|
transform-origin: 0% 50%;
|
||||||
|
}
|
||||||
|
|
||||||
|
@keyframes indeterminate-animation {
|
||||||
|
0% {
|
||||||
|
transform: translateX(0) scaleX(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
40% {
|
||||||
|
transform: translateX(0) scaleX(0.4);
|
||||||
|
}
|
||||||
|
|
||||||
|
100% {
|
||||||
|
transform: translateX(100%) scaleX(0.5);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-clipboard {
|
||||||
|
line-height: 1;
|
||||||
|
color: var(--bs-body-color);
|
||||||
|
background-color: var(--bd-pre-bg);
|
||||||
|
border: 0;
|
||||||
|
border-radius: .25rem;
|
||||||
|
margin-right: -.4em
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-clipboard:hover {
|
||||||
|
color: var(--bs-link-hover-color)
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-clipboard:focus {
|
||||||
|
z-index: 3
|
||||||
|
}
|
||||||
|
|
||||||
|
.btn-clipboard {
|
||||||
|
position: relative;
|
||||||
|
z-index: 2;
|
||||||
|
}
|
||||||
@@ -0,0 +1,280 @@
|
|||||||
|
@page "/documents"
|
||||||
|
@using MimeMapping
|
||||||
|
|
||||||
|
@inject IServiceProvider ServiceProvider
|
||||||
|
@inject IJSRuntime JSRuntime
|
||||||
|
|
||||||
|
<ConfirmDialog @ref="dialog" />
|
||||||
|
|
||||||
|
<PageTitle>Documents</PageTitle>
|
||||||
|
|
||||||
|
<h4 class="mb-4">
|
||||||
|
<Icon Name="IconName.Upload" class="me-2" />
|
||||||
|
Upload new document
|
||||||
|
</h4>
|
||||||
|
|
||||||
|
<EditForm EditContext="@editContext" OnValidSubmit="HandleValidSubmit">
|
||||||
|
<DataAnnotationsValidator />
|
||||||
|
|
||||||
|
<div class="row">
|
||||||
|
<div class="col-md-5 col-sm-4 col-5">
|
||||||
|
<div class="input-group">
|
||||||
|
<span class="input-group-text">
|
||||||
|
<Tooltip Title="PDF, DOCX, TXT and MD files are supported" Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
|
||||||
|
<Icon Class="d-flex text-body-secondary" Name="IconName.InfoCircle"></Icon>
|
||||||
|
</Tooltip>
|
||||||
|
</span>
|
||||||
|
<InputFile class="form-control" OnChange="HandleFileSelected" accept=".pdf,.docx,.txt,.md" />
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="col-md-5 col-sm-5 col-5">
|
||||||
|
<div class="input-group">
|
||||||
|
<span class="input-group-text">
|
||||||
|
<Tooltip Title="The unique identifier (GUID) of the document. If not provided, a new one will be generated. If you specify an existing Document ID, the corresponding document will be overwritten." Color="TooltipColor.Primary" Placement="TooltipPlacement.Bottom">
|
||||||
|
<Icon Class="d-flex text-body-secondary me-2" Name="IconName.InfoCircle"></Icon>
|
||||||
|
</Tooltip>
|
||||||
|
Document ID
|
||||||
|
</span>
|
||||||
|
<TextInput Placeholder="Enter a valid GUID or leave empty for auto-generation" @bind-Value="@uploadDocumentRequest.DocumentId" />
|
||||||
|
</div>
|
||||||
|
<ValidationMessage For="@(() => uploadDocumentRequest.DocumentId)" />
|
||||||
|
</div>
|
||||||
|
<div class="col-md-2 col-sm-3 col-2">
|
||||||
|
<div class="d-grid gap-2">
|
||||||
|
<Button @ref="uploadButton" Type="ButtonType.Submit" Color="ButtonColor.Primary" To="#" Disabled="@(uploadDocumentRequest.File is null)"><Icon Name="IconName.Upload" /><span class="d-none d-lg-inline ps-3">Upload</span></Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</EditForm>
|
||||||
|
|
||||||
|
@if (isLoading && documents.Count == 0)
|
||||||
|
{
|
||||||
|
<div class="text-center">
|
||||||
|
<Spinner Type="SpinnerType.Dots" Class="me-3 mt-4" Color="SpinnerColor.Primary" />
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
<h4 class="mt-4 mb-4">
|
||||||
|
<Icon Name="IconName.Files" class="me-2" />
|
||||||
|
Available documents
|
||||||
|
</h4>
|
||||||
|
|
||||||
|
<table class="table table-bordered table-striped table-hover">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th></th>
|
||||||
|
<th>ID</th>
|
||||||
|
<th>Name</th>
|
||||||
|
<th>Content type</th>
|
||||||
|
<th>Number of chunks</th>
|
||||||
|
<th>Creation date</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
@foreach (var document in documents)
|
||||||
|
{
|
||||||
|
<tr>
|
||||||
|
<td>
|
||||||
|
<CheckboxInput @bind-Value="document.IsSelected" />
|
||||||
|
</td>
|
||||||
|
<td>@document.Id</td>
|
||||||
|
<td>@document.Name</td>
|
||||||
|
<td>@document.ContentType</td>
|
||||||
|
<td>@document.ChunkCount</td>
|
||||||
|
<td>@document.LocalCreationDateString</td>
|
||||||
|
</tr>
|
||||||
|
}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
|
||||||
|
<div class="row">
|
||||||
|
<div class="col-md-2 col-sm-3 col-2">
|
||||||
|
<div class="d-grid gap-2">
|
||||||
|
<Button @ref="deleteButton" Color="ButtonColor.Danger" Disabled="@(!documents.Any(d => d.IsSelected))" @onclick="DeleteSelectedDocuments">
|
||||||
|
<Icon Name="IconName.Trash" /><span class="d-none d-lg-inline ps-3">Delete</span>
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
}
|
||||||
|
|
||||||
|
@code {
|
||||||
|
private ConfirmDialog dialog = default!;
|
||||||
|
private Button uploadButton = default!;
|
||||||
|
private Button deleteButton = default!;
|
||||||
|
|
||||||
|
private bool isLoading = true;
|
||||||
|
private IList<SelectableDocument> documents = [];
|
||||||
|
|
||||||
|
private UploadDocumentRequest uploadDocumentRequest = new();
|
||||||
|
private EditContext? editContext;
|
||||||
|
|
||||||
|
[Inject]
|
||||||
|
protected ToastService ToastService { get; set; } = default!;
|
||||||
|
|
||||||
|
protected override void OnInitialized()
|
||||||
|
{
|
||||||
|
editContext = new EditContext(uploadDocumentRequest);
|
||||||
|
base.OnInitialized();
|
||||||
|
}
|
||||||
|
|
||||||
|
protected override async Task OnAfterRenderAsync(bool firstRender)
|
||||||
|
{
|
||||||
|
if (!firstRender)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
await using var scope = ServiceProvider.CreateAsyncScope();
|
||||||
|
await LoadDocumentsAsync(scope.ServiceProvider);
|
||||||
|
|
||||||
|
StateHasChanged();
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task LoadDocumentsAsync(IServiceProvider services)
|
||||||
|
{
|
||||||
|
isLoading = true;
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
var documentService = services.GetRequiredService<DocumentService>();
|
||||||
|
var dbDocuments = await documentService.GetAsync();
|
||||||
|
|
||||||
|
documents.Clear();
|
||||||
|
foreach (var dbDocument in dbDocuments)
|
||||||
|
{
|
||||||
|
documents.Add(new SelectableDocument(dbDocument.Id, dbDocument.Name, dbDocument.CreationDate, dbDocument.ChunkCount)
|
||||||
|
{
|
||||||
|
LocalCreationDateString = await GetLocalDateTimeStringAsync(dbDocument.CreationDate)
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
isLoading = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void HandleFileSelected(InputFileChangeEventArgs e)
|
||||||
|
{
|
||||||
|
uploadDocumentRequest.File = e.File;
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task HandleValidSubmit()
|
||||||
|
{
|
||||||
|
if (uploadDocumentRequest.File is null)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
uploadButton.ShowLoading();
|
||||||
|
|
||||||
|
var fileName = uploadDocumentRequest.File.Name;
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await using var inputStream = uploadDocumentRequest.File.OpenReadStream(20 * 1024 * 1024); // 20 MB
|
||||||
|
await using var stream = await inputStream.GetMemoryStreamAsync();
|
||||||
|
|
||||||
|
await using var scope = ServiceProvider.CreateAsyncScope();
|
||||||
|
var vectorSearchService = scope.ServiceProvider.GetRequiredService<VectorSearchService>();
|
||||||
|
|
||||||
|
var documentId = string.IsNullOrWhiteSpace(uploadDocumentRequest.DocumentId) ? null : (Guid?)Guid.Parse(uploadDocumentRequest.DocumentId);
|
||||||
|
await vectorSearchService.ImportAsync(stream, fileName, MimeUtility.GetMimeMapping(fileName), documentId);
|
||||||
|
|
||||||
|
ToastService.Notify(await CreateToastMessageAsync(ToastType.Success, "Upload document", $"The document {fileName} has been successfully uploaded and indexed."));
|
||||||
|
|
||||||
|
await LoadDocumentsAsync(scope.ServiceProvider);
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
ToastService.Notify(await CreateToastMessageAsync(ToastType.Danger, "Upload error", $"There was an error while uploading the document {fileName}: {ex.Message}"));
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
uploadButton.HideLoading();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task DeleteSelectedDocuments()
|
||||||
|
{
|
||||||
|
var selectedDocumentIds = documents?.Where(d => d.IsSelected).Select(d => d.Id) ?? [];
|
||||||
|
|
||||||
|
var options = new ConfirmDialogOptions
|
||||||
|
{
|
||||||
|
YesButtonText = "Yes",
|
||||||
|
YesButtonColor = ButtonColor.Danger,
|
||||||
|
NoButtonText = "No",
|
||||||
|
NoButtonColor = ButtonColor.Secondary
|
||||||
|
};
|
||||||
|
|
||||||
|
var confirmation = await dialog.ShowAsync(
|
||||||
|
title: "Delete the selected documents?",
|
||||||
|
message1: "This will delete the documents and all the corresponding embeddings. The operation cannot be undone.",
|
||||||
|
message2: "Do you want to proceed?",
|
||||||
|
confirmDialogOptions: options);
|
||||||
|
|
||||||
|
if (!confirmation)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
deleteButton.ShowLoading();
|
||||||
|
|
||||||
|
await using var scope = ServiceProvider.CreateAsyncScope();
|
||||||
|
var documentService = scope.ServiceProvider.GetRequiredService<DocumentService>();
|
||||||
|
|
||||||
|
await documentService.DeleteAsync(selectedDocumentIds);
|
||||||
|
|
||||||
|
await LoadDocumentsAsync(scope.ServiceProvider);
|
||||||
|
ToastService.Notify(await CreateToastMessageAsync(ToastType.Info, "Delete documents", "The selected documents have been successfully deleted."));
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
ToastService.Notify(await CreateToastMessageAsync(ToastType.Danger, "Delete error", $"There was an error while deleting the documents: {ex.Message}"));
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
deleteButton.HideLoading();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task<ToastMessage> CreateToastMessageAsync(ToastType toastType, string title, string message)
|
||||||
|
{
|
||||||
|
var toastMessage = new ToastMessage
|
||||||
|
{
|
||||||
|
Type = toastType,
|
||||||
|
Title = title,
|
||||||
|
HelpText = await GetLocalDateTimeStringAsync(DateTimeOffset.UtcNow),
|
||||||
|
Message = message
|
||||||
|
};
|
||||||
|
|
||||||
|
return toastMessage;
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task<string> GetLocalDateTimeStringAsync(DateTimeOffset dateTime)
|
||||||
|
{
|
||||||
|
return await JSRuntime.InvokeAsync<string>("getLocalTime", dateTime);
|
||||||
|
}
|
||||||
|
|
||||||
|
private record class SelectableDocument(Guid Id, string Name, DateTimeOffset CreationDate, int ChunkCount) : Document(Id, Name, CreationDate, ChunkCount)
|
||||||
|
{
|
||||||
|
public bool IsSelected { get; set; }
|
||||||
|
|
||||||
|
public string ContentType => MimeUtility.GetMimeMapping(Name);
|
||||||
|
|
||||||
|
public string LocalCreationDateString { get; set; } = string.Empty;
|
||||||
|
}
|
||||||
|
|
||||||
|
public class UploadDocumentRequest
|
||||||
|
{
|
||||||
|
[SupplyParameterFromForm]
|
||||||
|
public IBrowserFile? File { get; set; }
|
||||||
|
|
||||||
|
[RegularExpression(@"^(\{|\()?[0-9a-fA-F]{8}(-?)[0-9a-fA-F]{4}(-?)[0-9a-fA-F]{4}(-?)[0-9a-fA-F]{4}(-?)[0-9a-fA-F]{12}(\}|\))?$", ErrorMessage = "Invalid GUID format.")]
|
||||||
|
public string? DocumentId { get; set; }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
@page "/Error"
|
||||||
|
@using System.Diagnostics
|
||||||
|
@rendermode @(new InteractiveServerRenderMode(prerender: false))
|
||||||
|
|
||||||
|
<div class="d-flex align-items-center justify-content-center">
|
||||||
|
<div class="text-center">
|
||||||
|
@if (Code == 404)
|
||||||
|
{
|
||||||
|
<PageTitle>Page Not Found</PageTitle>
|
||||||
|
|
||||||
|
<h1 class="display-1 fw-bold">404</h1>
|
||||||
|
<p class="fs-3"><span class="text-danger">Ops!</span> Page Not Found.</p>
|
||||||
|
<p class="lead">
|
||||||
|
The page you're looking for does not exists.
|
||||||
|
</p>
|
||||||
|
}
|
||||||
|
else if (Code > 0)
|
||||||
|
{
|
||||||
|
<PageTitle>Unexpected Error</PageTitle>
|
||||||
|
|
||||||
|
<h1 class="display-1 fw-bold">500</h1>
|
||||||
|
<p class="fs-3"><span class="text-danger">Ops!</span> Unexpected error.</p>
|
||||||
|
<p class="lead">
|
||||||
|
An unexpected error occurred while loading the page. Please, wait a minute and try again.
|
||||||
|
</p>
|
||||||
|
}
|
||||||
|
|
||||||
|
<a title="Back to Home" href="/" class="btn btn-primary"><i class="bi bi-house-door-fill"></i> Back to Home</a>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
@code {
|
||||||
|
[CascadingParameter]
|
||||||
|
private HttpContext? HttpContext { get; set; }
|
||||||
|
|
||||||
|
[Parameter]
|
||||||
|
[SupplyParameterFromQuery(Name = "code")]
|
||||||
|
public int Code { get; set; }
|
||||||
|
}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
@page "/"
|
||||||
|
@rendermode @(new InteractiveServerRenderMode(prerender: false))
|
||||||
|
|
||||||
|
<PageTitle>SQL Database Vector Search</PageTitle>
|
||||||
|
|
||||||
|
<h1>SQL Database Vector Search</h1>
|
||||||
|
<p class="mt-3">
|
||||||
|
How to use the native VECTOR type in <img src="/images/sqldatabase.svg" /> Azure SQL Database to perform embeddings and RAG with <img src="/images/openai.svg" /> Azure OpenAI.
|
||||||
|
</p>
|
||||||
|
<p>
|
||||||
|
This application allows to load documents, generate embeddings and save them into the database as Vectors, and perform searches using Vector Search and RAG. Currently, PDF, DOCX, TXT and MD files are supported. Vectors are saved and retrieved with Entity Framework Core using the <a href="https://github.com/efcore/EfCore.SqlServer.VectorSearch" target="_blank">EFCore.SqlServer.VectorSearch</a> library. Embedding and Chat Completion are integrated with <a href="https://github.com/microsoft/semantic-kernel" target="_blank">Semantic Kernel</a>.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<h3>Supported features</h3>
|
||||||
|
<ul>
|
||||||
|
<li>Conversation history with question reformulation</li>
|
||||||
|
<li>Information about token usage</li>
|
||||||
|
<li>Response streaming</li>
|
||||||
|
</ul>
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
<Router AppAssembly="typeof(Program).Assembly">
|
||||||
|
<Found Context="routeData">
|
||||||
|
<RouteView RouteData="routeData" DefaultLayout="typeof(Layout.MainLayout)" />
|
||||||
|
<FocusOnNavigate RouteData="routeData" Selector="h1" />
|
||||||
|
</Found>
|
||||||
|
</Router>
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
@using System.ComponentModel.DataAnnotations
|
||||||
|
@using System.Net.Http
|
||||||
|
@using System.Net.Http.Json
|
||||||
|
@using Microsoft.AspNetCore.Components.Forms
|
||||||
|
@using Microsoft.AspNetCore.Components.Routing
|
||||||
|
@using Microsoft.AspNetCore.Components.Web
|
||||||
|
@using static Microsoft.AspNetCore.Components.Web.RenderMode
|
||||||
|
@using Microsoft.AspNetCore.Components.Web.Virtualization
|
||||||
|
@using Microsoft.JSInterop
|
||||||
|
@using SqlDatabaseVectorSearch
|
||||||
|
@using SqlDatabaseVectorSearch.Components
|
||||||
|
@using SqlDatabaseVectorSearch.Extensions
|
||||||
|
@using SqlDatabaseVectorSearch.Models
|
||||||
|
@using SqlDatabaseVectorSearch.Services
|
||||||
|
@using BlazorBootstrap
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
// <auto-generated />
|
||||||
|
using System;
|
||||||
|
using Microsoft.EntityFrameworkCore;
|
||||||
|
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||||
|
using Microsoft.EntityFrameworkCore.Metadata;
|
||||||
|
using Microsoft.EntityFrameworkCore.Migrations;
|
||||||
|
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||||
|
using SqlDatabaseVectorSearch.DataAccessLayer;
|
||||||
|
|
||||||
|
#nullable disable
|
||||||
|
|
||||||
|
namespace SqlDatabaseVectorSearch.DataAccessLayer.Migrations
|
||||||
|
{
|
||||||
|
[DbContext(typeof(ApplicationDbContext))]
|
||||||
|
[Migration("20250224102351_Initial")]
|
||||||
|
partial class Initial
|
||||||
|
{
|
||||||
|
/// <inheritdoc />
|
||||||
|
protected override void BuildTargetModel(ModelBuilder modelBuilder)
|
||||||
|
{
|
||||||
|
#pragma warning disable 612, 618
|
||||||
|
modelBuilder
|
||||||
|
.HasAnnotation("ProductVersion", "9.0.2")
|
||||||
|
.HasAnnotation("Relational:MaxIdentifierLength", 128);
|
||||||
|
|
||||||
|
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
|
||||||
|
|
||||||
|
modelBuilder.Entity("SqlDatabaseVectorSearch.DataAccessLayer.Entities.Document", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTimeOffset>("CreationDate")
|
||||||
|
.HasColumnType("datetimeoffset");
|
||||||
|
|
||||||
|
b.Property<string>("Name")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(255)
|
||||||
|
.HasColumnType("nvarchar(255)");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.ToTable("Documents", (string)null);
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("SqlDatabaseVectorSearch.DataAccessLayer.Entities.DocumentChunk", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("Content")
|
||||||
|
.IsRequired()
|
||||||
|
.HasColumnType("nvarchar(max)");
|
||||||
|
|
||||||
|
b.Property<Guid>("DocumentId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.PrimitiveCollection<string>("Embedding")
|
||||||
|
.IsRequired()
|
||||||
|
.HasColumnType("vector(1536)");
|
||||||
|
|
||||||
|
b.Property<int>("Index")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.HasIndex("DocumentId");
|
||||||
|
|
||||||
|
b.ToTable("DocumentChunks", (string)null);
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("SqlDatabaseVectorSearch.DataAccessLayer.Entities.DocumentChunk", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("SqlDatabaseVectorSearch.DataAccessLayer.Entities.Document", "Document")
|
||||||
|
.WithMany("Chunks")
|
||||||
|
.HasForeignKey("DocumentId")
|
||||||
|
.OnDelete(DeleteBehavior.Cascade)
|
||||||
|
.IsRequired()
|
||||||
|
.HasConstraintName("FK_DocumentChunks_Documents");
|
||||||
|
|
||||||
|
b.Navigation("Document");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("SqlDatabaseVectorSearch.DataAccessLayer.Entities.Document", b =>
|
||||||
|
{
|
||||||
|
b.Navigation("Chunks");
|
||||||
|
});
|
||||||
|
#pragma warning restore 612, 618
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
using System;
|
||||||
|
using Microsoft.EntityFrameworkCore.Migrations;
|
||||||
|
|
||||||
|
#nullable disable
|
||||||
|
|
||||||
|
namespace SqlDatabaseVectorSearch.DataAccessLayer.Migrations
|
||||||
|
{
|
||||||
|
/// <inheritdoc />
|
||||||
|
public partial class Initial : Migration
|
||||||
|
{
|
||||||
|
/// <inheritdoc />
|
||||||
|
protected override void Up(MigrationBuilder migrationBuilder)
|
||||||
|
{
|
||||||
|
migrationBuilder.CreateTable(
|
||||||
|
name: "Documents",
|
||||||
|
columns: table => new
|
||||||
|
{
|
||||||
|
Id = table.Column<Guid>(type: "uniqueidentifier", nullable: false),
|
||||||
|
Name = table.Column<string>(type: "nvarchar(255)", maxLength: 255, nullable: false),
|
||||||
|
CreationDate = table.Column<DateTimeOffset>(type: "datetimeoffset", nullable: false)
|
||||||
|
},
|
||||||
|
constraints: table =>
|
||||||
|
{
|
||||||
|
table.PrimaryKey("PK_Documents", x => x.Id);
|
||||||
|
});
|
||||||
|
|
||||||
|
migrationBuilder.CreateTable(
|
||||||
|
name: "DocumentChunks",
|
||||||
|
columns: table => new
|
||||||
|
{
|
||||||
|
Id = table.Column<Guid>(type: "uniqueidentifier", nullable: false),
|
||||||
|
DocumentId = table.Column<Guid>(type: "uniqueidentifier", nullable: false),
|
||||||
|
Index = table.Column<int>(type: "int", nullable: false),
|
||||||
|
Content = table.Column<string>(type: "nvarchar(max)", nullable: false),
|
||||||
|
Embedding = table.Column<string>(type: "vector(1536)", nullable: false)
|
||||||
|
},
|
||||||
|
constraints: table =>
|
||||||
|
{
|
||||||
|
table.PrimaryKey("PK_DocumentChunks", x => x.Id);
|
||||||
|
table.ForeignKey(
|
||||||
|
name: "FK_DocumentChunks_Documents",
|
||||||
|
column: x => x.DocumentId,
|
||||||
|
principalTable: "Documents",
|
||||||
|
principalColumn: "Id",
|
||||||
|
onDelete: ReferentialAction.Cascade);
|
||||||
|
});
|
||||||
|
|
||||||
|
migrationBuilder.CreateIndex(
|
||||||
|
name: "IX_DocumentChunks_DocumentId",
|
||||||
|
table: "DocumentChunks",
|
||||||
|
column: "DocumentId");
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <inheritdoc />
|
||||||
|
protected override void Down(MigrationBuilder migrationBuilder)
|
||||||
|
{
|
||||||
|
migrationBuilder.DropTable(
|
||||||
|
name: "DocumentChunks");
|
||||||
|
|
||||||
|
migrationBuilder.DropTable(
|
||||||
|
name: "Documents");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
// <auto-generated />
|
||||||
|
using System;
|
||||||
|
using Microsoft.EntityFrameworkCore;
|
||||||
|
using Microsoft.EntityFrameworkCore.Infrastructure;
|
||||||
|
using Microsoft.EntityFrameworkCore.Metadata;
|
||||||
|
using Microsoft.EntityFrameworkCore.Storage.ValueConversion;
|
||||||
|
using SqlDatabaseVectorSearch.DataAccessLayer;
|
||||||
|
|
||||||
|
#nullable disable
|
||||||
|
|
||||||
|
namespace SqlDatabaseVectorSearch.DataAccessLayer.Migrations
|
||||||
|
{
|
||||||
|
[DbContext(typeof(ApplicationDbContext))]
|
||||||
|
partial class ApplicationDbContextModelSnapshot : ModelSnapshot
|
||||||
|
{
|
||||||
|
protected override void BuildModel(ModelBuilder modelBuilder)
|
||||||
|
{
|
||||||
|
#pragma warning disable 612, 618
|
||||||
|
modelBuilder
|
||||||
|
.HasAnnotation("ProductVersion", "9.0.2")
|
||||||
|
.HasAnnotation("Relational:MaxIdentifierLength", 128);
|
||||||
|
|
||||||
|
SqlServerModelBuilderExtensions.UseIdentityColumns(modelBuilder);
|
||||||
|
|
||||||
|
modelBuilder.Entity("SqlDatabaseVectorSearch.DataAccessLayer.Entities.Document", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<DateTimeOffset>("CreationDate")
|
||||||
|
.HasColumnType("datetimeoffset");
|
||||||
|
|
||||||
|
b.Property<string>("Name")
|
||||||
|
.IsRequired()
|
||||||
|
.HasMaxLength(255)
|
||||||
|
.HasColumnType("nvarchar(255)");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.ToTable("Documents", (string)null);
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("SqlDatabaseVectorSearch.DataAccessLayer.Entities.DocumentChunk", b =>
|
||||||
|
{
|
||||||
|
b.Property<Guid>("Id")
|
||||||
|
.ValueGeneratedOnAdd()
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.Property<string>("Content")
|
||||||
|
.IsRequired()
|
||||||
|
.HasColumnType("nvarchar(max)");
|
||||||
|
|
||||||
|
b.Property<Guid>("DocumentId")
|
||||||
|
.HasColumnType("uniqueidentifier");
|
||||||
|
|
||||||
|
b.PrimitiveCollection<string>("Embedding")
|
||||||
|
.IsRequired()
|
||||||
|
.HasColumnType("vector(1536)");
|
||||||
|
|
||||||
|
b.Property<int>("Index")
|
||||||
|
.HasColumnType("int");
|
||||||
|
|
||||||
|
b.HasKey("Id");
|
||||||
|
|
||||||
|
b.HasIndex("DocumentId");
|
||||||
|
|
||||||
|
b.ToTable("DocumentChunks", (string)null);
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("SqlDatabaseVectorSearch.DataAccessLayer.Entities.DocumentChunk", b =>
|
||||||
|
{
|
||||||
|
b.HasOne("SqlDatabaseVectorSearch.DataAccessLayer.Entities.Document", "Document")
|
||||||
|
.WithMany("Chunks")
|
||||||
|
.HasForeignKey("DocumentId")
|
||||||
|
.OnDelete(DeleteBehavior.Cascade)
|
||||||
|
.IsRequired()
|
||||||
|
.HasConstraintName("FK_DocumentChunks_Documents");
|
||||||
|
|
||||||
|
b.Navigation("Document");
|
||||||
|
});
|
||||||
|
|
||||||
|
modelBuilder.Entity("SqlDatabaseVectorSearch.DataAccessLayer.Entities.Document", b =>
|
||||||
|
{
|
||||||
|
b.Navigation("Chunks");
|
||||||
|
});
|
||||||
|
#pragma warning restore 612, 618
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,40 @@
|
|||||||
|
using System.Text.RegularExpressions;
|
||||||
|
using Microsoft.Net.Http.Headers;
|
||||||
|
|
||||||
|
namespace SqlDatabaseVectorSearch.Extensions;
|
||||||
|
|
||||||
|
public static partial class RequestExtensions
|
||||||
|
{
|
||||||
|
[GeneratedRegex("(android|bb\\d+|meego).+mobile|avantgo|bada\\/|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|ip(hone|od)|iris|kindle|lge |maemo|midp|mmp|mobile.+firefox|netfront|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\\/|plucker|pocket|psp|series(4|6)0|symbian|treo|up\\.(browser|link)|vodafone|wap|windows ce|xda|xiino", RegexOptions.IgnoreCase | RegexOptions.Multiline | RegexOptions.Compiled)]
|
||||||
|
private static partial Regex MobileBrowserRegex { get; }
|
||||||
|
|
||||||
|
[GeneratedRegex("1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\\-(n|u)|c55\\/|capi|ccwa|cdm\\-|cell|chtm|cldc|cmd\\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\\-s|devi|dica|dmob|do(c|p)o|ds(12|\\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\\-|_)|g1 u|g560|gene|gf\\-5|g\\-mo|go(\\.w|od)|gr(ad|un)|haie|hcit|hd\\-(m|p|t)|hei\\-|hi(pt|ta)|hp( i|ip)|hs\\-c|ht(c(\\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\\-(20|go|ma)|i230|iac( |\\-|\\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\\/)|klon|kpt |kwc\\-|kyo(c|k)|le(no|xi)|lg( g|\\/(k|l|u)|50|54|\\-[a-w])|libw|lynx|m1\\-w|m3ga|m50\\/|ma(te|ui|xo)|mc(01|21|ca)|m\\-cr|me(rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\\-2|po(ck|rt|se)|prox|psio|pt\\-g|qa\\-a|qc(07|12|21|32|60|\\-[2-7]|i\\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\\-|oo|p\\-)|sdk\\/|se(c(\\-|0|1)|47|mc|nd|ri)|sgh\\-|shar|sie(\\-|m)|sk\\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\\-|v\\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\\-|tdg\\-|tel(i|m)|tim\\-|t\\-mo|to(pl|sh)|ts(70|m\\-|m3|m5)|tx\\-9|up(\\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|yas\\-|your|zeto|zte\\-", RegexOptions.IgnoreCase | RegexOptions.Multiline | RegexOptions.Compiled)]
|
||||||
|
private static partial Regex MobileBrowserVersionRegex { get; }
|
||||||
|
|
||||||
|
[GeneratedRegex(@"^/(?<culture>[a-z]{2})(/|$)", RegexOptions.IgnoreCase | RegexOptions.Compiled)]
|
||||||
|
private static partial Regex RouteCultureRegex { get; }
|
||||||
|
|
||||||
|
public static bool IsMobileRequest(this HttpContext httpContext)
|
||||||
|
=> httpContext.Request.IsMobile();
|
||||||
|
|
||||||
|
public static bool IsMobile(this HttpRequest request)
|
||||||
|
{
|
||||||
|
var userAgent = request.Headers[HeaderNames.UserAgent].ToString();
|
||||||
|
var isMobileBrowser = false;
|
||||||
|
if (userAgent?.Length > 4 && (MobileBrowserRegex.IsMatch(userAgent) || MobileBrowserVersionRegex.IsMatch(userAgent.AsSpan(0, 4))))
|
||||||
|
{
|
||||||
|
isMobileBrowser = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
return isMobileBrowser;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static bool IsApiRequest(this HttpContext httpContext)
|
||||||
|
=> httpContext.Request.Path.StartsWithSegments("/api");
|
||||||
|
|
||||||
|
public static bool IsSwaggerRequest(this HttpContext httpContext)
|
||||||
|
=> httpContext.Request.Path.StartsWithSegments("/swagger");
|
||||||
|
|
||||||
|
public static bool IsWebRequest(this HttpContext httpContext)
|
||||||
|
=> !httpContext.IsApiRequest() && !httpContext.IsSwaggerRequest();
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
namespace SqlDatabaseVectorSearch.Extensions;
|
||||||
|
|
||||||
|
public static class StreamExtensions
|
||||||
|
{
|
||||||
|
public static async Task<MemoryStream> GetMemoryStreamAsync(this Stream stream)
|
||||||
|
{
|
||||||
|
// Use a BufferedStream to read the file in chunks
|
||||||
|
using var bufferedStream = new BufferedStream(stream);
|
||||||
|
|
||||||
|
var ms = new MemoryStream();
|
||||||
|
await bufferedStream.CopyToAsync(ms);
|
||||||
|
|
||||||
|
ms.Position = 0;
|
||||||
|
return ms;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -5,8 +5,10 @@ using Microsoft.AspNetCore.Http.HttpResults;
|
|||||||
using Microsoft.EntityFrameworkCore;
|
using Microsoft.EntityFrameworkCore;
|
||||||
using Microsoft.SemanticKernel;
|
using Microsoft.SemanticKernel;
|
||||||
using MimeMapping;
|
using MimeMapping;
|
||||||
|
using SqlDatabaseVectorSearch.Components;
|
||||||
using SqlDatabaseVectorSearch.ContentDecoders;
|
using SqlDatabaseVectorSearch.ContentDecoders;
|
||||||
using SqlDatabaseVectorSearch.DataAccessLayer;
|
using SqlDatabaseVectorSearch.DataAccessLayer;
|
||||||
|
using SqlDatabaseVectorSearch.Extensions;
|
||||||
using SqlDatabaseVectorSearch.Models;
|
using SqlDatabaseVectorSearch.Models;
|
||||||
using SqlDatabaseVectorSearch.Services;
|
using SqlDatabaseVectorSearch.Services;
|
||||||
using SqlDatabaseVectorSearch.Settings;
|
using SqlDatabaseVectorSearch.Settings;
|
||||||
@@ -21,6 +23,11 @@ builder.Configuration.AddJsonFile("appsettings.local.json", optional: true, relo
|
|||||||
var aiSettings = builder.Services.ConfigureAndGet<AzureOpenAISettings>(builder.Configuration, "AzureOpenAI")!;
|
var aiSettings = builder.Services.ConfigureAndGet<AzureOpenAISettings>(builder.Configuration, "AzureOpenAI")!;
|
||||||
var appSettings = builder.Services.ConfigureAndGet<AppSettings>(builder.Configuration, nameof(AppSettings))!;
|
var appSettings = builder.Services.ConfigureAndGet<AppSettings>(builder.Configuration, nameof(AppSettings))!;
|
||||||
|
|
||||||
|
builder.Services.AddRazorComponents()
|
||||||
|
.AddInteractiveServerComponents();
|
||||||
|
|
||||||
|
builder.Services.AddBlazorBootstrap();
|
||||||
|
|
||||||
builder.Services.ConfigureHttpJsonOptions(options =>
|
builder.Services.ConfigureHttpJsonOptions(options =>
|
||||||
{
|
{
|
||||||
options.SerializerOptions.Converters.Add(new JsonStringEnumConverter());
|
options.SerializerOptions.Converters.Add(new JsonStringEnumConverter());
|
||||||
@@ -79,28 +86,55 @@ builder.Services.AddDefaultProblemDetails();
|
|||||||
builder.Services.AddDefaultExceptionHandler();
|
builder.Services.AddDefaultExceptionHandler();
|
||||||
|
|
||||||
var app = builder.Build();
|
var app = builder.Build();
|
||||||
|
await ConfigureDatabaseAsync(app.Services);
|
||||||
|
|
||||||
// Configure the HTTP request pipeline.
|
// Configure the HTTP request pipeline.
|
||||||
app.UseHttpsRedirection();
|
app.UseHttpsRedirection();
|
||||||
|
|
||||||
app.UseExceptionHandler(new ExceptionHandlerOptions
|
app.UseWhen(context => context.IsWebRequest(), builder =>
|
||||||
{
|
{
|
||||||
StatusCodeSelector = exception => exception switch
|
if (!app.Environment.IsDevelopment())
|
||||||
{
|
{
|
||||||
NotSupportedException => StatusCodes.Status501NotImplemented,
|
builder.UseExceptionHandler("/error");
|
||||||
_ => StatusCodes.Status500InternalServerError
|
|
||||||
|
// The default HSTS value is 30 days.
|
||||||
|
builder.UseHsts();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
builder.UseStatusCodePagesWithRedirects("/error?code={0}");
|
||||||
});
|
});
|
||||||
|
|
||||||
app.UseStatusCodePages();
|
app.UseWhen(context => context.IsApiRequest(), builder =>
|
||||||
|
{
|
||||||
|
app.UseExceptionHandler(new ExceptionHandlerOptions
|
||||||
|
{
|
||||||
|
StatusCodeSelector = exception => exception switch
|
||||||
|
{
|
||||||
|
NotSupportedException => StatusCodes.Status501NotImplemented,
|
||||||
|
_ => StatusCodes.Status500InternalServerError
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
builder.UseStatusCodePages();
|
||||||
|
});
|
||||||
|
|
||||||
app.MapOpenApi();
|
app.MapOpenApi();
|
||||||
app.UseSwaggerUI(options =>
|
app.UseSwaggerUI(options =>
|
||||||
{
|
{
|
||||||
options.RoutePrefix = string.Empty;
|
|
||||||
options.SwaggerEndpoint("/openapi/v1.json", builder.Environment.ApplicationName);
|
options.SwaggerEndpoint("/openapi/v1.json", builder.Environment.ApplicationName);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
app.UseRouting();
|
||||||
|
// app.UseRateLimiter();
|
||||||
|
app.UseRequestLocalization();
|
||||||
|
// app.UseCors();
|
||||||
|
|
||||||
|
app.UseAntiforgery();
|
||||||
|
|
||||||
|
app.MapStaticAssets();
|
||||||
|
app.MapRazorComponents<App>()
|
||||||
|
.AddInteractiveServerRenderMode();
|
||||||
|
|
||||||
app.MapPost("/api/ask", async (Question question, VectorSearchService vectorSearchService, CancellationToken cancellationToken,
|
app.MapPost("/api/ask", async (Question question, VectorSearchService vectorSearchService, CancellationToken cancellationToken,
|
||||||
[Description("If true, the question will be reformulated taking into account the context of the chat identified by the given ConversationId.")] bool reformulate = true) =>
|
[Description("If true, the question will be reformulated taking into account the context of the chat identified by the given ConversationId.")] bool reformulate = true) =>
|
||||||
{
|
{
|
||||||
@@ -185,3 +219,11 @@ documentsApiGroup.MapDelete("{documentId:guid}", async (Guid documentId, Documen
|
|||||||
.WithDescription("This endpoint deletes the document and all its chunks.");
|
.WithDescription("This endpoint deletes the document and all its chunks.");
|
||||||
|
|
||||||
app.Run();
|
app.Run();
|
||||||
|
|
||||||
|
static async Task ConfigureDatabaseAsync(IServiceProvider serviceProvider)
|
||||||
|
{
|
||||||
|
await using var scope = serviceProvider.CreateAsyncScope();
|
||||||
|
var dbContext = scope.ServiceProvider.GetRequiredService<ApplicationDbContext>();
|
||||||
|
|
||||||
|
await dbContext.Database.MigrateAsync();
|
||||||
|
}
|
||||||
@@ -5,8 +5,7 @@
|
|||||||
"commandName": "Project",
|
"commandName": "Project",
|
||||||
"dotnetRunMessages": true,
|
"dotnetRunMessages": true,
|
||||||
"launchBrowser": true,
|
"launchBrowser": true,
|
||||||
"launchUrl": "",
|
"applicationUrl": "https://localhost:7025;http://localhost:5178",
|
||||||
"applicationUrl": "https://localhost:7024;http://localhost:5178",
|
|
||||||
"environmentVariables": {
|
"environmentVariables": {
|
||||||
"ASPNETCORE_ENVIRONMENT": "Development"
|
"ASPNETCORE_ENVIRONMENT": "Development"
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -36,4 +36,7 @@ public class DocumentService(ApplicationDbContext dbContext)
|
|||||||
|
|
||||||
public Task DeleteAsync(Guid documentId, CancellationToken cancellationToken = default)
|
public Task DeleteAsync(Guid documentId, CancellationToken cancellationToken = default)
|
||||||
=> dbContext.Documents.Where(d => d.Id == documentId).ExecuteDeleteAsync(cancellationToken);
|
=> dbContext.Documents.Where(d => d.Id == documentId).ExecuteDeleteAsync(cancellationToken);
|
||||||
|
|
||||||
|
public Task DeleteAsync(IEnumerable<Guid> documentIds, CancellationToken cancellationToken = default)
|
||||||
|
=> dbContext.Documents.Where(d => documentIds.Contains(d.Id)).ExecuteDeleteAsync(cancellationToken);
|
||||||
}
|
}
|
||||||
@@ -8,22 +8,27 @@
|
|||||||
</PropertyGroup>
|
</PropertyGroup>
|
||||||
|
|
||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
|
<PackageReference Include="Blazor.Bootstrap" Version="3.3.1" />
|
||||||
<PackageReference Include="DocumentFormat.OpenXml" Version="3.2.0" />
|
<PackageReference Include="DocumentFormat.OpenXml" Version="3.2.0" />
|
||||||
<PackageReference Include="EFCore.SqlServer.VectorSearch" Version="9.0.0-preview.2" />
|
<PackageReference Include="EFCore.SqlServer.VectorSearch" Version="9.0.0-preview.2" />
|
||||||
<PackageReference Include="EntityFrameworkCore.Exceptions.SqlServer" Version="8.1.3" />
|
<PackageReference Include="EntityFrameworkCore.Exceptions.SqlServer" Version="8.1.3" />
|
||||||
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="9.0.2" />
|
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="9.0.2" />
|
||||||
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="9.0.2" />
|
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer" Version="9.0.2" />
|
||||||
|
<PackageReference Include="Microsoft.EntityFrameworkCore.Tools" Version="9.0.2">
|
||||||
|
<PrivateAssets>all</PrivateAssets>
|
||||||
|
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||||
|
</PackageReference>
|
||||||
<PackageReference Include="Microsoft.Extensions.Caching.Hybrid" Version="9.2.0-preview.1.25105.6" />
|
<PackageReference Include="Microsoft.Extensions.Caching.Hybrid" Version="9.2.0-preview.1.25105.6" />
|
||||||
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="9.2.0" />
|
<PackageReference Include="Microsoft.Extensions.Http.Resilience" Version="9.2.0" />
|
||||||
<PackageReference Include="Microsoft.ML.Tokenizers" Version="1.0.1" />
|
<PackageReference Include="Microsoft.ML.Tokenizers" Version="1.0.2" />
|
||||||
<PackageReference Include="Microsoft.ML.Tokenizers.Data.Cl100kBase" Version="1.0.1" />
|
<PackageReference Include="Microsoft.ML.Tokenizers.Data.Cl100kBase" Version="1.0.2" />
|
||||||
<PackageReference Include="Microsoft.ML.Tokenizers.Data.O200kBase" Version="1.0.1" />
|
<PackageReference Include="Microsoft.ML.Tokenizers.Data.O200kBase" Version="1.0.2" />
|
||||||
<PackageReference Include="Microsoft.SemanticKernel" Version="1.37.0" />
|
<PackageReference Include="Microsoft.SemanticKernel" Version="1.40.0" />
|
||||||
<PackageReference Include="MimeMapping" Version="3.1.0" />
|
<PackageReference Include="MimeMapping" Version="3.1.0" />
|
||||||
<PackageReference Include="MinimalHelpers.OpenApi" Version="2.1.4" />
|
<PackageReference Include="MinimalHelpers.OpenApi" Version="2.1.4" />
|
||||||
<PackageReference Include="PdfPig" Version="0.1.9" />
|
<PackageReference Include="PdfPig" Version="0.1.9" />
|
||||||
<PackageReference Include="Swashbuckle.AspNetCore.SwaggerUI" Version="7.2.0" />
|
<PackageReference Include="Swashbuckle.AspNetCore.SwaggerUI" Version="7.3.1" />
|
||||||
<PackageReference Include="TinyHelpers.AspNetCore" Version="4.0.19" />
|
<PackageReference Include="TinyHelpers.AspNetCore" Version="4.0.20" />
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
|
|
||||||
</Project>
|
</Project>
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
body, html {
|
||||||
|
margin: 0;
|
||||||
|
padding: 0;
|
||||||
|
height: 100%;
|
||||||
|
}
|
||||||
|
|
||||||
|
:root {
|
||||||
|
--bb-sidebar2-width: 270px;
|
||||||
|
--bb-sidebar2-collapsed-width: 50px;
|
||||||
|
--bb-sidebar2-background-color: rgba(234, 234, 234, 1);
|
||||||
|
--bb-sidebar2-top-row-background-color: rgba(0,0,0,0.08);
|
||||||
|
--bb-sidebar2-top-row-border-color: rgb(194,192,192);
|
||||||
|
--bb-sidebar2-title-text-color: rgb(0,0,0);
|
||||||
|
--bb-sidebar2-brand-icon-color: rgb(0,0,0);
|
||||||
|
--bb-sidebar2-brand-image-width: 24px;
|
||||||
|
--bb-sidebar2-brand-image-height: 24px;
|
||||||
|
--bb-sidebar2-title-badge-text-color: rgb(255,255,255);
|
||||||
|
--bb-sidebar2-title-badge-background-color: rgba(25,135,84,var(--bs-bg-opacity,1));
|
||||||
|
--bb-sidebar2-navbar-toggler-icon-color: rgb(0,0,0);
|
||||||
|
--bb-sidebar2-navbar-toggler-background-color: rgba(0,0,0,0.08);
|
||||||
|
--bb-sidebar2-content-border-color: rgb(194,192,192);
|
||||||
|
--bb-sidebar2-nav-item-text-color: rgba(0,0,0,0.9);
|
||||||
|
--bb-sidebar2-nav-item-text-active-color-rgb: 0,0,0;
|
||||||
|
--bb-sidebar2-nav-item-text-hover-color: rgba(var(--bb-sidebar-nav-item-text-active-color-rgb),0.9);
|
||||||
|
--bb-sidebar2-nav-item-text-active-color: rgba(var(--bb-sidebar-nav-item-text-active-color-rgb),0.9);
|
||||||
|
--bb-sidebar2-nav-item-background-hover-color: rgba(var(--bb-sidebar-nav-item-text-active-color-rgb),0.08);
|
||||||
|
--bb-sidebar2-nav-item-group-background-color: rgba(var(--bb-sidebar-nav-item-text-active-color-rgb),0.08);
|
||||||
|
}
|
||||||
|
|
||||||
|
.bb-sidebar2 nav .nav-item a:hover {
|
||||||
|
background-color: rgba(0,0,0,0.08) !important;
|
||||||
|
color: rgba(0,0,0,0.9) !important;
|
||||||
|
}
|
||||||
|
|
||||||
|
.bb-sidebar2 nav .nav-item a.active {
|
||||||
|
background-color: rgb(194,192,192) !important;
|
||||||
|
color: rgba(0,0,0,0.9) !important;
|
||||||
|
}
|
||||||
|
|
||||||
|
h1:focus {
|
||||||
|
outline: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.valid.modified:not([type=checkbox]) {
|
||||||
|
outline: 1px solid #26b050;
|
||||||
|
}
|
||||||
|
|
||||||
|
.invalid {
|
||||||
|
outline: 1px solid red;
|
||||||
|
}
|
||||||
|
|
||||||
|
.validation-message {
|
||||||
|
color: red;
|
||||||
|
}
|
||||||
|
|
||||||
|
.blazor-error-boundary {
|
||||||
|
background: url(data:image/svg+xml;base64,PHN2ZyB3aWR0aD0iNTYiIGhlaWdodD0iNDkiIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgeG1sbnM6eGxpbms9Imh0dHA6Ly93d3cudzMub3JnLzE5OTkveGxpbmsiIG92ZXJmbG93PSJoaWRkZW4iPjxkZWZzPjxjbGlwUGF0aCBpZD0iY2xpcDAiPjxyZWN0IHg9IjIzNSIgeT0iNTEiIHdpZHRoPSI1NiIgaGVpZ2h0PSI0OSIvPjwvY2xpcFBhdGg+PC9kZWZzPjxnIGNsaXAtcGF0aD0idXJsKCNjbGlwMCkiIHRyYW5zZm9ybT0idHJhbnNsYXRlKC0yMzUgLTUxKSI+PHBhdGggZD0iTTI2My41MDYgNTFDMjY0LjcxNyA1MSAyNjUuODEzIDUxLjQ4MzcgMjY2LjYwNiA1Mi4yNjU4TDI2Ny4wNTIgNTIuNzk4NyAyNjcuNTM5IDUzLjYyODMgMjkwLjE4NSA5Mi4xODMxIDI5MC41NDUgOTIuNzk1IDI5MC42NTYgOTIuOTk2QzI5MC44NzcgOTMuNTEzIDI5MSA5NC4wODE1IDI5MSA5NC42NzgyIDI5MSA5Ny4wNjUxIDI4OS4wMzggOTkgMjg2LjYxNyA5OUwyNDAuMzgzIDk5QzIzNy45NjMgOTkgMjM2IDk3LjA2NTEgMjM2IDk0LjY3ODIgMjM2IDk0LjM3OTkgMjM2LjAzMSA5NC4wODg2IDIzNi4wODkgOTMuODA3MkwyMzYuMzM4IDkzLjAxNjIgMjM2Ljg1OCA5Mi4xMzE0IDI1OS40NzMgNTMuNjI5NCAyNTkuOTYxIDUyLjc5ODUgMjYwLjQwNyA1Mi4yNjU4QzI2MS4yIDUxLjQ4MzcgMjYyLjI5NiA1MSAyNjMuNTA2IDUxWk0yNjMuNTg2IDY2LjAxODNDMjYwLjczNyA2Ni4wMTgzIDI1OS4zMTMgNjcuMTI0NSAyNTkuMzEzIDY5LjMzNyAyNTkuMzEzIDY5LjYxMDIgMjU5LjMzMiA2OS44NjA4IDI1OS4zNzEgNzAuMDg4N0wyNjEuNzk1IDg0LjAxNjEgMjY1LjM4IDg0LjAxNjEgMjY3LjgyMSA2OS43NDc1QzI2Ny44NiA2OS43MzA5IDI2Ny44NzkgNjkuNTg3NyAyNjcuODc5IDY5LjMxNzkgMjY3Ljg3OSA2Ny4xMTgyIDI2Ni40NDggNjYuMDE4MyAyNjMuNTg2IDY2LjAxODNaTTI2My41NzYgODYuMDU0N0MyNjEuMDQ5IDg2LjA1NDcgMjU5Ljc4NiA4Ny4zMDA1IDI1OS43ODYgODkuNzkyMSAyNTkuNzg2IDkyLjI4MzcgMjYxLjA0OSA5My41Mjk1IDI2My41NzYgOTMuNTI5NSAyNjYuMTE2IDkzLjUyOTUgMjY3LjM4NyA5Mi4yODM3IDI2Ny4zODcgODkuNzkyMSAyNjcuMzg3IDg3LjMwMDUgMjY2LjExNiA4Ni4wNTQ3IDI2My41NzYgODYuMDU0N1oiIGZpbGw9IiNGRkU1MDAiIGZpbGwtcnVsZT0iZXZlbm9kZCIvPjwvZz48L3N2Zz4=) no-repeat 1rem/1.8rem, #b32121;
|
||||||
|
padding: 1rem 1rem 1rem 3.7rem;
|
||||||
|
color: white;
|
||||||
|
}
|
||||||
|
|
||||||
|
.blazor-error-boundary::after {
|
||||||
|
content: "An error has occurred."
|
||||||
|
}
|
||||||
|
After Width: | Height: | Size: 1.8 KiB |
|
After Width: | Height: | Size: 2.0 KiB |
@@ -0,0 +1 @@
|
|||||||
|
<svg id="uuid-adbdae8e-5a41-46d1-8c18-aa73cdbfee32" xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 18 18"><defs><radialGradient id="uuid-2a7407aa-b787-48dd-a96a-0d81ab6e93bb" cx="-67.981" cy="793.199" r=".45" gradientTransform="translate(-17939.03 20368.029) rotate(45) scale(25.091 -34.149)" gradientUnits="userSpaceOnUse"><stop offset="0" stop-color="#83b9f9" /><stop offset="1" stop-color="#0078d4" /></radialGradient></defs><path d="m0,2.7v12.6c0,1.491,1.209,2.7,2.7,2.7h12.6c1.491,0,2.7-1.209,2.7-2.7V2.7c0-1.491-1.209-2.7-2.7-2.7H2.7C1.209,0,0,1.209,0,2.7ZM10.8,0v3.6c0,3.976,3.224,7.2,7.2,7.2h-3.6c-3.976,0-7.199,3.222-7.2,7.198v-3.598c0-3.976-3.224-7.2-7.2-7.2h3.6c3.976,0,7.2-3.224,7.2-7.2Z" fill="url(#uuid-2a7407aa-b787-48dd-a96a-0d81ab6e93bb)" stroke-width="0" /></svg>
|
||||||
|
After Width: | Height: | Size: 805 B |
@@ -0,0 +1 @@
|
|||||||
|
<svg id="a96792b7-ce28-4ca3-9767-4e065ef4820f" xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 18 18"><defs><linearGradient id="ef16bf9d-a8b6-4181-b6cd-66fc5203f956" x1="2.59" y1="10.16" x2="15.41" y2="10.16" gradientUnits="userSpaceOnUse"><stop offset="0" stop-color="#005ba1" /><stop offset="0.07" stop-color="#0060a9" /><stop offset="0.36" stop-color="#0071c8" /><stop offset="0.52" stop-color="#0078d4" /><stop offset="0.64" stop-color="#0074cd" /><stop offset="0.82" stop-color="#006abb" /><stop offset="1" stop-color="#005ba1" /></linearGradient><radialGradient id="bf3846c3-4d74-4743-ab9a-f334c248bd92" cx="9.36" cy="10.57" r="7.07" gradientUnits="userSpaceOnUse"><stop offset="0" stop-color="#f2f2f2" /><stop offset="0.58" stop-color="#eee" /><stop offset="1" stop-color="#e6e6e6" /></radialGradient></defs><title>Icon-databases-130</title><path d="M9,5.14c-3.54,0-6.41-1-6.41-2.32V15.18c0,1.27,2.82,2.3,6.32,2.32H9c3.54,0,6.41-1,6.41-2.32V2.82C15.41,4.11,12.54,5.14,9,5.14Z" fill="url(#ef16bf9d-a8b6-4181-b6cd-66fc5203f956)" /><path d="M15.41,2.82c0,1.29-2.87,2.32-6.41,2.32s-6.41-1-6.41-2.32S5.46.5,9,.5s6.41,1,6.41,2.32" fill="#e8e8e8" /><path d="M13.92,2.63c0,.82-2.21,1.48-4.92,1.48S4.08,3.45,4.08,2.63,6.29,1.16,9,1.16s4.92.66,4.92,1.47" fill="#50e6ff" /><path d="M9,3a11.55,11.55,0,0,0-3.89.57A11.42,11.42,0,0,0,9,4.11a11.15,11.15,0,0,0,3.89-.58A11.84,11.84,0,0,0,9,3Z" fill="#198ab3" /><path d="M12.9,11.4V8H12v4.13h2.46V11.4ZM5.76,9.73a1.83,1.83,0,0,1-.51-.31.44.44,0,0,1-.12-.32.34.34,0,0,1,.15-.3.68.68,0,0,1,.42-.12,1.62,1.62,0,0,1,1,.29V8.11a2.58,2.58,0,0,0-1-.16,1.64,1.64,0,0,0-1.09.34,1.08,1.08,0,0,0-.42.89c0,.51.32.91,1,1.21a2.88,2.88,0,0,1,.62.36.42.42,0,0,1,.15.32.38.38,0,0,1-.16.31.81.81,0,0,1-.45.11,1.66,1.66,0,0,1-1.09-.42V12a2.17,2.17,0,0,0,1.07.24,1.88,1.88,0,0,0,1.18-.33A1.08,1.08,0,0,0,6.84,11a1.05,1.05,0,0,0-.25-.7A2.42,2.42,0,0,0,5.76,9.73ZM11,11.32a2.34,2.34,0,0,0,.33-1.26A2.32,2.32,0,0,0,11,9a1.81,1.81,0,0,0-.7-.75,2,2,0,0,0-1-.26,2.11,2.11,0,0,0-1.08.27A1.86,1.86,0,0,0,7.49,9a2.46,2.46,0,0,0-.26,1.14,2.26,2.26,0,0,0,.24,1,1.76,1.76,0,0,0,.69.74,2.06,2.06,0,0,0,1,.3l.86,1h1.21L10,12.08A1.79,1.79,0,0,0,11,11.32ZM10,11.07a.94.94,0,0,1-.76.35.92.92,0,0,1-.76-.36,1.52,1.52,0,0,1-.29-1,1.53,1.53,0,0,1,.29-1,1,1,0,0,1,.78-.37.87.87,0,0,1,.75.37,1.62,1.62,0,0,1,.27,1A1.46,1.46,0,0,1,10,11.07Z" fill="url(#bf3846c3-4d74-4743-ab9a-f334c248bd92)" /></svg>
|
||||||
|
After Width: | Height: | Size: 2.4 KiB |
|
After Width: | Height: | Size: 1010 B |
@@ -0,0 +1,15 @@
|
|||||||
|
window.setFocus = (element) => {
|
||||||
|
if (element) {
|
||||||
|
element.focus();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
window.scrollTo = (element) => {
|
||||||
|
if (element) {
|
||||||
|
element.scrollIntoView();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function getLocalTime(utcDateTime) {
|
||||||
|
return new Date(utcDateTime).toLocaleString();
|
||||||
|
}
|
||||||
|
Before Width: | Height: | Size: 70 KiB After Width: | Height: | Size: 70 KiB |
|
After Width: | Height: | Size: 85 KiB |