Commit Graph

265 Commits

Author SHA1 Message Date
Marco Minerva d6c0e6630f Rename and enhance ChatService methods
Renamed `CreateQuestionAsync` to `CreateReformulateQuestionAsync` for clearer intent. Updated `CreateChatAsync` to include a new `ChatSystemPrompt` property in `AzureOpenAIPromptExecutionSettings` and modified `ChatHistory` initialization to simplify setup. Removed redundant `prompt` string construction and added user messages directly to `ChatHistory`.

Updated `VectorSearchService` to use the renamed `CreateReformulateQuestionAsync` method for consistency. These changes improve code clarity, maintainability, and functionality.
2025-11-17 15:49:42 +01:00
Marco Minerva dba2fb1c41 Update to stable version and enhance OpenAPI configuration
Updated `ProductVersion` in `ApplicationDbContextModelSnapshot.cs`
to target the stable release version `10.0.0`.

Added `TinyHelpers.AspNetCore.OpenApi` to `Program.cs` and
enabled OpenAPI options by uncommenting `RemoveServerList`
and `AddDefaultProblemDetailsResponse`. These changes improve
API documentation and error handling.
2025-11-17 10:20:36 +01:00
Marco Minerva 502c8756b1 Update C# version in guidelines to C# 14
Updated the copilot-instructions.md file to reflect the use of the latest version of C#, changing the reference from "C# 13" to "C# 14". This ensures the guidelines remain up-to-date with the latest language features.
2025-11-13 18:05:10 +01:00
Marco Minerva 522da4d5d0 Refactor embeddings and add reconnect modal
Enhanced support for SQL Server's `SqlVector<float>` for embeddings, replacing `float[]` to improve vector search compatibility. Added a reconnect modal for better handling of server disconnections, including UI, styles, and JavaScript logic.

Upgraded to .NET 10.0 and EF Core 10.0, updated dependencies, and improved token counting logic for embeddings. Updated documentation and configuration files to reflect these changes. Migrated database schema to support `SqlVector<float>`.
2025-11-13 17:59:06 +01:00
Marco Minerva d6458892f7 Update package dependencies to stable and latest versions
Updated multiple NuGet package dependencies to their latest stable versions, including:
- `Microsoft.AspNetCore.OpenApi` to `10.0.0`
- `Microsoft.EntityFrameworkCore.SqlServer` to `10.0.0`
- `Microsoft.Extensions.Caching.Hybrid` to `10.0.0`
- `Microsoft.ML.Tokenizers` to `2.0.0`
- `Microsoft.SemanticKernel` to `1.67.1`
- `Swashbuckle.AspNetCore.SwaggerUI` to `10.0.1`
- Other minor updates to `PdfPig`, `TinyHelpers.AspNetCore`, and more.

These updates ensure compatibility with the latest features and improvements.
2025-11-13 17:57:27 +01:00
Marco Minerva 5ac8175080 Update package dependencies to latest versions
Updated the following NuGet packages to their latest versions:
- `FluentValidation.DependencyInjectionExtensions` to `12.1.0`
- `Microsoft.ML.Tokenizers` to `1.0.3`
- `Microsoft.ML.Tokenizers.Data.Cl100kBase` to `1.0.3`
- `Microsoft.ML.Tokenizers.Data.O200kBase` to `1.0.3`
- `Microsoft.SemanticKernel` to `1.67.0`

Removed older versions of the above packages and replaced them with the updated versions.
2025-11-04 09:57:41 +01:00
Marco Minerva bef955831d Update package versions for FluentValidation and SemanticKernel
Updated the `FluentValidation.DependencyInjectionExtensions` package
from version 12.0.0 to 12.1.0 to incorporate potential bug fixes
and improvements.

Updated the `Microsoft.SemanticKernel` package from version 1.66.0
to 1.67.0, which may include enhancements, bug fixes, or new
functionality.
2025-11-04 09:57:03 +01:00
Marco Minerva b4e47ef552 Update Microsoft.ML.Tokenizers to version 1.0.3
Updated the `<PackageReference>` for `Microsoft.ML.Tokenizers` and its related data packages (`Data.Cl100kBase` and `Data.O200kBase`) from version `1.0.2` to `1.0.3`. These updates may include bug fixes, performance improvements, or new features introduced in the latest version.
2025-10-31 17:37:56 +01:00
Marco Minerva 59877ac6c2 Merge branch 'master' of https://github.com/marcominerva/SqlDatabaseVectorSearch 2025-10-22 12:46:50 +02:00
Marco Minerva 5fcfd11326 Update package versions for various dependencies
Updated the following package versions:
- Microsoft.AspNetCore.OpenApi to 9.0.10
- Microsoft.EntityFrameworkCore.SqlServer to 9.0.10
- Microsoft.EntityFrameworkCore.Tools to 9.0.10
- Microsoft.Extensions.Caching.Hybrid to 9.10.0
- Microsoft.Extensions.Http.Resilience to 9.10.0
- Microsoft.SemanticKernel to 1.66.0
- TinyHelpers.AspNetCore to 4.1.8

These updates ensure compatibility and leverage the latest features and fixes.
2025-10-22 12:46:46 +02:00
Marco Minerva 0cc969c164 Switch to CountEmbeddingTokens in chunkers
Updated the token counting method from CountChatCompletionTokens to CountEmbeddingTokens in VectorSearchService, DefaultTextChunker, and MarkdownTextChunker to align with embedding token counting. Added a new logging configuration for Microsoft.AspNetCore.Watch.BrowserRefresh in appsettings.Development.json to manage log verbosity during development.
2025-10-16 12:32:48 +02:00
Marco Minerva 6361bfd8d1 Add ReconnectModal component and optimize resource loading
Introduced a new `<ReconnectModal />` component in `App.razor` to handle reconnection scenarios with a user interface. Added a `<ResourcePreloader />` to optimize resource loading. Updated `_Imports.razor` with a new namespace for layout components. Enhanced error handling in `Program.cs` by modifying `UseExceptionHandler` to create a new scope for errors. Added `ReconnectModal.razor`, `ReconnectModal.razor.css`, and `ReconnectModal.razor.js` to implement the modal's HTML, CSS, and JavaScript logic.
2025-10-16 12:15:00 +02:00
Marco Minerva f0873bc9bf Update package dependencies to latest versions
Updated several package dependencies in the SqlDatabaseVectorSearch.csproj file to their latest versions:
- Microsoft.AspNetCore.OpenApi, Microsoft.EntityFrameworkCore.SqlServer, and Microsoft.EntityFrameworkCore.Tools to 10.0.0-rc.2.25502.107
- Microsoft.Extensions.Caching.Hybrid and Microsoft.Extensions.Http.Resilience to 9.10.0
- Microsoft.SemanticKernel to 1.66.0
- TinyHelpers.AspNetCore to 4.1.8

These updates may include bug fixes, performance improvements, or new features.
2025-10-15 11:33:14 +02:00
Marco Minerva ab66bce35a Update package versions for stability and features
Updated the following package versions to ensure compatibility, stability, and access to the latest features:
- Microsoft.AspNetCore.OpenApi: 9.0.9 to 9.0.10
- Microsoft.EntityFrameworkCore.SqlServer: 9.0.9 to 9.0.10
- Microsoft.EntityFrameworkCore.Tools: 9.0.9 to 9.0.10
- Microsoft.Extensions.Caching.Hybrid: 9.9.0 to 9.10.0
- Microsoft.Extensions.Http.Resilience: 9.9.0 to 9.10.0
- TinyHelpers.AspNetCore: 4.1.6 to 4.1.8
2025-10-15 11:31:38 +02:00
Marco Minerva d53df6b526 Merge branch 'master' of https://github.com/marcominerva/SqlDatabaseVectorSearch 2025-10-09 10:05:14 +02:00
Marco Minerva eb7813c277 Update package versions in csproj
Updated the `Microsoft.SemanticKernel` package from version 1.65.0 to 1.66.0, indicating potential new features or improvements. Also updated the `Swashbuckle.AspNetCore.SwaggerUI` package from version 9.0.5 to 9.0.6, likely addressing bug fixes or minor enhancements.
2025-10-09 10:05:08 +02:00
Marco Minerva edae0c35b9 Update ModelId guidance for gpt-4.1 and gpt-5
Updated README.md to specify that for gpt-4.1 and gpt-5 models,
the ModelId should be set to gpt-4o for proper token counting.
Aligned appsettings.json comments with this guidance to ensure
consistency and clarity for users configuring Azure OpenAI settings.
2025-10-07 10:13:36 +02:00
Marco Minerva 8788a013e6 Update README with ModelId and VECTOR size guidelines
Enhanced documentation to clarify the configuration of `ModelId` values for `ChatCompletion` and `Embedding` in `appsettings.json`, ensuring compatibility with `Microsoft.ML.Tokenizers`. Added guidance on setting `Dimensions` for embedding models and provided instructions for updating `ApplicationDbContext` and migrations when modifying VECTOR size.
2025-10-06 11:54:06 +02:00
Marco Minerva 85ffe575ca Merge branch 'master' into net10 2025-10-06 11:36:05 +02:00
Marco Minerva 9a31ad1400 Update SwaggerUI package to version 9.0.6
Upgraded the `Swashbuckle.AspNetCore.SwaggerUI` package in the `SqlDatabaseVectorSearch.csproj` file from version 9.0.5 to 9.0.6. This minor version update likely includes bug fixes, enhancements, or other improvements while maintaining backward compatibility.
2025-10-06 11:35:15 +02:00
Marco Minerva d730b1f760 Update Swashbuckle.AspNetCore.SwaggerUI to v9.0.5
Upgraded the `Swashbuckle.AspNetCore.SwaggerUI` package in the
`SqlDatabaseVectorSearch.csproj` file from version 9.0.4 to 9.0.5.
This update may include bug fixes, new features, or other
improvements provided in the latest version.
2025-09-30 10:49:49 +02:00
Marco Minerva 13c08c1752 Update .editorconfig with new style and diagnostic rules
Added `csharp_style_prefer_simple_property_accessors` to promote
simple property accessors with suggestion severity. Disabled
CA1873 warnings for potentially expensive logging. Retained
IDE0305 rule for simplifying collection initialization for
clarity. These changes enhance code style consistency and
suppress irrelevant diagnostics.
2025-09-19 10:26:37 +02:00
Marco Minerva a80e132f8f Update Microsoft.SemanticKernel to version 1.65.0
Upgraded the `Microsoft.SemanticKernel` package reference in
`SqlDatabaseVectorSearch.csproj` from version 1.64.0 to 1.65.0.
This update may include new features, bug fixes, or performance
improvements.
2025-09-11 09:25:32 +02:00
Marco Minerva d4753ca665 Update Microsoft.SemanticKernel to version 1.65.0
Upgraded the `Microsoft.SemanticKernel` package reference in
`SqlDatabaseVectorSearch.csproj` from version 1.64.0 to 1.65.0.
This update may include bug fixes, new features, or performance
improvements introduced in the newer version.
2025-09-11 09:24:52 +02:00
Marco Minerva 404cd7565a Switch to SqlVector<float> for embeddings
Updated the application to use SQL Server's native vector data type (`SqlVector<float>`) for embeddings, replacing the previous `float[]` or `string` representations.

- Updated `.editorconfig` with new code style preferences and diagnostic rule severities.
- Modified `DocumentChunk.cs` to use `SqlVector<float>` for the `Embedding` property.
- Updated migrations and `ApplicationDbContextModelSnapshot` to reflect the new `SqlVector<float>` type.
- Replaced `AddAzureSql` with `AddSqlServer` in `Program.cs` and removed `UseVectorSearch`.
- Adjusted `DocumentService` and `VectorSearchService` to handle `SqlVector<float>` and updated vector search logic.
- Removed the `EFCore.SqlServer.VectorSearch` package and upgraded EF Core to `10.0.0-rc.1`.
- Made minor adjustments to OpenAPI configuration and dependency management.
2025-09-10 16:45:16 +02:00
Marco Minerva f5011bc44b Upgrade to .NET 10.0 and update NuGet dependencies
Updated the project to target .NET 10.0, replacing the previous .NET 9.0 target framework. Upgraded several NuGet packages to their latest pre-release versions:
- `Microsoft.AspNetCore.OpenApi` to `10.0.0-rc.1.25451.107`
- `Microsoft.EntityFrameworkCore.SqlServer` to `10.0.0-rc.1.25451.107`
- `Microsoft.EntityFrameworkCore.Tools` to `10.0.0-rc.1.25451.107`

Retained `<PrivateAssets>` and `<IncludeAssets>` settings for `Microsoft.EntityFrameworkCore.Tools`. These changes ensure compatibility with the updated framework and leverage new features and improvements.
2025-09-10 16:05:10 +02:00
Marco Minerva e0220da84e Update NuGet package versions to latest releases
Updated the following NuGet packages to their latest versions:
- `Microsoft.AspNetCore.OpenApi` to `9.0.9`
- `Microsoft.EntityFrameworkCore.SqlServer` to `9.0.9`
- `Microsoft.EntityFrameworkCore.Tools` to `9.0.9`
- `Microsoft.Extensions.Caching.Hybrid` to `9.9.0`
- `Microsoft.Extensions.Http.Resilience` to `9.9.0`
- `TinyHelpers.AspNetCore` to `4.1.6`
2025-09-10 16:02:10 +02:00
Marco Minerva ed90a22888 Update copilot instructions and package version
Updated EFCore.SqlServer.VectorSearch package version from 9.0.0-preview.2 to 9.0.0 for stability.
2025-09-08 12:57:16 +02:00
Marco Minerva 98c18139f4 Update documentation and package versions
- Clarified test practices in `copilot-instructions.md`.
- Revised Azure rules for best practices and tool usage.
- Added instruction in `ChatService.cs` for asking clarifying questions.
- Updated package versions in `SqlDatabaseVectorSearch.csproj` for several dependencies.
2025-09-01 11:58:11 +02:00
Marco Minerva 261853e4a7 Fix typos and update dependencies and configurations
Updated the version of
`TinyHelpers.AspNetCore` in `SqlDatabaseVectorSearch.csproj`
from `4.1.2` to `4.1.3`. Clarified comments in
`appsettings.json` for the `ModelId` field under
`ChatCompletion` to specify requirements for `gpt-4.1` models.
2025-08-01 16:11:06 +02:00
Marco Minerva 0ba84a35ba Update package versions for dependencies
Updated the following packages:
- `MinimalHelpers.Routing.Analyzers` to version `1.2.2`
- `PdfPig` to version `0.1.11`
- `TinyHelpers.AspNetCore` to version `4.1.2`
2025-07-28 12:41:46 +02:00
Marco Minerva b20491f247 Refactor solution structure and update package version
Updated the Visual Studio solution file to remove obsolete project configuration lines, indicating a cleanup.
The `Microsoft.SemanticKernel` package version has been upgraded from `1.60.0` to `1.61.0`.
Introduced a new solution file format (`SqlDatabaseVectorSearch.slnx`) to better organize solution items in a structured XML format.
2025-07-25 12:24:05 +02:00
Marco Minerva f6fa60247a Enhance language handling and formatting in ChatService
Updated system prompts in ChatService to emphasize that responses must match the user's question language. Added formatting requirements for answers, including the need for a period before citations. Clarified citation format and adjusted prompt construction for improved readability.
2025-07-24 15:08:43 +02:00
Marco Minerva caa2ed4542 Improve streaming content layout and spinner positioning
Updated `Ask.razor` to conditionally apply a CSS class for the `streaming-text` div based on message status, enabling a spinner display during streaming. Adjusted spinner position from bottom right to bottom left and ensured proper layout with a minimum height for `streaming-content`. Modified a comment for clarity regarding the `Citations` property.

In `Ask.razor.css`, added padding to `streaming-text` for spinner accommodation and adjusted spinner styling to maintain layout integrity.
2025-07-22 10:49:12 +02:00
Marco Minerva d8343b16e5 Refactor message handling and UI in Ask.razor
- Introduced a new `MessageStatus` enum to replace the `IsCompleted` boolean in the `Message` class, allowing for better tracking of message states (New, Streaming, Completed).
- Updated UI rendering to enhance message display with new div elements and conditional spinner for streaming messages.
- Modified the `AskQuestion` method to set the status of user and assistant messages appropriately.
- Added new CSS styles for improved layout and alignment of message components.
- Overall improvements enhance functionality and user experience in the messaging system.
2025-07-22 10:30:16 +02:00
Marco Minerva 505ccff8b7 Enhance ChatService with new prompts and simplifications
Updated ChatService to include static readonly fields for
reformulation and answering prompts. Replaced the existing
ChatSystemPrompt in CreateQuestionAsync. Simplified
GetTokenUsage using expression-bodied members. Modified
SetChatHistoryAsync to ensure correct chat history updates
in the cache.
2025-07-16 16:33:52 +02:00
Marco Minerva 595d6f974f Enhance ChatService with new prompt settings
Updated ChatService to use AzureOpenAIPromptExecutionSettings for improved chat message generation.
The CreateChatAsync method now returns both ChatHistory and settings, allowing for better configuration.
Refined question reformulation prompts to ensure context relevance and language consistency.
These changes enhance the overall functionality and interaction of the chat service.
2025-07-16 15:45:09 +02:00
Marco Minerva 7ca51232fb Add copilot-instructions.md file to the Solution 2025-07-15 12:23:10 +02:00
Marco Minerva 1065dbbf83 Update package references in SqlDatabaseVectorSearch
Updated several package references in the `SqlDatabaseVectorSearch.csproj` file to newer versions, including:
- `Microsoft.AspNetCore.OpenApi` to `9.0.7`
- `Microsoft.EntityFrameworkCore.SqlServer` to `9.0.7`
- `Microsoft.EntityFrameworkCore.Tools` to `9.0.7`
- `Microsoft.Extensions.Caching.Hybrid` to `9.7.0`
- `Microsoft.Extensions.Http.Resilience` to `9.7.0`
- `Microsoft.SemanticKernel` to `1.60.0`
- `MinimalHelpers.FluentValidation` to `1.1.4`
- `Swashbuckle.AspNetCore.SwaggerUI` to `9.0.3`
2025-07-10 10:15:57 +02:00
Marco Minerva f8a48930f3 Enhance UI and improve code documentation
Updated button styles in Documents.razor for better appearance and usability. Improved the document table layout for responsiveness, integrating checkboxes into rows. Corrected comments in VectorSearchService.cs for clarity. Added new styles for content-type badges in app.css.
2025-07-03 11:54:16 +02:00
Marco Minerva b849c78594 Sort citations by filename and page number
Updated the return statement in the `VectorSearchService` class to return citations sorted by `FileName` and `PageNumber`, improving the organization of the output.
2025-07-03 09:31:06 +02:00
Marco Minerva 0a2aaa9e6e Update package versions in SqlDatabaseVectorSearch.csproj
- Updated `Blazor.Bootstrap` from `3.3.1` to `3.4.0`.
- Updated `Microsoft.SemanticKernel` from `1.58.0` to `1.59.0`.
2025-07-02 10:11:29 +02:00
Marco Minerva ddc8ab8791 Enhance documentation and update DocumentChunk model
Updated `copilot-instructions.md` to include new guidelines for Markdown documentation and testing practices. Modified the `DocumentChunk` record in `DocumentChunk.cs` to add `PageNumber` and `IndexOnPage` properties. Adjusted `GetChunksAsync` and `GetChunkEmbeddingAsync` methods in `DocumentService.cs` to accommodate the new properties.
2025-07-01 16:21:25 +02:00
Marco Minerva b971b6bf05 Update copilot-instructions with new guidelines
Enhance `copilot-instructions.md` to include updated best practices for code suggestions, style, architecture, documentation, and testing. Key changes emphasize high confidence suggestions, the latest C# features, specific formatting preferences, and comprehensive documentation requirements. Additionally, new testing practices are introduced to streamline unit test additions and ensure code quality.
2025-06-27 11:41:29 +02:00
Marco Minerva b473511fb6 Update Microsoft.SemanticKernel to version 1.58.0
Updated the `Microsoft.SemanticKernel` package from version 1.57.0 to 1.58.0 in the `SqlDatabaseVectorSearch.csproj` file.
2025-06-26 09:45:26 +02:00
Marco Minerva 523fac2743 Enhance response formatting in ChatService
Updated response guidelines in ChatService.cs to clarify
requirements for answer formatting, including citation rules
and examples of correct and incorrect responses.
Binary differences noted in SqlDatabaseVectorSearch_WebApp.png,
indicating modifications to the image file.
2025-06-24 15:37:07 +02:00
Marco Minerva 13c0beeee6 Merge pull request #12 from marcominerva/decoder_updates
Add citation supports
2025-06-24 14:54:53 +02:00
Marco Minerva 06c1741f14 Enforce response formatting in ChatService
Updated the ChatService class to require that all responses end with a period and a space. Added a condition to include citations in a specified XML format when the answer is known, and to omit citations when the answer is unknown.
2025-06-24 14:53:14 +02:00
Marco Minerva 9eaed9176c Enhance application description and user guidance
Updated `Home.razor` to provide a more detailed overview of the application's capabilities, including document loading, embedding generation, and semantic search. Improved clarity in the supported features section and added a new feature for citations. Included new paragraphs to encourage user interaction and referenced the README for API usage details.
2025-06-24 14:33:43 +02:00
Marco Minerva 476a7734ef Update README.md to add Minimal API badge
Added a badge for "Minimal API" availability to the README.md file, enhancing the visibility of project features. Existing badges for .NET 9 and Blazor remain unchanged.
2025-06-24 12:57:37 +02:00