Mirrors/SqlDatabaseVectorSearch
1
0
Fork 0
mirror of https://github.com/marcominerva/SqlDatabaseVectorSearch.git synced 2026-06-20 12:23:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

Filter out empty paragraphs in PdfContentDecoder

Browse Source 
Updated the paragraph processing to exclude empty or whitespace-only entries before creating Chunk objects, ensuring only meaningful text is included.
This commit is contained in:
Marco Minerva
2025-05-27 17:19:25 +02:00
parent fa81f01c27
commit 1e531e5ad6
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SqlDatabaseVectorSearch/ContentDecoders/PdfContentDecoder.cs
+1 -1
View File
@@ -28,6 +28,6 @@ public class PdfContentDecoder(IServiceProvider serviceProvider) : IContentDecod
var paragraphs = textChunker.Split(pageText); var paragraphs = textChunker.Split(pageText);
return paragraphs.Select((text, index) => new Chunk(pdfPage.Number, index, text)); return paragraphs.Where(p => !string.IsNullOrWhiteSpace(p)).Select((text, index) => new Chunk(pdfPage.Number, index, text));
} }
} }