How To: Using the Elasticsearch Vector Store Connector with Semantic Kernel : jamie_maguire

December 14, 2024 December 14, 2024

How To: Using the Elasticsearch Vector Store Connector with Semantic Kernel
by: jamie_maguire
blow post content copied from Jamie Maguire
click here to view original post

I had been working with Elasticsearch to implement a site wide search feature and an agentic AI system that will help developers find the right information at the right time for the task at hand.

The content being searched and reasoned over is technical in nature.

To support the agent AI feature, a vector database is required. I did find an open-source Elastic connector project but could not get it to fully work without a lot of integration code.

Fortunately, just a few weeks ago, Elastic released a Vector Store connector that is compatible with Semantic Kernel.

In this blog post, I outline the steps you need to implement to use this new connector.

The following is covered:

Creating index mapping
Creating an object with relevant attributes to represent vector content
Generating content embeddings
Chunking content
Indexing content
Representing the search query as an array of vectors
Running the search using the array of vectors

I did have an issue with defining mappings but found a work around. The connector is currently in preview mode so either there is an issue with it, or my original mapping implementation requires some work.

Creating the Index Mapping

For reference, Elasticsearch version 8.10 is used and running locally on port 9200. The following code we create a connection do this instance using the http://client.

I could not get the vector mappings configured correctly in code so hard to define them using the JSON object you can see in the code below.

private static async Task CreateIndexAsync()
  {
      // creating index example using httpclient
      using var hclient = new HttpClient();


      // Define the Elasticsearch endpoint
      var uri = "http://localhost:9200/contentvectorcollection";



      // Define the raw JSON mapping
      var mappingJson = @"
                          {
                            ""mappings"": {
                             ""properties"": {
                                ""KEY"": {
                                  ""type"": ""keyword""
                                },
                               ""DOCUMENT_URI"": {
                                  ""type"": ""text""
                                },
                                ""PARAGRAPH_ID"": {
                                  ""type"": ""text""
                                },
                                ""TEXT"": {
                                  ""type"": ""text""
                                },
                                ""TEXT_EMBEDDING"": {
                                  ""type"": ""dense_vector"",
                                  ""dims"": 1536,
                                  ""index"": true,
                                  ""similarity"": ""cosine""
                                }
                              }
                            }
                          }";


      // Create the index with raw JSON

      var content = new StringContent(mappingJson, Encoding.UTF8, "application/json");
      var credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes($"admin:password1"));

      hclient.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", credentials);

      var createIndexResponse = await hclient.PutAsync(uri, content);


      if (createIndexResponse.StatusCode == System.Net.HttpStatusCode.OK)
      {
          Console.WriteLine("Index created successfully.");
      }
      else
      {
          Console.WriteLine($"Failed to create index: {createIndexResponse.StatusCode}");
          var reply = createIndexResponse.Content.ReadAsStringAsync().Result;
      }
  }

In the code above we create a StringContent object and define the credentials which are converted to a base 64 string. The method PutAsync in our code creates the index.

Note the definition for the TEXT_EMBEDDING mapping:

"TEXT_EMBEDDING": 
"type": "dense_vector",
"dims"": 1536,
"index"": true,
"similarity": "cosine"

                                }

A description of each attribute follows:

type: “dense_vector”

Specifies that this field will store dense vector data, which is commonly used for representing high-dimensional numerical data, such as embeddings generated by machine learning models (e.g., text embeddings from models like OpenAI’s CLIP or sentence transformers).

dims: 1536

Indicates the dimensionality of the vector. In this case, each vector stored in the TEXT_EMBEDDING field will have 1,536 dimensions. This matches the output size of the embedding model used to generate these vectors.

index: true

Enables indexing of the vectors for similarity search. When set to true, Elasticsearch can perform operations like nearest neighbour (KNN) search on this field.

similarity: “cosine”`

Specifies the similarity metric to use for comparing vectors during queries. Cosine similarity measures the angular similarity between vectors, making it suitable for comparing embeddings regardless of their magnitude.

Learn more about cosine similarity here.

Indexing Content

With the index created data can then be inserted. For reference I need to store a URL and raw HTML content. The following code does this:

private static async Task IndexDataAsync(ElasticsearchClient client, DataUploaderService dataUploader, string url, string htmlContent)
 {
     var collection = new ElasticsearchVectorStoreRecordCollection<Content>(client, "contentvectorcollection");

     await collection.CreateCollectionIfNotExistsAsync();

     // Load the data.
     var paragraphs = DataReaderService.ReadTextFromHtml(htmlContent, url);

     await dataUploader.GenerateEmbeddingsAndUpload(client, paragraphs);
}

In the above code, the collection contentvectorcollection will store any ingested content.

Before ingesting content, we need away for the entire HTML to be split into chunks.

Splitting documents into manageable focused pieces (chunks) is a key step when implementing for Retrieval-Augmented Generation (RAG) solutions.

It enhances retrieval efficiency, improves context relevance for generation, and enables scalable processing.

The method ReadTextFromHTML creates the chunks. It returns a list of TextParagraph objects You can see this here:

public static IEnumerable<TextParagraph> ReadTextFromHtml(string htmlContents, string url)
 {

     // Load the HTML document
     HtmlDocument htmlDoc = new HtmlDocument();


     try
     {
         htmlDoc.LoadHtml(htmlContents);
     }
     catch (Exception ex)
     {
         Console.WriteLine($"Error loading HTML: {ex.Message}");
         yield break;
     }


     // Start extracting text from the root node
     int paragraphCount = 0;
     foreach (var text in ExtractTextFromNode(htmlDoc.DocumentNode))
     {
         if (!string.IsNullOrWhiteSpace(text))
         {
             Console.WriteLine("Found text content:");
             Console.WriteLine(text);
             Console.WriteLine();

             yield return new TextParagraph
             {
                 Key = Guid.NewGuid().ToString(),
                 DocumentUri = url,
                 ParagraphId = (++paragraphCount).ToString(),
                 Text = text.Trim()
             };
         }
     }
 }

For clarity, the recursive method ExtractTextFromNode code is shown:

private static IEnumerable<string> ExtractTextFromNode(HtmlNode node)
{

    // Base case: If the node is a text node, return its text content
    if (node.NodeType == HtmlNodeType.Text)
    {
        var text = node.InnerText;
        if (!string.IsNullOrWhiteSpace(text))
        {
            yield return System.Text.RegularExpressions.Regex.Replace(text, @"\s+", " "); // Normalize whitespace
        }
    }


    // Recursive case: Traverse child nodes
    foreach (var child in node.ChildNodes)
    {
        foreach (var text in ExtractTextFromNode(child))
        {
            yield return text;
        }
    }
}

The returned list is of type TextParagraph objects.

A definition of this is show below. Pay attention to the decorator attributes, these dictate the behaviour of each property in relation to the mappings defined for the vector store in the earlier section:

public class TextParagraph
 {
     /// <summary>A unique key for the text paragraph.</summary>
     [VectorStoreRecordKey]
     [Key]
     public required string Key { get; init; }


     /// <summary>A uri that points at the original location of the document containing the text.</summary>
     [VectorStoreRecordData]
     public required string DocumentUri { get; init; }

     /// <summary>The id of the paragraph from the document containing the text.</summary>
     [VectorStoreRecordData]
     public required string ParagraphId { get; init; }

     /// <summary>The text of the paragraph.</summary>
     [VectorStoreRecordData]
     public required string Text { get; init; }


     /// <summary>The embedding generated from the Text.</summary>
     [VectorStoreRecordVector(1536)]
     public ReadOnlyMemory<float> TextEmbedding { get; set; }
 }

At this point, we have:

defined the mappings for the index
created the index
a model to represent chunked content (TextParagraph)
a method to generate chunked content from a raw HTML string

The next step is to generate upload embeddings.

Generate Embeddings and Upload

After generating a list of TextParagraph objects from an input HTML string, we can index them using the method GenerateEmbeddingsAndUpload.

The code for this method is show below:

public async Task GenerateEmbeddingsAndUpload(ElasticsearchClient elasticClient, IEnumerable<TextParagraph> textParagraphs)
{
foreach (var paragraph in textParagraphs)
 {
  // Generate the text embedding.
  Console.WriteLine($"Generating embedding for paragraph: {paragraph.ParagraphId}");

  paragraph.TextEmbedding = await textEmbeddingGenerationService.GenerateEmbeddingAsync(paragraph.Text);

  var serializedParagraph = JsonConvert.SerializeObject(paragraph);

  Console.WriteLine(serializedParagraph);

  var response = await elasticClient.IndexAsync(paragraph, idx => idx.Index("contentvectorcollection").Id(paragraph.Key));

  Console.WriteLine();
  Console.WriteLine($"Indexed paragraph: {paragraph.ParagraphId}");
 }
}

In the above code, we loop through the list of TextParagraph objects.

For each object in the list, we generate the embeddings for the paragraph text.

The method generate GenerateEmbeddingAsync performs this:

paragraph.TextEmbedding = await textEmbeddingGenerationService.GenerateEmbeddingAsync(paragraph.Text);

For example, the following text embedding represents the text from content found in the following url: https://www.scichart.com/documentation/js/current/Axis%20Alignment%20-%20Create%20a%20Vertical%20Chart.html

"hits": [
{
 "_index": "contentvectorcollection",
 "_id": "9a41cf98-0ce4-46c2-aa07-05b6b2f92641",
 "_score": 1.0,
 "_source": {
 "KEY": "9a41cf98-0ce4-46c2-aa07-05b6b2f92641",
 "DOCUMENT_URI": "\"https://www.scichart.com/documentation/js/current/Axis%20Alignment%20-%20Create%20a%20Vertical%20Chart.html",
 "PARAGRAPH_ID": "1",
 "TEXT": "\\n \\n \\n Vertical Charts (Rotate, Transpose Axis)\\n It is possible to create&nbsp;Vertical (Rotated) Charts&nbsp;with SciChart. This transposes the entire chart, swapping X-Axis for Y and renders series top to bottom intead of left to right. Tooltips and markers also are transposed to the final effect is like a vertical chart.\\r\\n\\r\\nAbove: The&nbsp;JavaScript Oil and Gas Dashboard showcase from the SciChart.js Demo, showing a use-case of transposing the X,Y axis to achieve&nbsp;a vertical chart, visualising well drill depth.\\r\\n\\r\\nTo&nbsp;achieve this, simply set&nbsp;axis.axisAlignment to&nbsp;Left or&nbsp;Right for X Axis and&nbsp;Top or&nbsp;Bottom for Y Axis. And that's it - SciChart takes care of the rest:\\r\\n\\r\\n\\r\\n \\r\\n Javascript\\r\\n\\r\\n json-builder\\r\\n \\r\\n\\r\\n \\r\\n \\r\\n// Demonstrates how to configure a vertical chart in SciChart.js\\r\\nconst {\\r\\n SciChartSurface,\\r\\n NumericAxis,\\r\\n SciChartJsNavyTheme,\\r\\n EAxisAlignment,\\r\\n HorizontalLineAnnotation,\\r\\n ELabelPlacement,\\r\\n FastLineRenderableSeries,\\r\\n XyDataSeries\\r\\n} = SciChart;\\r\\n\\r\\n// or, for npm, import { SciChartSurface, ... } from \\\"scichart\\\"\\r\\n\\r\\nconst { wasmContext, sciChartSurface } = await SciChartSurface.create(divElementId, {\\r\\n theme: new SciChartJsNavyTheme()\\r\\n});\\r\\n\\r\\n// Add the xAxis to the chart\\r\\nsciChartSurface.xAxes.add(new NumericAxis(wasmContext, {\\r\\n axisTitle: \\\"X Axis\\\",\\r\\n axisAlignment: EAxisAlignment.Left\\r\\n}));\\r\\n\\r\\n// Creating a NumericAxis as a YAxis on the left\\r\\nsciChartSurface.yAxes.add(new NumericAxis(wasmContext, {\\r\\n axisTitle: \\\"Y Axis\\\",\\r\\n axisAlignment: EAxisAlignment.Top\\r\\n}));\\r\\n\\r\\n// Show how a line series responds to vertical chart\\r\\nconst xValues = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19];\\r\\nconst yValues = xValues.map(x =&gt; Math.sin(x * 0.4));\\r\\nsciChartSurface.renderableSeries.add(new FastLineRenderableSeries(wasmContext, {\\r\\n dataSeries: new XyDataSeries(wasmContext, {\\r\\n xValues,\\r\\n yValues\\r\\n }),\\r\\n stroke: \\\"#0066FF\\\",\\r\\n strokeThickness: 3,\\r\\n}));\\r\\n\\r\\n// Show how a HorizontalLineAnnotation responds to vertical chart\\r\\nsciChartSurface.annotations.add(new HorizontalLineAnnotation({\\r\\n // normally we set y1 but with vertical charts, we set annotation.x1\\r\\n x1: 10,\\r\\n labelValue: \\\"HorizontalLineAnnotation with x1 = 10\\\",\\r\\n showLabel: true,\\r\\n stroke: \\\"#F48420\\\",\\r\\n strokeThickness: 2,\\r\\n labelPlacement: ELabelPlacement.TopLeft\\r\\n}));\\r\\n\\r\\n \\r\\n\\r\\n \\r\\n \\r\\n// Demonstrates how to configure a vertical chart in SciChart.js using the Builder API\\r\\nconst {\\r\\n chartBuilder,\\r\\n EThemeProviderType,\\r\\n EAxisType,\\r\\n EAxisAlignment\\r\\n} = SciChart;\\r\\n\\r\\n// or, for npm, import { chartBuilder, ... } from \\\"scichart\\\"\\r\\n\\r\\nconst xValues = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19];\\r\\nconst yValues = xValues.map(x =&gt; Math.sin(x * 0.4));\\r\\n\\r\\nconst { wasmContext, sciChartSurface } = await chartBuilder.build2DChart(divElementId, {\\r\\n surface: { theme: { type: EThemeProviderType.Dark } },\\r\\n xAxes: {\\r\\n type: EAxisType.NumericAxis,\\r\\n options: {\\r\\n axisTitle: \\\"X Axis\\\",\\r\\n axisAlignment: EAxisAlignment.Left\\r\\n }\\r\\n },\\r\\n yAxes: {\\r\\n type: EAxisType.NumericAxis,\\r\\n options: {\\r\\n axisTitle: \\\"Y Axis\\\",\\r\\n axisAlignment: EAxisAlignment.Top\\r\\n }\\r\\n },\\r\\n series: [\\r\\n {\\r\\n type: ESeriesType.LineSeries,\\r\\n options: {\\r\\n stroke: \\\"#0066FF\\\",\\r\\n strokeThickness: 3,\\r\\n },\\r\\n xyData: {\\r\\n xValues,\\r\\n yValues\\r\\n }}\\r\\n ]\\r\\n});\\r\\n\\r\\n \\r\\n\\r\\n\\r\\nThis results in the following output, where the XAxis is on the left, the YAxis is on the top. The chart series is rotated automatically and now draws top to bottom, rather than left to right.\\r\\n\\r\\n\\r\\n \\r\\n&lt;div id=\\\"scichart-root\\\" &gt;&lt;/div&gt;\\r\\n \\r\\n\\r\\n \\r\\nbody { margin: 0; }\\r\\n#scichart-root { width: 100%; height: 100vh; }\\r\\n \\r\\n\\r\\n \\r\\nasync function verticalCharts(divElementId) {\\r\\n // #region ExampleA\\r\\n // Demonstrates how to configure a vertical chart in SciChart.js\\r\\n const {\\r\\n SciChartSurface,\\r\\n NumericAxis,\\r\\n SciChartJsNavyTheme,\\r\\n EAxisAlignment,\\r\\n HorizontalLineAnnotation,\\r\\n ELabelPlacement,\\r\\n FastLineRenderableSeries,\\r\\n XyDataSeries\\r\\n } = SciChart;\\r\\n\\r\\n // or, for npm, import { SciChartSurface, ... } from \\\"scichart\\\"\\r\\n\\r\\n const { wasmContext, sciChartSurface } = await SciChartSurface.create(divElementId, {\\r\\n theme: new SciChartJsNavyTheme()\\r\\n });\\r\\n\\r\\n // Add the xAxis to the chart\\r\\n sciChartSurface.xAxes.add(new NumericAxis(wasmContext, {\\r\\n axisTitle: \\\"X Axis\\\",\\r\\n axisAlignment: EAxisAlignment.Left\\r\\n }));\\r\\n\\r\\n // Creating a NumericAxis as a YAxis on the left\\r\\n sciChartSurface.yAxes.add(new NumericAxis(wasmContext, {\\r\\n axisTitle: \\\"Y Axis\\\",\\r\\n axisAlignment: EAxisAlignment.Top\\r\\n }));\\r\\n\\r\\n // Show how a line series responds to vertical chart\\r\\n const xValues = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19];\\r\\n const yValues = xValues.map(x =&gt; Math.sin(x * 0.4));\\r\\n sciChartSurface.renderableSeries.add(new FastLineRenderableSeries(wasmContext, {\\r\\n dataSeries: new XyDataSeries(wasmContext, {\\r\\n xValues,\\r\\n yValues\\r\\n }),\\r\\n stroke: \\\"#0066FF\\\",\\r\\n strokeThickness: 3,\\r\\n }));\\r\\n\\r\\n // Show how a HorizontalLineAnnotation responds to vertical chart\\r\\n sciChartSurface.annotations.add(new HorizontalLineAnnotation({\\r\\n // normally we set y1 but with vertical charts, we set annotation.x1\\r\\n x1: 10,\\r\\n labelValue: \\\"HorizontalLineAnnotation with x1 = 10\\\",\\r\\n showLabel: true,\\r\\n stroke: \\\"#F48420\\\",\\r\\n strokeThickness: 2,\\r\\n labelPlacement: ELabelPlacement.TopLeft\\r\\n }));\\r\\n\\r\\n // #endregion\\r\\n};\\r\\n\\r\\nverticalCharts(\\\"scichart-root\\\");\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n\\r\\nasync function builderExample(divElementId) {\\r\\n // #region ExampleB\\r\\n // Demonstrates how to configure a vertical chart in SciChart.js using the Builder API\\r\\n const {\\r\\n chartBuilder,\\r\\n EThemeProviderType,\\r\\n EAxisType,\\r\\n EAxisAlignment\\r\\n } = SciChart;\\r\\n\\r\\n // or, for npm, import { chartBuilder, ... } from \\\"scichart\\\"\\r\\n\\r\\n const xValues = [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19];\\r\\n const yValues = xValues.map(x =&gt; Math.sin(x * 0.4));\\r\\n\\r\\n const { wasmContext, sciChartSurface } = await chartBuilder.build2DChart(divElementId, {\\r\\n surface: { theme: { type: EThemeProviderType.Dark } },\\r\\n xAxes: {\\r\\n type: EAxisType.NumericAxis,\\r\\n options: {\\r\\n axisTitle: \\\"X Axis\\\",\\r\\n axisAlignment: EAxisAlignment.Left\\r\\n }\\r\\n },\\r\\n yAxes: {\\r\\n type: EAxisType.NumericAxis,\\r\\n options: {\\r\\n axisTitle: \\\"Y Axis\\\",\\r\\n axisAlignment: EAxisAlignment.Top\\r\\n }\\r\\n },\\r\\n series: [\\r\\n {\\r\\n type: ESeriesType.LineSeries,\\r\\n options: {\\r\\n stroke: \\\"#0066FF\\\",\\r\\n strokeThickness: 3,\\r\\n },\\r\\n xyData: {\\r\\n xValues,\\r\\n yValues\\r\\n }}\\r\\n ]\\r\\n });\\r\\n // #endregion\\r\\n};\\r\\n\\r\\n\\r\\n\\r\\n// Uncomment this to use the builder example\\r\\n //builderExample(\\\"scichart-root\\\");\\r\\n\\r\\n \\r\\n\\r\\n\\r\\n\\r\\n&nbsp;\\r\\n\\r\\nFlipping the Axis when Horizontal or Vertical.\\r\\n\\r\\nAn Axis may be flipped when horizontal or vertical (coordinates drawn in opposite directions) by setting the&nbsp;AxisCore.flippedCoordinates property.\\r\\n\\r\\nFor example, taking the code sample above, and setting xAxis.flippedCoordinates = true, we get the following result. Notice the XAxis is now drawn in reverse and the series is now drawn from bottom to top..\\r\\n\\r\\n\\r\\n\\r\\n\\r\\n\\r\\nConsiderations when using Vertical Charts\\r\\n\\r\\nThis Flexibility of SciChart allows for some pretty interesting configurations of charts. However, here are some considerations when using a Vertical Chart.\\r\\n\\r\\n\\r\\n Tooltips,&nbsp;Cursors and the&nbsp;RolloverModifier will also be transposed (rotated 90 degrees). When applying a RolloverModifier the cursor line is usually vertical, but in a vertical chart the cursor line will be horizontal.\\r\\n\\r\\n Annotations will behave differently. For example a&nbsp;HorizontalLineAnnotation will still draw horizontally but instead of setting the y1 property to place on the YAxis, now you must set x1 property to place on the XAxis.\\r\\n\\r\\n\\r\\n&nbsp;\\r\\n\\r\\n&nbsp;\\n \\n See Also\\nVertical (Rotated) Chart Example\\r\\n\\r\\n\\n \\n \\n \\n",

"TEXT_EMBEDDING": [
-0.00260936772,
-0.020350378,
0.00168717816,
-0.00864184927,
-0.00564914662,
0.0070210821,
……………….
-0.0248158928,
-0.010175189,
-0.0077070496,
-0.0416422784,
0.0340024829,
0.00753219519,
0.0012071688,
0.00479168678,
-0.00308181113
]
}
}

After we have generated embedding for any text it can be indexed, the method index async performs this:

var response = await elasticClient.IndexAsync(paragraph, idx => idx.Index("contentvectorcollection").Id(paragraph.Key));

We can verify content has been indexed by querying the Elasticsearch instance using postman:

Generate Embeddings for Search Query

Before executing a search, we need to implement a method that generates vectors from the search query text. The method below does this:

private static async Task<ReadOnlyMemory<float>> GenerateVectorsFromSearchQueryText(Kernel kernel, string queryText)
{
 Console.WriteLine("Generating embedding for query text...");
 var embeddingGenerator = kernel.Services.GetRequiredService<ITextEmbeddingGenerationService>();
 var queryVector = await embeddingGenerator.GenerateEmbeddingAsync(queryText);
 return queryVector;
}

The vectorized representation of the query-text can then be used to perform a search against the vector database.

Run Search

The code below accepts a query vector which is used to perform a search against the vector database.:

private static async Task<Elastic.Clients.Elasticsearch.SearchResponse<TextParagraph>> QueryDataAsyncUsingVectorQuery(Kernel kernel, ReadOnlyMemory<float> queryVector, ElasticsearchClient client)
{
 // Perform vector search
 var response = await client.SearchAsync<TextParagraph>(s => s
 .Index("contentvectorcollection")
 .Knn(k => k
 .Field(f => f.TextEmbedding) // Specify the dense_vector field
 .QueryVector(queryVector.ToArray())    // Use the query vector
 .k(5)                        // Number of nearest neighbours to return
 .NumCandidates(10)           // Number of candidates to consider
 )
);


// Check the response
if (response.ApiCallDetails.HasSuccessfulStatusCode)
{
 Console.WriteLine("Found search results:");
 return response;
}
else
{
 Console.WriteLine($"Search failed: {response.DebugInformation}");
 return null;
}
}

The property KNN Is responsible for setting the vector field, supplying the query vector.

Specifying the amount of neighbours to return in this instance (5) and the number of candidates to consider.

We can execute a natural language query that references the content we generated vectors for:

var vectorisedSearchTerm = await GenerateVectorsFromSearchQueryText(kernel, "create vertical charts");

var results = await QueryDataAsyncUsingVectorQuery(kernel, vectorisedSearchTerm, client);

foreach (var hit in results.Hits)
{
    Console.WriteLine($"Id: {hit.Id}, Score: {hit.Score}");
}

We get several hits:

Search results are sorted by the relevance probability score:

Several results are found:

Expanding the Source property for shows the associated indexed/vector/document :

We can examine the text property for the embedding in the Visual Studio debugger:

Vertical Charts (Rotate, Transpose Axis)\n It is possible to create Vertical (Rotated) Charts with SciChart.

..and browse to the associated URL that we defined for vector storage:

"https://www.scichart.com/documentation/js/current/Axis%20Alignment%20-%20Create%20a%20Vertical%20Chart.html

Perfect.

You can learn more about the KNN (or known nearest neighbour) algorithm here.

Summary

In this blog post, we have seen how to use the Elasticsearch Vector Store Connector with Semantic Kernel.

We’ve also seen how to: creating index mappings, generating content embeddings, chunk and index content.

We saw how to represent the search query as an array of vectors and how to use this as search criteria to return indexed and vectorised data.

The concepts in this blog pave the way for you to implement a RAG AI solution using Elasticsearch vector database with Semantic Kernel.

Dotnet Reader