Summary

• S3 Connector

  • New object upload action.

• Ingestion Processors

  • All Ingestion content processors now have an option to change the batch size of the output jobs.

• Website Connector

  • Support for JavaScript sites using Norconex WebDriver Fetcher.

• Parquet File Processor

  • New Parquet File Ingestion processor that extracts rows from a parquet file and creates a record for each one.

• Thumbnail Generator

  • New Thumbnail Generator for Ingestion that is able to create thumbnails for websites and PDF files.

• OpenAI Processor

  • New completion action for the OpenAI Ingestion Processor.

• Bugs Fixed: 1

• Minor Improvements: 8

Release Notes

• Website Connector

  • Option to add custom Norconex Importer configurations

  • Support for Forms authentication.

  • Option to set maximum documents per crawl url.

• Html Processor

  • Option to extract multiple selector matches into a single string or an array.

• Staging Hydrator

  • Hydrate record id is now configurable.

• Bugs Fixed: 0

• Minor Improvements: 0

Release Notes

• Elasticsearch Connector:

  • Option to use aggregations as a starting point has been enabled.

• Language Detector:

  • Option to define more than 1 field to be recognized by the language detector.

• Bugs Fixed: 0

• Minor Improvements: 0

Release Notes

• Chunk Processor:

  • Overlapping sentences: Overlap size is configurable.

• Breaking changes:

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after the upgrade has been performed.

• Bugs Fixed: 0

• Minor Improvements: 0

Release Notes

• ChatGPT Completions Processor (alpha version):

  • Parameter added so model is configurable.

• Hugging Face Service now supports:

  • Summarization: Given a text the main ideas will be extracted from within that text.

  • Sentiment Analysis: Given a text, it will be classified into positive, negative, or neutral according to its perceived emotion.

  • Zero-shot Classification: Given a text and a set of labels, the text will be classified into the best fitting label.

• Chunk Processor:

  • Overlapping sentences: The chunks are created using a fixed number of contiguous sentences, incrementing the position of the first sentence for each chunk, until the full text is exhausted.

• Breaking changes:

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after the upgrade has been performed.

• Bugs Fixed: 0

• Minor Improvements: 0

Release Notes

• Discovery API supports OpenAI integration (alpha version):

  • OpenAI Query Embeddings Processor

  • ChatGPT Completions Processor

• Ingestion API supports OpenAI integration (alpha version):

  • OpenAI Embeddings Content Processor

• RDB Connector supports connections to MSSQL. It can also execute and retrieve data from a stored procedure within MSSQL.

• Staging API supports:

  • Regex filters

  • ‘Lower than or equal' and 'Greater than or equal’ operations.

• Breaking changes:

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after the upgrade has been performed.

• Bugs Fixed: 1

• Minor Improvements: 4

Release Notes

• Search UI endpoint is configurable

• Website connector can filter by content type

• Breaking changes:

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after the upgrade has been performed.

• Bugs Fixed: 8

• Minor Improvements: 3

Release Notes

• Binary Data Service supports Azure Blob Storage.

• ‘Show more’ option for featured snippets

• Breaking changes:

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after the upgrade has been performed.

• Bugs Fixed: 3

• Minor Improvements: 9

• Discovery 1.9.0 was tested in AKS v1.25.11, v1.26.6, and v1.27.1

Release Notes

• Binary Data Service supports GCS

• Semantic Title Search

• Breaking changes:

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after the upgrade has been performed.

• Bugs Fixed: 8

• Minor Improvements: 6

Release Notes

• Binary Data Service supports S3

• Breaking changes:

  • If upgrading from an older version, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after the upgrade has been performed.

• Bugs Fixed: 8

• Minor Improvements: 5

Release Notes

• HuggingFace service

• Discovery API components integrated with credentials service

• Upgraded MongoDB dependency to version 6.0.3

• Breaking changes:

  • When configuring the S3 connector, the region must be in the format AWS requires it. (e.g. “us-east-1” instead of “"US_EAST_1")

• Bugs Fixed: 6

• Minor Improvements: 7

Release Notes

• Engine scoring for Search Relevancy Scoring Dashboard

• Discovery API supports Atlas vector search

• Discovery API: Elastic component upgraded to work with ES 8.x

• Bugs Fixed: 2

• Minor Improvements: 13

Release Notes

• Mongo Atlas connector

• Staging API supports batch requests

• Discovery API supports Mongo Atlas

• User feedback can be applied to search results

• Bugs Fixed: 3

• Minor Improvements: 7

Release Notes

• Mongo Atlas hydrator

• Staging repository can be used in Discovery API

• Staging repository supports DocumentDB

• Ingestion API: Seeds can be paused and resumed

• Bugs Fixed: 6

• Minor Improvements: 18

Release Notes

• Secrets service

• Discovery API: Snap Component can be configured to use “OR” or “AND”

• Basic authentication for several components

• Staging Repository: Bucket status can be retrieved and paginated

• Bugs Fixed: 2

• Minor Improvements: 9

Release Notes

• Elasticsearch connector

• Find entity processor

• Opensearch hydrator

• Breaking changes:

  • “recordData” parameter has been changed to “recordDataStrategy”.

  • To use “recordDataStrategy”, it must be set outside of the config block.

• Bugs Fixed: 4

• Minor Improvements: 9

Release Notes