Release Notes

Release Date: November 23rd, 2023

What’s New

In this release, we are introducing our latest updates to enhance your experience with Chat GPT, Hugging Face Service, and Chunk Processor. With the Chat GPT Completion Processor, a new parameter has been added, allowing users to configure the model according to their specific needs. This customization option provides greater flexibility in tailoring the model's behavior to better suit individual preferences and use cases. Additionally, the Hugging Face Service has been extended to support three powerful features: Summarization, Sentiment Analysis, and Zero-shot Classification. Finally, the chunk processor now supports the overlapping sentences strategy.

New Features

Bugs Fixed

• None

Improvements

• None

Breaking changes

• If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

Others

• Discovery 1.11.0 was tested in GKE v1.27

Supported Versions

Discovery provides support and bug fixing for the following versions:

• Version 1.11.0

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

  • Trim UUIDs on API

• Version 1.10.0

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

  • Json exception when script processor compilation fails

  • BatchId is stored as null

  • OCR component records don't continue in the pipeline

  • Internal server error importing a zip file

  • [Search UI] Queries with special regex characters causes page to fail

  • Scheduler entity is ignoring some properties from the configuration

  • Website Connector requires a mountpoint to work in k8s

  • Hugging Face service creates outdated file structure

• Version 1.9.0 (If you are in this version, plan to upgrade soon)

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

  • Improvements in JsonUtils.substitute

  • Token scanner writes checksums to the scan index.

  • Staging Connector - Aggregation by group doesn't support Arrays

Deprecated Versions

Releases that are no longer recommended for use and their deprecation dates are listed below.

• Version 1.8.0 (November 2023)

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

• Version 1.7.0 (October 2023)

  • Close record collector in AbstractRequestActionExecutor for errored jobs

  • Incorrect log message from Staging Hydrator

  • Error importing zip

  • [Discovery API] Feedback Atlas default template

  • [Ingestion] Possible memory leakage in Elasticsearch connector

  • ES Connector: Checksum is not hashed before adding a record.

  • Search UI showing more search result pages than required

  • Job in the wrong state should be retried

• Version 1.6.0 (August 2023)

  • Pipeline configuration should not allow null action

  • Ingestion Admin should allow deep cloning multiple times

  • Single seed schedules should not be enqueued if the same seed is already running

  • Discovery API - Endpoints should handle an empty body request

  • Discovery API - Mongo Component should store the response body as JsonNode

  • Discovery API - Snap Component not casting error message when facets field points to an array

  • Breaking changes:

    • When configuring the S3 connector, the region must be in the format AWS requires it.

      • e.g. “us-east-1” instead of “"US_EAST_1"