Release Notes

Release Date: January 2nd, 2024

What’s New

In this version, we've introduced minor enhancements to AI-related components. Notably, we now handle empty source fields by ignoring them. Additionally, when the input surpasses the token limit set by OpenAI, we have implemented a mechanism to handle and ignore the exceeding tokens. Furthermore, we've made improvements to the chunk processor by making the overlap size configurable, providing users with greater flexibility in their processing preferences.

Feel free to contact our development team if you have any questions, suggestions, or feedback. We value your input and are committed to continuously improving our services.

New Features

Bugs Fixed

• None

Improvements

• Staging API: Add parentId index when creating a new bucket.

• OpenAI Processor: Ignore empty source fields

• OpenAI Processor: Split requests based on token count.

Breaking changes

• If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

Others

• Discovery 1.12.0 was tested in AKS v1.28 with Azure CNI.

• Discovery 1.11.0 was tested in GKE v1.28.4 with GCP.

Supported Versions

Discovery provides support and bug fixing for the following versions:

• Version 1.12.0

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

  • Discovery 1.12.0 was tested in AKS v1.28 with Azure CNI.

• Version 1.11.0

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

  • Trim UUIDs on API

  • Discovery 1.11.0 was tested in GKE v1.28.4 with GCP.

• Version 1.10.0 (If you are in this version, plan to upgrade soon)

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

  • Json exception when script processor compilation fails

  • BatchId is stored as null

  • OCR component records don't continue in the pipeline

  • Internal server error importing a zip file

  • [Search UI] Queries with special regex characters causes page to fail

  • Scheduler entity is ignoring some properties from the configuration

  • Website Connector requires a mountpoint to work in k8s

  • Hugging Face service creates outdated file structure

Deprecated Versions

Releases that are no longer recommended for use and their deprecation dates are listed below.

• Version 1.9.0 (January 2024)

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

  • Improvements in JsonUtils.substitute

  • Token scanner writes checksums to the scan index.

  • Staging Connector - Aggregation by group doesn't support Arrays

  • Discovery 1.9.0 was tested in AKS v1.25.11, v1.26.6, and v1.27.1

• Version 1.8.0 (November 2023)

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

• Version 1.7.0 (October 2023)

  • Close record collector in AbstractRequestActionExecutor for errored jobs

  • Incorrect log message from Staging Hydrator

  • Error importing zip

  • [Discovery API] Feedback Atlas default template

  • [Ingestion] Possible memory leakage in Elasticsearch connector

  • ES Connector: Checksum is not hashed before adding a record.

  • Search UI showing more search result pages than required

  • Job in the wrong state should be retried