Release Notes

Release Date: August 30th, 2023

What’s New

In this minor version, we've focused on enhancing the stability and functionality of Discovery by addressing various bugs reported. Additionally, we've taken the opportunity to implement a range of minor improvements that enhance the overall usability of our components.

Feel free to contact our development team if you have any questions, suggestions, or feedback. We value your input and are committed to continuously improving our services. Continue to read on for all new features, bug fixes, and improvements.

New Features

Bugs Fixed

• Json exception when script processor compilation fails

• BatchId is stored as null

• OCR component records don't continue in the pipeline

• Internal server error importing a zip file

• [Search UI] Queries with special regex characters causes page to fail

• Scheduler entity is ignoring some properties from the configuration

• Website Connector requires a mountpoint to work in k8s

• Hugging Face service creates outdated file structure

Improvements

• Set Jobs as FAILED after exceeding message requeues

• Update examples for BERT and hugging face clients in documentation

• Filter entities by name

Breaking changes

• If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

Others

• Discovery 1.9.0 was tested in AKS v1.25.11, v1.26.6, and v1.27.1

• SnakeYAML upgraded to version 2.0

• Graalvm upgraded to version 23.0.1

Supported Versions

Discovery provides support and bug fixing for the following versions:

• Version 1.9.0

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

  • Improvements in JsonUtils.substitute

  • Token scanner writes checksums to the scan index.

  • Staging Connector - Aggregation by group doesn't support Arrays

• Version 1.8.0

  • If upgrading from a version prior to 1.7.0, export the existing configuration, delete the configuration indices (ingestion: seed, processor, and pipeline / discovery: endpoint and processor), and re-import it after upgrade has been performed.

• Version 1.7.0 (If you are in this version, plan to upgrade soon)

  • Close record collector in AbstractRequestActionExecutor for errored jobs

  • Incorrect log message from Staging Hydrator

  • Error importing zip

  • [Discovery API] Feedback Atlas default template

  • [Ingestion] Possible memory leakage in Elasticsearch connector

  • ES Connector: Checksum is not hashed before adding a record.

  • Search UI showing more search result pages than required

  • Job in the wrong state should be retried

Deprecated Versions

Releases that are no longer recommended for use and their deprecation dates are listed below.

• Version 1.6.0 (August 2023)

  • Pipeline configuration should not allow null action

  • Ingestion Admin should allow deep cloning multiple times

  • Single seed schedules should not be enqueued if the same seed is already running

  • Discovery API - Endpoints should handle an empty body request

  • Discovery API - Mongo Component should store the response body as JsonNode

  • Discovery API - Snap Component not casting error message when facets field points to an array

  • Breaking changes:

    • When configuring the S3 connector, the region must be in the format AWS requires it.

      • e.g. “us-east-1” instead of “"US_EAST_1"

• Version 1.5.0 (July 2023)

  • Internal server should not error when adding item with no "body" to Staging repo

  • JsonUtils can not substitute value properties of intNode objects when are into an array

• Version 1.4.0 (June 2023)

  • Environment variables are not substituted in processors if there are no seed properties

  • Credentials Service cannot deserialize value '_id' from cache

  • Distilbert by HuggingFace Service causes node to reset