/
RDBMS Connector

RDBMS Connector

RDBMS Connector

Scans data from a RDBMS Server instance.

Pre-Requisites

RDBMS Server running

Configuration

Example credentials for this seed:

{
  "config": {
    "password": "Admin1234",
    "username": "root"
  },
  "name": "MySQL Server RDBMS credentials",
  "type": "rdbms-connector"
}

Example configuration in a seed:

In all actions the batch size is used as the fetch size.

{
  "type": "rdbms-connector",
  "name": "MySQL Server RDBMS Seed",
  "config": {
    "seed": {
      "pull": {
        "objectSQL": "SELECT PersonID, LastName, FirstName, Address, City FROM Persons WHERE PersonID IN(?);",
        "id": "PersonID"
      },
      "scan": {
        "fields": {
          "id": "PersonID",
          "hash": [
            "PersonID"
          ]
        },
        "fullScanSQL": "SELECT PersonID, LastName, FirstName, Address, City FROM Persons;"
      },
      "url": "jdbc:mysql://localhost:1433/testDatabase"
    }
  },
  "pipelineId": "af856747-2343-46b1-a1d0-afd4d189d7c4",
  "credentialId": "004ec9f2-43f2-48b7-ae6f-8770bf6cc8e9",
  "scanAction": "scan",
  "processAction": "process",
  "batchSize": 100
}

Configuration parameters:

seed.url

(Required, String) A JDBC connection url, containing the database name.

seed.scan.fullScanSQL

(Required, String) SQL statement to be used for fetching record ids from the database.

seed.scan.fields.id

(Required, String) Identifier column name on the database for each record.

seed.scan.fields.hash

(Required, List<String>) List of fields used for hashing the data in order to provide incremental updates.

seed.pull.objectSQL

(Optional, String) SQL statement to retrieve each record after scanning the data in the scan action.

pull.id

(Required, String) Identifier column name on the database for each record.

Scan action

Runs a SQL statement to extract records identifiers before processing each of them.

Example configuration in a processor:

{
  "config": {
    "seed": {
      "pull": {
        "objectSQL": "SELECT PersonID, LastName, FirstName, Address, City FROM Persons WHERE PersonID IN(?);",
        "id": "PersonID"
      },
      "scan": {
        "fields": {
          "id": "PersonID",
          "hash": [
            "PersonID"
          ]
        },
        "fullScanSQL": "SELECT PersonID, LastName, FirstName, Address, City FROM Persons;"
      },
      "url": "jdbc:mysql://localhost:1433/testDatabase"
    }
  },
  "scanAction": "scan"
}

©2024 Pureinsights Technology Corporation