JSON Connector

This connector loads the content of a json file into the record's data.

Incoming record must contain a reference to a JSON file.

The file can be one of:

• A Json file with a single JSON object
• A NDJSON file with multiple JSON objects, one per line

In case where the file is NDJSON, each line will be read and a new child record will be created and enqueued on the current processing pipeline .

All child items will have the same action as the parent item.

API

No external API other the one already used by the PDP framework (Jackson).

Configuration

Sample configuration in a processor:

{
  "json": {
    "key": "fieldWithJsonContent",
    "ndjson": true,
    "idPointer": "id pointer"
  },
  "name": "Json processor",
  "type": "json-processor"
}

##Configuration parameters:

json.key - Required, string

Name of the record's field containing the JSON file content

json.ndjson - Optional, boolean (default false)

Whether the incoming file will contain a NDJSON file.

If false, JSON will be assumed.

json.idPointer - Optional, string (default null)

Only required if json.ndjson=true

Points to the field on the JSON objects where an id can be found

#Examples:

Example 1:

A processor configured with

{
  "json": {
    "key": "json"
  },
  "name": "Json processor",
  "type": "json-processor"
}

receives a record with a JSON field with the content

{  
  "key":"value",  
  "key2":1,
  "key3":{
    "subKey":"123",    
    "list":[
      {"k1":"v11","k2":"v12"},
      {"k1":"v21","k2":"v22"}
    ]
  }
}

will produce an output the looks like

{
  "__pdp__" : { ... pdp section ... },
  "json" : {... json content, probably binary reference ...},
  "key":"value",
  "key2":1,
  "key3":{
    "subKey":"123",
    "list":[
      {"k1":"v11","k2":"v12"},
      {"k1":"v21","k2":"v22"}
    ]
  }
}

Example 2:

A processor configured with

{
  "json": {
    "key": "json",
    "ndjson": true,
    "idPointer": "/key"
  },
  "name": "Json processor",
  "type": "json-processor"
}

receives a record with a NDJSON field with the content

{"key":"value", "key2":1, "key3":{"subKey":"123","list":[{"k1":"v11","k2":"v12"},{"k1":"v21","k2":"v22"}]}}
{"key":"value2", "key2":1, "key3":{"subKey":"123","list":[{"k1":"v11","k2":"v12"},{"k1":"v21","k2":"v22"}]}}

will produce two child records:

{
  "__pdp__" : { ... pdp section ... },  
  "key":"value",
  "key2":1,
  "key3":{
    "subKey":"123",
    "list":[
      {"k1":"v11","k2":"v12"},
      {"k1":"v21","k2":"v22"}
    ]
  }
}

and

{
  "__pdp__" : { ... pdp section ... },  
  "key":"value2",
  "key2":1,
  "key3":{
    "subKey":"123",
    "list":[
      {"k1":"v11","k2":"v12"},
      {"k1":"v21","k2":"v22"}
    ]
  }
}