Field Mapper Processor

This processor is on charge of mapping fields and execute specific actions to them.

Configuration

Example configuration in a processor:

 {
  "type": "field-mapper",
  "name": "field mapper description",
  "mapping": [
    {
      "field": {
        "operation": {
          <operation configuration>
        }
      }
    }
  ]
}

Configuration parameters:

mapping (Array) List of fields in which the operation is going to be executed.

field (String) Field where the action is going to be executed. This field is going to be the one that will be persisted after the execution of this processor.

operation (String) Operation to execute on field.

operation configuration (Json) Configuration for the operation to be executed.

Operations:

CHOOSE

Copies the value of the first non-empty source field into a new field.

{
  "field": {
    "choose": [
      "/path/to/source/field",
      "source_field"
    ]
  }
}

choose

(List<String>) A list of field names and/or field pointers to fetch the data from the record.

CLEANHTML

Cleans data from the specified field returning only text found within HTML tags.

{
  "field": {
	"cleanHtml": "source_field"
  }
}

cleanHtml

(String) Field name to clean HTML tags from.

CONDITIONALTOKENKEEP

Conditionally keeps a field based on a criteria.

{
  "field": {
    "conditionalTokenKeep": {
	  "keep": true,
	  "gt": 1,
	  "tokenSplitter": "\\|"
	}
  }
}

keep

(Boolean) Flag indicating if field should be kept.

criteria

(Json) Criteria needed to be accomplished in order to keep/discard the field. Available comparators are: gt, gte, eq, lt, lte.

tokenSplitter

(Optional, String) Token to be used for splitting the field to keep. By default, the space token is the one to use.

COPY

Copies the value of a single field to another one.

{
  "field": {
    "copy": "source_field"
  }
}

copy

(String) Field name to copy the value from.

COPYTOCHILD

Copies the value of a single field to every element in a list.

{
  "field": {
    "copytochild": "source_field"
  }
}

copytochild

(String) Field name where a list of json documents is stored.

DEDUPLICATE

Deduplicates a list, leaving only unique values.

{
  "field": {
    "deduplicate": {
      "field": "/path/to/list"
    }
  }
}

field

(String) Field name or field pointer to a list within a record.

FLATTEN

Flattens a list.

{
  "field": {
    "flatten": {
      "field": "/path/to/list",
      "separator": "/"
    }
  }
}

field

(String) Field name or field pointer to a list within a record.

separator

(String) Token to be used as a separator for the flattened list.

HASH

Sets the hash code of a text or a list of texts into a field.

{
  "field": {
    "hash": "source_field"
  }
}

hash

(String) Field name where a string value or a list of strings is stored.

JOIN

Concats the values of fields using the specified separator.

{
  "field": {
    "join": {
      "fields": [
        "/path/to/source/field",
        "source_field"
      ],
      "separator": "/"
    }
  }
}

fields

(List<String>) List of field names and pointers to fetch the values from.

separator

(String) Token to be used as a separator for the joined fields.

JOINARRAYS

Joins multiple array fields into one.

{
  "field": {
    "joinarrays": {
      "fields": [
        "/path/to/source/field",
        "source_field"
      ]
    }
  }
}

fields

(List<String>) List of field names and pointers to fetch the values from.

KEEP

Keeps a field.

{
  "field": {
    "keep": true
  }
}

keep

(Boolean) Flag indicating if the field should be kept or not.

LOWERCASE

Converts all of the characters in a field to lower case.

{
  "field": {
    "lowercase": "source_field"
  }
}

lowercase

(String) Field name where a string value or a list of strings is stored.

NORMALIZE

Normalizes a field removing accents.

{
  "field": {
    "normalize": "source_field"
  }
}

normalize

(String) Field name where a string value is stored.

REGEX

Matches a regular expression against textual values coming with the record. Text can be replaced if a replacement text is passed, or matches could be extracted to a new field.

Replacement:

{
  "field": {
    "regex" : {
      "field": "source_field",
      "expression": "[^a-zA-Z ]",
      "replacement": ""
    }
  }
}

Move to field:

{
  "field": {
    "regex" : {
      "field": "source_field",
      "expression": "[^a-zA-Z ]"
    }
  }
}

field

(String) Field name where a string value is stored.

expression

(String) Regular expression to be matched.

replacement

(Optional, String) Text to be used as a replacement.

REMOVESPACES

Removes space characters from a field.

{
  "field": {
    "removespaces" : "source_field"
  }
}

removespaces

(String) Field name where a string value is stored.

REPLACE

Replaces content from a given field based on the parameters set.

{
  "field": {
    "replace": {
      "field": "source_field",
      "toReplace": "set",
      "replacement": "replace"
    }
  }
}

field

(String) Field name or field pointer to a string or a list of strings within a record.

toReplace

(String) Text to be replaced.

replacement

(String) The replacement text.

SET

Sets a literal value for a field.

{
  "field": {
    "set": "New value to set"
  }
}

set

(Object) Value to set for a field.

TEMPLATEREPLACE

Replaces placeholders in a template with actual values from the record data. The template can be either provided inline (starting with $) or referencing a file in the binary server.

{
  "field": {
    "templateReplace": {
      "jsonPathArraySeparator" : " ",
      "templates": [
        "${$.fieldArr[*]}"
      ],
      "toReplace": "1",
      "replacement": "8"
    }
  }
}

If you need to fetch a template from the binary server, you can use the following configuration:

{
  "field": {
    "templateReplace": {
      "jsonPathArraySeparator" : " ",
      "templates": [
        "templateName"
      ],
      "toReplace": "1",
      "replacement": "8"
    }
  }
}

templates

(List<String>) List of templates to use.

jsonPathArraySeparator

(String) String to be used as a list separator. By default, a space characters is used.

toReplace

(String) Text to be replaced.

replacement

(String) Replacement text to be used.

TOARRAY

Transforms a string value into a list of values using the specified separator character.

{
  "field": {
    "toarray": {
      "field": "source_field", 
      "separator": ",",
      "distinct": true
    }
  }
}

field

(String) Field name to extract the text content from.

separator

(String) Token used as a separator in the original text.

distinct

(Optional, Boolean) Flag to indicate whether the resulting list must contain only unique values or not.

UPPERCASE

Converts all of the characters in a field to upper case.

{
  "field": {
    "uppercase": "source_field"
  }
}

uppercase

(String) Field name where a string value or a list of strings is stored.

APPEND

Adds a constant string value at the end of the input string. If the value is a String it is added directly and if it is an array is added for each value of the array.

{
  "field": {
    "append": "Value to be appended"
  }
}

append

(String) Value to append for a field.

PREPEND

Adds a constant string value at the beginning of the input string. If the value is a String it is added directly and if it is an array is added for each value of the array.

{
  "field": {
    "prepend": "Value to be prepended"
  }
}

prepend

(String) Value to prepend for a field.