Mongo connector

Connects to a MongoDB server requested and scans all the data from a database collection

Pre-Requisites

MongoDB server running

Configuration

Example configuration in a processor:

{
  "name": "MongoDB Connector",
  "type": "mongo-connector",
  "scan": {
    "servers": [
      {
        "host": "localhost",
        "port": 8080
      }
    ],
    "url": "mongodb+srv://atlascluster.igqrdsg.mongodb.net",
    "tls": true,
    "database": "MyDatabase",
    "collection": "MyCollection"
  }
}

Configuration parameters:

servers.host

(Required*, String) The host(s) where the instance of the MongoDB is located

servers.port

(Required*, Integer) The port(s) on which MongoDB is running

url

(Required*, String) A MongoDB connection url, optionally containing the database and collection names.

• Example
  • mongodb+srv://atlascluster.igqrdsg.mongodb.net/myDatabase.myCollection

* Only one connection method (either servers or url is required. If both are given, the url takes precedence)

tls

(Optional, Boolean) Whether tls should be enabled. Defaults to false unless a Mongo Atlas url (mongo+srv://) is used in which case the default is true

database

(Required*, String) The name of the MongoDB database from where the data will be scanned

collection

(Required*, String) The name of the MongoDB collection to retrieve data from

* The database name (database) and collection name (collection) need not be explicitly defined if they are included in the url parameter

Scan action

Example configuration in a processor:

{
  "name": "MongoDB Connector",
  "type": "mongo-connector",
  "scan": {
    "idField": "field"
  }
}

Configuration parameters:

idField

(Optional, String) Field from the data that will be used as the identifier. Defaults to _id

Query action

Queries the MongoDB collection with an aggregation pipeline that will output a record per distinct field value with a total amount of repetitions.

Example configuration in a processor:

{
  "name": "MongoDB Connector",
  "type": "mongo-connector",
  "scan": {
    "fields": [
      "fieldA",
      "fieldB"
    ],
    "unwind": true
  },
  "scanAction": "query"
}

Configuration parameters:

fields

(Required*, List) A list of strings for each field to group by. Fields cannot be blank.

unwind

(Required*, boolean) Whether to use unwind on the field. Defaults to false.

Example

With an Atlas collection such as the following:

[
  {
    "keyExample": "fieldA"
  },
  {
    "keyExample": [
      "fieldA",
      "fieldB"
    ]
  }
]

If the unwind field is set to true each different value will be returned as an individual record.

IDs are generated as MD5 hash values using the field and value in that order.

[
  {
    "_id": "3dfef114c7192d37e1c44efda34a9093",
    "field": "keyExample",
    "value": "fieldA",
    "total": 2
  },
  {
    "_id": "3dfef114c7192d37e1c44efda34a9093",
    "field": "keyExample",
    "value": "fieldB",
    "total": 1
  }
]

Otherwise, if the unwind field is set to false the array will count as an individual instance.

[
  {
    "_id": "3dfef114c7192d37e1c44efda34a9093",
    "field": "keyExample",
    "value": "fieldA",
    "total": 1
  },
  {
    "_id": "965b337914099d52d611c99cfcd1f6e7",
    "field": "fieldA",
    "value": [
      "fieldA",
      "fieldB"
    ],
    "total": 1
  }
]