...
(Optional, Integer) Maximum amount of times a processor tries to check for a passed cooldown before failing a job. Defaults to 25
ignoreExceedingInput
(Optional, Boolean) Defines if a record text that exceeds the limit of tokens is ignored or truncated. If true, the record is ignored. Otherwise, the record text is truncated. Defaults to false
Supported models generations are V1 and V2, respectively denoted as -001 and -002 in the model ID. In case of truncating a text, V1 models limit the number of tokens to 2046, and the limit in V2 models is of 8192 tokens. A token is equivalent to 3 characters.
In case the configured model is from an unsupported version, the default model generation is V2.
Credentials
This processor requires a special type of credentials due to the authentication to the Open AI API being done through an API token. The credentials also allow some configuration of the request cooldown functionality, which will be explained latter. The processor expects the presence of the following values in the config of the credential:
...
{
"name": "Embeddings processor",
"type": "open-ai-processor",
"sourceField": "description",
"output": "embeddings",
"user": "pdp-embeddings",
"backoffType": "constant",
"backoffInitialDelay": "PT10S",
"backoffMaxRetries": 25,
"ignoreExceedingInput": false,
"credentialId": "b0fa9c0f-912f-4345-88cf-1d78bdf462f1",
"id": "fa16215c-1ff1-4233-8519-bb763144b7e9"
}
...
{
"description": "JavaScript lies at the heart of almost every modern web application, from social apps like Twitter to browser-based game frameworks like Phaser and Babylon. Though simple for beginners to pick up and play with, JavaScript is a flexible, complex language that you can use to build full-scale applications",
"embeddings": [
-0.0070322175,
0.031458195,
0.02097213,
...,
0.019307466,
-0.037907124
]
}
Empty/Null text to encode
It is required that the text provided in the source fields is not null or empty. If the text provided is invalid, the processor will ignore it and the output will be an empty array.
- Processor
{
"name": "Embeddings processor",
"type": "open-ai-processor",
"sourceField": "description",
"output": "embeddings",
"user": "pdp-embeddings",
"backoffType": "constant",
"backoffInitialDelay": "PT10S",
"backoffMaxRetries": 25,
"credentialId": "b0fa9c0f-912f-4345-88cf-1d78bdf462f1",
"id": "fa16215c-1ff1-4233-8519-bb763144b7e9"
}
- Input
{
"description": ""
}
- Output
{
"description": "",
"embeddings": []
}