Open AI Processor

This processor uses Open AI's API for embedding and chat completion features (see here).

Action: Process (Embeddings)

Uses Open AI's API embeddings feature (see here) to get a vector representation of a given input.

Configuration

Example configuration in a processor:

{
  "type": "open-ai-processor",
  "sourceField": "description",
  "output": "embeddings",
  "multiSourceFieldSeparator": " ",
  "model": "text-embedding-ada-002",
  "timeout": "PT120S",
  "user": "pdp-embeddings",
  "backoffType": "exponential",
  "backoffInitialDelay": "PT10S",
  "backoffMaxRetries": 7,
  "credentialId": "b0fa9c0f-912f-4345-88cf-1d78bdf462f1"
}

Configuration parameters:

sourceField

(Optional, String/List) field with the input text. Default is cleanContent. If multiple fields are provided, they will be concatenated with the value specified in multiSourceFieldSeparator.

...

In case the configured model is from an unsupported version, the default model generation is V2.

Input/Output examples

• Processor

{
  "name": "Embeddings processor",
  "type": "open-ai-processor",
  "sourceField": "description",
  "output": "embeddings",
  "user": "pdp-embeddings",
  "backoffType": "constant",
  "backoffInitialDelay": "PT10S",
  "backoffMaxRetries": 25,
  "ignoreExceedingInput": false,
  "credentialId": "b0fa9c0f-912f-4345-88cf-1d78bdf462f1",
  "id": "fa16215c-1ff1-4233-8519-bb763144b7e9"
}

• Input

{
  "description": "JavaScript lies at the heart of almost every modern web application, from social apps like Twitter to browser-based game frameworks like Phaser and Babylon. Though simple for beginners to pick up and play with, JavaScript is a flexible, complex language that you can use to build full-scale applications."
}

• Output

{
  "description": "JavaScript lies at the heart of almost every modern web application, from social apps like Twitter to browser-based game frameworks like Phaser and Babylon. Though simple for beginners to pick up and play with, JavaScript is a flexible, complex language that you can use to build full-scale applications",
  "embeddings": [
    -0.0070322175,
    0.031458195,
    0.02097213,
    ...,
    0.019307466,
    -0.037907124
  ]
}

Empty/Null text to encode

It is required that the text provided in the source fields is not null or empty. If the text provided is invalid, the processor will ignore it and the output will be an empty array.

• Processor

{
  "name": "Embeddings processor",
  "type": "open-ai-processor",
  "sourceField": "description",
  "output": "embeddings",
  "user": "pdp-embeddings",
  "backoffType": "constant",
  "backoffInitialDelay": "PT10S",
  "backoffMaxRetries": 25,
  "credentialId": "b0fa9c0f-912f-4345-88cf-1d78bdf462f1",
  "id": "fa16215c-1ff1-4233-8519-bb763144b7e9"
}

• Input

{
  "description": ""
}

• Output

{
  "description": "",
  "embeddings": []
}

Action: Completion

Uses Open AI's API chat completion feature (see here) to get a response from a given prompt.

Image Added Important Note: Be aware that regardless of the number of records configured per job, a separate request to the OpenAI API will be made for each record. This means that the number of API calls could be equal to the number of records, which might impact your API usage and potentially your billing if you’re using a paid API. Please consider this when configuring and running your jobs.

Configuration

Example configuration in a processor:

{
  "type": "open-ai-processor",
  "name": "Chat Completion",
  "model": "gpt-3.5-turbo",
  "promptField": "prompt",
  "output": "response",
  "temperature": "0.8",
  "maxPromptTokens": 8192,
  "user": "pdp-chat-completion",
  "timeout": "PT120S",
  "backoffType": "exponential",
  "backoffInitialDelay": "PT10S",
  "backoffMaxRetries": 7,
  "credentialId": "b0fa9c0f-912f-4345-88cf-1d78bdf462f1"
}

Configuration parameters:

promptField

(String) Field with the prepared prompt for chat completion. The prompt must contain the context if needed.

As a recommendation to add the contents of other fields into the prompt in the prompt field, use a field mapper template or a script processor to insert the values as needed.

output

(Optional, String) The records field where the response will be stored, defaults to chatGptMessage

model

(Optional, String) The Open AI chat completion model to use in requests, defaults to gpt-3.5-turbo. See Model endpoint compatibility (see here).

temperature

(Optional, String of a Double) What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Defaults to "0.8"

maxPromptTokens

(Optional, Integer) The maximum amount of tokens that the prompt field can have. Implemented because if it surpasses 8192, the API will not give an answer even though the request was made. Defaults to 8192.

timeout

(Optional, String) The timeout on chat completion requests. Expressed in Duration notation, defaults to PT120S

user

(Optional, String) The user to be included in chat completion requests. If specified as null, no user will be included in requests. If not specified at all, defaults to pdp-content-processor.

backoffType

(Optional, String) The type of backoff to apply to the retries of the check for a passed cooldown. Options are none, constant, and exponential. Defaults to constant.

backoffInitialDelay

(Optional, String) The initial delay between backoff checks for a passed cooldown. Expressed in Duration notation, defaults to PT10S

backoffMaxRetries

(Optional, Integer) Maximum amount of times a processor tries to check for a passed cooldown before failing a job. Defaults to 25

Input/Output examples

• Processor

{
  "name": "Chat Completion Action",
  "type": "open-ai-processor",
  "credentialId": "b0fa9c0f-912f-4345-88cf-1d78bdf462f1",
  "model": "gpt-3.5-turbo-16k",
  "promptField": "prompt",
  "output": "response",
  "temperature": 0.3,
  "user": "pdp-chat-completion-all",
  "timeout": "PT120S",
  "backoffType": "exponential",
  "backoffInitialDelay": "PT10S",
  "backoffMaxRetries": 7
}

• Input

{
  "prompt": "For the title given 'Guest Blog: What is OpenSearch? - Pureinsights', generate questions that can be asked."
}

• Output

{
  "prompt": "For the title given 'Guest Blog: What is OpenSearch? - Pureinsights', generate questions that can be asked.",
  "response": "1. What is the main purpose of OpenSearch?\n2. How does OpenSearch differ from other search engines?\n3. What are the key features of OpenSearch?\n4. How can businesses benefit from using OpenSearch?\n5. What are the potential challenges or limitations of implementing OpenSearch?\n6. How does OpenSearch ensure data privacy and security?\n7. Can OpenSearch be integrated with existing systems and applications?\n8. What is the process for setting up and configuring OpenSearch?\n9. How does OpenSearch handle scalability and performance issues?\n10. Are there any case studies or success stories of organizations using OpenSearch effectively?"
}

Credentials

This processor requires a special type of credentials due to the authentication to the Open AI API being done through an API token. The credentials also allow some configuration of the request cooldown functionality, which will be explained latter. The processor expects the presence of the following values in the config of the credential:

...

The credential must be of type open-ai-component, otherwise the processors using them will fail.

Request cooldown

Open AI processors come with a built-in request cooldown functionality to try to handle and minimize rate limit errors returned by Open AI API. How it works is that once a request made by a processor returns a rate limit error, the requests of all processors that share the same credential config values and timeout config value will be put in cooldown for the length of time specified in the credential. This cooldown can be applied again if, once the first cooldown has passed, a new request receives a rate limit error.

...

Given that the rate limiting on Open AI API's side applies at the organization level first, and API token second, it is recommended to keep as many processors as possible sharing the same cooldown if they share the same API token and organization id in their credentials. Or in other words, it is recommended, if two or more processors share the same API token and organization id, to have them share the same requestCooldown value in the credential and timeout value in their configs, because then they will have a shared request cooldown and so less rate limit errors will be returned.

Input/Output examples

• Processor

{
  "name": "Embeddings processor",
  "type": "open-ai-processor",
  "sourceField": "description",
  "output": "embeddings",
  "user": "pdp-embeddings",
  "backoffType": "constant",
  "backoffInitialDelay": "PT10S",
  "backoffMaxRetries": 25,
  "ignoreExceedingInput": false,
  "credentialId": "b0fa9c0f-912f-4345-88cf-1d78bdf462f1",
  "id": "fa16215c-1ff1-4233-8519-bb763144b7e9"
}

• Input

{
  "description": "JavaScript lies at the heart of almost every modern web application, from social apps like Twitter to browser-based game frameworks like Phaser and Babylon. Though simple for beginners to pick up and play with, JavaScript is a flexible, complex language that you can use to build full-scale applications."
}

• Output

{
  "description": "JavaScript lies at the heart of almost every modern web application, from social apps like Twitter to browser-based game frameworks like Phaser and Babylon. Though simple for beginners to pick up and play with, JavaScript is a flexible, complex language that you can use to build full-scale applications",
  "embeddings": [
    -0.0070322175,
    0.031458195,
    0.02097213,
    ...,
    0.019307466,
    -0.037907124
  ]
}

Empty/Null text to encode

It is required that the text provided in the source fields is not null or empty. If the text provided is invalid, the processor will ignore it and the output will be an empty array.

• Processor

{
  "name": "Embeddings processor",
  "type": "open-ai-processor",
  "sourceField": "description",
  "output": "embeddings",
  "user": "pdp-embeddings",
  "backoffType": "constant",
  "backoffInitialDelay": "PT10S",
  "backoffMaxRetries": 25,
  "credentialId": "b0fa9c0f-912f-4345-88cf-1d78bdf462f1",
  "id": "fa16215c-1ff1-4233-8519-bb763144b7e9"
}

• Input

{
  "description": ""
}

• Output

...

Additional Reference

Open AI's public API reference