...
This processor uses the text received to add new vocabulary and train the designated model. To do this the processor makes use of Distilbert-squad HuggingFace Service.
Training a model takes a considerate amount of time, reason why the execution of this processor will probably take longer than expected.
...
Example configuration in a processor:
{
"trainingBatch": 2,
"batchSizetrainingBatchSize": 1000,
"model": "C:\\dev\\model",
"active": true,
"type": "bert-train-processor",
"sourceField": "text",
"servers": [
{
"port": 8888,
"host": "localhost"
}
],
"trainConnectTimeout": "PT30s",
"trainReadTimeout": "PT5m",
"timeInterval":"PT10s",
"name": "BERT trainer",
"id": "50dcc5e2-1fcd-40bc-82b7-a257b0ec38ed"
}
...
Batch to be used while training the model.
batchSize trainingBatchSize - Required, Int
Maximum amount of data to send to DistilBert-SquadHuggingFace Service.
model - Required, String
Path to the model to train, take into account that the model in the specified path will be overwritten by the trained model.
...
servers.host - Required, String
Host where Distilbert-Squad HuggingFace Service is located.
servers.port - Required, Int
Host port where Distilbert-Squad as a HuggingFace Service is located.
trainConnectTimeout - Optional, String
...