# Opengauss Integration (`wyswyz/opengauss-integration`)

This integration transfers data from Apify Actors to an openGauss database and is a good starting point for a question-answering, search, or RAG use case.

- **URL**: https://beta-securitybyobscurity.apify.com/wyswyz/opengauss-integration.md
- **Developed by:** [Y Wang](https://beta-securitybyobscurity.apify.com/wyswyz) (community)
- **Categories:** Integrations, AI, Open source
- **Stats:** 2 total users, 2 monthly users, 87.5% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage

# README

## openGauss integration

The Apify openGauss integration transfers selected data from Apify Actors to a [openGauss](https://opengauss.org/) database.
It processes the data, optionally splits it into chunks, computes embeddings, and saves them to openGauss.

This integration supports incremental updates, updating only the data that has changed.
This approach reduces unnecessary embedding computation and storage operations, making it suitable for search and retrieval augmented generation (RAG) use cases.

💡 **Note**: This Actor is meant to be used together with other Actors' integration sections.
For instance, if you are using the [Website Content Crawler](https://apify.com/apify/website-content-crawler), you can activate openGauss integration to save web data as vectors to openGauss.

### 📋 How does Apify-openGauss integration work?

Apify openGauss integration computes text embeddings and store them in openGauss.
It uses [LangChain](https://www.langchain.com/) to compute embeddings and interact with [openGauss](https://opengauss.org/).

1. Retrieve a dataset as output from an Actor
2. *\[Optional]* Split text data into chunks using `langchain`'s `RecursiveCharacterTextSplitter`
   (enable/disable using `performChunking` and specify `chunkSize`, `chunkOverlap`)
3. *\[Optional]* Update only changed data (select `dataUpdatesStrategy`)
4. Compute embeddings, e.g. using `OpenAI` or `Cohere` (specify `embeddings` and `embeddingsConfig`)
5. Save data into the database

### ✅ Before you start

To utilize this integration, ensure you have:

- Created or existing `openGauss` database. You need to know `opengaussHost`, `opengaussPort`, `opengaussUser`, `opengaussPassword`, `opengaussDBName`and `opengaussTableName`.
- An account to compute embeddings using one of the providers, e.g., [OpenAI](https://platform.openai.com/docs/guides/embeddings) or [Cohere](https://docs.cohere.com/docs/cohere-embed).

### 👉 Examples

The configuration consists of three parts: openGauss, embeddings provider, and data.

Ensure that the vector size of your embeddings aligns with the configuration of your openGauss.
For instance, if you're using the `text-embedding-3-small` model from `OpenAI`, it generates vectors of size `1536`.
This means your openGauss vector should also be configured to accommodate vectors of the same size, `1536` in this case.

For detailed input information refer to the [Input page]().

##### Database: openGauss

```json
{
  "opengaussHost": "YOUR-opengaussHost",
  "opengaussPort": "YOUR-opengaussPort",
  "opengaussUser": "YOUR-opengaussUser",
  "opengaussPassword": "YOUR-opengaussPassword",
  "opengaussDBname": "YOUR-opengaussDBname",
  "opengaussTableName": "apif_collection"
}
```

##### Embeddings provider: OpenAI

```json
{
  "embeddingsProvider": "OpenAIEmbeddings",
  "embeddingsApiKey": "YOUR-OPENAI-API-KEY",
  "embeddingsConfig": {"model":  "text-embedding-3-large"}
}
```

#### Save data from Website Content Crawler to openGauss

Data is transferred in the form of a dataset from [Website Content Crawler](https://apify.com/apify/website-content-crawler), which provides a dataset with the following output fields (truncated for brevity):

```json
{
  "url": "https://www.apify.com",
  "text": "Apify is a platform that enables developers to build, run, and share automation tasks.",
  "metadata": {"title": "Apify"}
}
```

This dataset is then processed by the openGauss integration.
In the integration settings you need to specify which fields you want to save to openGauss, e.g., `["text"]` and which of them should be used as metadata, e.g., `{"title": "metadata.title"}`.
Without any other configuration, the data is saved to openGauss as is.

```json
{
  "datasetFields": ["text"],
  "metadataDatasetFields": {"title": "metadata.title"}
}
```

#### Create chunks from Website Content Crawler data and save them to the database

Assume that the text data from the [Website Content Crawler](https://apify.com/apify/website-content-crawler) is too long to compute embeddings.
Therefore, we need to divide the data into smaller pieces called chunks.
We can leverage LangChain's `RecursiveCharacterTextSplitter` to split the text into chunks and save them into a database.
The parameters `chunkSize` and `chunkOverlap` are important.
The settings depend on your use case where a proper chunking helps optimize retrieval and ensures accurate responses.

```json
{
  "datasetFields": ["text"],
  "metadataDatasetFields": {"title": "metadata.title"},
  "performChunking": true,
  "chunkSize": 1000,
  "chunkOverlap": 0
}
```

#### Configure update strategy

To control how the integration updates data in the database, use the `dataUpdatesStrategy` parameter. This parameter allows you to choose between different update strategies based on your use case, such as adding new data, upserting records, or incrementally updating records based on changes (deltas). Below are the available strategies and explanations for when to use each:

- **Add data (`add`)**:
  - Appends new data to the database without checking for duplicates or updating existing records.
  - Suitable for cases where deduplication or updates are unnecessary, and the data simply needs to be added.
  - For example, you might use this strategy to continually append data from independent crawls without regard for overlaps.

- **Upsert data (`upsert`)**:
  - Delete existing records in the database if they match a key or identifier and inserts new records.
  - Ideal when you want to maintain accurate and up-to-date data while avoiding duplication.
  - For instance, this is useful in cases where unique items (such as user profiles or documents) need to be managed, ensuring the database reflects the latest changes.
  - Check the `dataUpdatesPrimaryDatasetFields` parameter to specify which fields are used to uniquely identify each dataset item.

- **Delta updates (`deltaUpdates`)**:
  - Incrementally updates records by identifying differences (deltas) between the new dataset and the existing database records.
  - Ensures only new or modified records are processed, leaving unchanged records untouched. This minimizes unnecessary database operations and improves efficiency.
  - This is the most efficient strategy when integrating data that evolves over time, such as website content or recurring crawls.
  - Check the `dataUpdatesPrimaryDatasetFields` parameter to specify which fields are used to uniquely identify each dataset item.

#### Incrementally update database from the Website Content Crawler

To incrementally update data from the [Website Content Crawler](https://apify.com/apify/website-content-crawler) to database, configure the integration to update only the changed or new data.
This is controlled by the `dataUpdatesStrategy` setting.
This way, the integration minimizes unnecessary updates and ensures that only new or modified data is processed.

A checksum is computed for each dataset item (together with all metadata) and stored in the database alongside the vectors.
When the data is re-crawled, the checksum is recomputed and compared with the stored checksum.
If the checksum is different, the old data (including vectors) is deleted and new data is saved.
Otherwise, only the `last_seen_at` metadata field is updated to indicate when the data was last seen.

##### Provide unique identifier for each dataset item

To incrementally update the data, you need to be able to uniquely identify each dataset item.
The variable `dataUpdatesPrimaryDatasetFields` specifies which fields are used to uniquely identify each dataset item and helps track content changes across different crawls.
For instance, when working with the Website Content Crawler, you can use the URL as a unique identifier.

```json
{
  "dataUpdatesStrategy": "deltaUpdates",
  "dataUpdatePrimaryDatasetFields": ["url"]
}
```

To fully maximize the potential of incremental data updates, it is recommended to start with an empty database.
While it is possible to use this feature with an existing database, records that were not originally saved using a prefix or metadata will not be updated.

#### Delete outdated (expired) data

The integration can delete data from the database that hasn't been crawled for a specified period, which is useful when data becomes outdated, such as when a page is removed from a website.

The deletion feature can be enabled or disabled using the `deleteExpiredObjects` setting.

For each crawl, the `last_seen_at` metadata field is created or updated.
This field records the most recent time the data object was crawled.
The `expiredObjectDeletionPeriodDays` setting is used to control number of days since the last crawl, after which the data object is considered expired.
If a database object has not been seen for more than the `expiredObjectDeletionPeriodDays`, it will be deleted automatically.

The specific value of `expiredObjectDeletionPeriodDays` depends on your use case.

- If a website is crawled daily, `expiredObjectDeletionPeriodDays` can be set to 7.
- If you crawl weekly, it can be set to 30.

To disable this feature, set `deleteExpiredObjects` to `false`.

```json
{
  "deleteExpiredObjects": true,
  "expiredObjectDeletionPeriodDays": 30
}
```

💡 If you are using multiple Actors to update the same database, ensure that all Actors crawl the data at the same frequency.
Otherwise, data crawled by one Actor might expire due to inconsistent crawling schedules.

### 💾 Outputs

This integration will save the selected fields from your Actor to openGauss.

### 🔢 Example configuration

##### Full Input Example for Website Content Crawler Actor with openGauss integration

```json
{
  "opengaussHost": "YOUR-opengaussHost",
  "opengaussPort": "YOUR-opengaussPort",
  "opengaussUser": "YOUR-opengaussUser",
  "opengaussPassword": "YOUR-opengaussPassword",
  "opengaussDBname": "YOUR-opengaussDBname",
  "opengaussTableName": "apif_collection",
  "embeddingsApiKey": "YOUR-OPENAI-API-KEY",
  "embeddingsConfig": {
    "model": "text-embedding-3-small"
  },
  "embeddingsProvider": "OpenAI",
  "datasetFields": [
    "text"
  ],
  "dataUpdatesStrategy": "deltaUpdates",
  "dataUpdatePrimaryDatasetFields": ["url"],
  "expiredObjectDeletionPeriodDays": 7,
  "performChunking": true,
  "chunkSize": 2000,
  "chunkOverlap": 200
}
```

##### openGauss

```json
{
  "opengaussHost": "YOUR-opengaussHost",
  "opengaussPort": "YOUR-opengaussPort",
  "opengaussUser": "YOUR-opengaussUser",
  "opengaussPassword": "YOUR-opengaussPassword",
  "opengaussDBname": "YOUR-opengaussDBname",
  "opengaussTableName": "apif_collection"
}
```

##### OpenAI embeddings

```json
{
  "embeddingsApiKey": "YOUR-OPENAI-API-KEY",
  "embeddings": "OpenAI",
  "embeddingsConfig": {"model":  "text-embedding-3-large"}
}
```

##### Cohere embeddings

```json
{
  "embeddingsApiKey": "YOUR-COHERE-API-KEY",
  "embeddings": "Cohere",
  "embeddingsConfig": {"model":  "embed-multilingual-v3.0"}
}
```

# Actor input Schema

## `opengaussHost` (type: `string`):

The Host of openGauss

## `opengaussPort` (type: `string`):

The Port of openGauss

## `opengaussUser` (type: `string`):

The User of openGauss

## `opengaussPassword` (type: `string`):

The Password of openGauss

## `opengaussDBname` (type: `string`):

The DBname of openGauss

## `opengaussTableName` (type: `string`):

The name of the table to use

## `embeddingsProvider` (type: `string`):

Choose the embeddings provider to use for generating embeddings

## `embeddingsConfig` (type: `object`):

Configure the parameters for the LangChain embedding class. Key points to consider:

1. Typically, you only need to specify the model name. For example, for OpenAI, set the model name as {"model": "text-embedding-3-small"}.

2. It's required to ensure that the vector size of your embeddings matches the size of embeddings in the database.

3. Here are examples of embedding models:
   - [OpenAI](https://platform.openai.com/docs/guides/embeddings): `text-embedding-3-small`, `text-embedding-3-large`, etc.
   - [Cohere](https://docs.cohere.com/docs/cohere-embed): `embed-english-v3.0`, `embed-multilingual-light-v3.0`, etc.

4. For more details about other parameters, refer to the [LangChain documentation](https://python.langchain.com/docs/integrations/text_embedding/).

## `embeddingsApiKey` (type: `string`):

Value of the API KEY for the embeddings provider (if required).

For example for OpenAI it is OPENAI\_API\_KEY, for Cohere it is COHERE\_API\_KEY)

## `datasetFields` (type: `array`):

This array specifies the dataset fields to be selected and stored in the vector store. Only the fields listed here will be included in the vector store.

For instance, when using the Website Content Crawler, you might choose to include fields such as `text`, `url`, and `metadata.title` in the vector store.

## `metadataDatasetFields` (type: `object`):

A list of dataset fields which should be selected from the dataset and stored as metadata in the vector stores.

For example, when using the Website Content Crawler, you might want to store `url` in metadata. In this case, use `metadataDatasetFields parameter as follows {"url": "url"}`

## `metadataObject` (type: `object`):

This object allows you to store custom metadata for every item in the vector store.

For example, if you want to store the `domain` as metadata, use the `metadataObject` like this: {"domain": "apify.com"}.

## `datasetId` (type: `string`):

Dataset ID (when running standalone without integration)

## `dataUpdatesStrategy` (type: `string`):

Choose the update strategy for the integration. The update strategy determines how the integration updates the data in the database.

The available options are:

- **Add data** (`add`):
  - Always adds new records to the database.
  - No checks for existing records or updates are performed.
  - Useful when appending data without concern for duplicates.

- **Upsert data** (`upsert`):
  - Updates existing records if they match a key or identifier.
  - Inserts new records into the database if they don't already exist.
  - Ideal for ensuring the database contains the most up-to-date data, avoiding duplicates.

- **Update changed data based on deltas** (`deltaUpdates`):
  - Performs incremental updates by identifying differences (deltas) between the new dataset and the existing records.
  - Only adds new records and updates those that have changed.
  - Unchanged records are left untouched.
  - Maximizes efficiency by reducing unnecessary updates.

Select the strategy that best fits your use case.

## `dataUpdatesPrimaryDatasetFields` (type: `array`):

This array contains fields that are used to uniquely identify dataset items, which helps to handle content changes across different runs.

For instance, in a web content crawling scenario, the `url` field could serve as a unique identifier for each item.

## `enableDeltaUpdates` (type: `boolean`):

When set to true, this setting enables incremental updates for objects in the database by comparing the changes (deltas) between the crawled dataset items and the existing objects, uniquely identified by the `datasetKeysToItemId` field.

The integration will only add new objects and update those that have changed, reducing unnecessary updates. The `datasetFields`, `metadataDatasetFields`, and `metadataObject` fields are used to determine the changes.

## `deltaUpdatesPrimaryDatasetFields` (type: `array`):

This array contains fields that are used to uniquely identify dataset items, which helps to handle content changes across different runs.

For instance, in a web content crawling scenario, the `url` field could serve as a unique identifier for each item.

## `deleteExpiredObjects` (type: `boolean`):

When set to true, delete objects from the database that have not been crawled for a specified period.

## `expiredObjectDeletionPeriodDays` (type: `integer`):

This setting allows the integration to manage the deletion of objects from the database that have not been crawled for a specified period. It is typically used in subsequent runs after the initial crawl.

When the value is greater than 0, the integration checks if objects have been seen within the last X days (determined by the expiration period). If the objects are expired, they are deleted from the database. The specific value for `deletedExpiredObjectsDays` depends on your use case and how frequently you crawl data.

For example, if you crawl data daily, you can set `deletedExpiredObjectsDays` to 7 days. If you crawl data weekly, you can set `deletedExpiredObjectsDays` to 30 days.

## `performChunking` (type: `boolean`):

When set to true, the text will be divided into smaller chunks based on the settings provided below. Proper chunking helps optimize retrieval and ensures accurate and efficient responses.

## `chunkSize` (type: `integer`):

Defines the maximum number of characters in each text chunk. Choosing the right size balances between detailed context and system performance. Optimal sizes ensure high relevancy and minimal response time.

## `chunkOverlap` (type: `integer`):

Specifies the number of overlapping characters between consecutive text chunks. Adjusting this helps maintain context across chunks, which is crucial for accuracy in retrieval-augmented generation systems.

## Actor input object example

```json
{
  "embeddingsProvider": "OpenAI",
  "datasetFields": [
    "text"
  ],
  "dataUpdatesStrategy": "deltaUpdates",
  "dataUpdatesPrimaryDatasetFields": [
    "url"
  ],
  "enableDeltaUpdates": true,
  "deltaUpdatesPrimaryDatasetFields": [
    "url"
  ],
  "deleteExpiredObjects": true,
  "expiredObjectDeletionPeriodDays": 30,
  "performChunking": true,
  "chunkSize": 2000,
  "chunkOverlap": 0
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "datasetFields": [
        "text"
    ],
    "dataUpdatesStrategy": "deltaUpdates",
    "dataUpdatesPrimaryDatasetFields": [
        "url"
    ],
    "deltaUpdatesPrimaryDatasetFields": [
        "url"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("wyswyz/opengauss-integration").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "datasetFields": ["text"],
    "dataUpdatesStrategy": "deltaUpdates",
    "dataUpdatesPrimaryDatasetFields": ["url"],
    "deltaUpdatesPrimaryDatasetFields": ["url"],
}

# Run the Actor and wait for it to finish
run = client.actor("wyswyz/opengauss-integration").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "datasetFields": [
    "text"
  ],
  "dataUpdatesStrategy": "deltaUpdates",
  "dataUpdatesPrimaryDatasetFields": [
    "url"
  ],
  "deltaUpdatesPrimaryDatasetFields": [
    "url"
  ]
}' |
apify call wyswyz/opengauss-integration --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=wyswyz/opengauss-integration",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Opengauss Integration",
        "description": "This integration transfers data from Apify Actors to an openGauss database and is a good starting point for a question-answering, search, or RAG use case.",
        "version": "0.0",
        "x-build-id": "MBvE2FSFujTOJSgY5"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/wyswyz~opengauss-integration/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-wyswyz-opengauss-integration",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/wyswyz~opengauss-integration/runs": {
            "post": {
                "operationId": "runs-sync-wyswyz-opengauss-integration",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/wyswyz~opengauss-integration/run-sync": {
            "post": {
                "operationId": "run-sync-wyswyz-opengauss-integration",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "opengaussHost",
                    "opengaussPort",
                    "opengaussUser",
                    "opengaussPassword",
                    "opengaussDBname",
                    "opengaussTableName",
                    "embeddingsProvider",
                    "embeddingsApiKey",
                    "datasetFields"
                ],
                "properties": {
                    "opengaussHost": {
                        "title": "openGauss Host",
                        "type": "string",
                        "description": "The Host of openGauss"
                    },
                    "opengaussPort": {
                        "title": "openGauss Port",
                        "type": "string",
                        "description": "The Port of openGauss"
                    },
                    "opengaussUser": {
                        "title": "openGauss User",
                        "type": "string",
                        "description": "The User of openGauss"
                    },
                    "opengaussPassword": {
                        "title": "openGauss Password",
                        "type": "string",
                        "description": "The Password of openGauss"
                    },
                    "opengaussDBname": {
                        "title": "openGauss DBname",
                        "type": "string",
                        "description": "The DBname of openGauss"
                    },
                    "opengaussTableName": {
                        "title": "openGauss SQL table name",
                        "type": "string",
                        "description": "The name of the table to use"
                    },
                    "embeddingsProvider": {
                        "title": "Embeddings provider (as defined in the langchain API)",
                        "enum": [
                            "OpenAI",
                            "Cohere"
                        ],
                        "type": "string",
                        "description": "Choose the embeddings provider to use for generating embeddings",
                        "default": "OpenAI"
                    },
                    "embeddingsConfig": {
                        "title": "Configuration for embeddings provider",
                        "type": "object",
                        "description": "Configure the parameters for the LangChain embedding class. Key points to consider:\n\n1. Typically, you only need to specify the model name. For example, for OpenAI, set the model name as {\"model\": \"text-embedding-3-small\"}.\n\n2. It's required to ensure that the vector size of your embeddings matches the size of embeddings in the database.\n\n3. Here are examples of embedding models:\n   - [OpenAI](https://platform.openai.com/docs/guides/embeddings): `text-embedding-3-small`, `text-embedding-3-large`, etc.\n   - [Cohere](https://docs.cohere.com/docs/cohere-embed): `embed-english-v3.0`, `embed-multilingual-light-v3.0`, etc.\n\n4. For more details about other parameters, refer to the [LangChain documentation](https://python.langchain.com/docs/integrations/text_embedding/)."
                    },
                    "embeddingsApiKey": {
                        "title": "Embeddings API KEY (whenever applicable, depends on provider)",
                        "type": "string",
                        "description": "Value of the API KEY for the embeddings provider (if required).\n\n For example for OpenAI it is OPENAI_API_KEY, for Cohere it is COHERE_API_KEY)"
                    },
                    "datasetFields": {
                        "title": "Dataset fields to select from the dataset results and store in the database",
                        "type": "array",
                        "description": "This array specifies the dataset fields to be selected and stored in the vector store. Only the fields listed here will be included in the vector store.\n\nFor instance, when using the Website Content Crawler, you might choose to include fields such as `text`, `url`, and `metadata.title` in the vector store.",
                        "default": [
                            "text"
                        ],
                        "items": {
                            "type": "string"
                        }
                    },
                    "metadataDatasetFields": {
                        "title": "Dataset fields to select from the dataset and store as metadata in the database",
                        "type": "object",
                        "description": "A list of dataset fields which should be selected from the dataset and stored as metadata in the vector stores.\n\nFor example, when using the Website Content Crawler, you might want to store `url` in metadata. In this case, use `metadataDatasetFields parameter as follows {\"url\": \"url\"}`"
                    },
                    "metadataObject": {
                        "title": "Custom object to be stored as metadata in the vector store database",
                        "type": "object",
                        "description": "This object allows you to store custom metadata for every item in the vector store.\n\nFor example, if you want to store the `domain` as metadata, use the `metadataObject` like this: {\"domain\": \"apify.com\"}."
                    },
                    "datasetId": {
                        "title": "Dataset ID",
                        "type": "string",
                        "description": "Dataset ID (when running standalone without integration)"
                    },
                    "dataUpdatesStrategy": {
                        "title": "Update strategy (add, upsert, deltaUpdates (default))",
                        "enum": [
                            "add",
                            "upsert",
                            "deltaUpdates"
                        ],
                        "type": "string",
                        "description": "Choose the update strategy for the integration. The update strategy determines how the integration updates the data in the database.\n\nThe available options are:\n\n- **Add data** (`add`):\n  - Always adds new records to the database.\n  - No checks for existing records or updates are performed.\n  - Useful when appending data without concern for duplicates.\n\n- **Upsert data** (`upsert`):\n  - Updates existing records if they match a key or identifier.\n  - Inserts new records into the database if they don't already exist.\n  - Ideal for ensuring the database contains the most up-to-date data, avoiding duplicates.\n\n- **Update changed data based on deltas** (`deltaUpdates`):\n  - Performs incremental updates by identifying differences (deltas) between the new dataset and the existing records.\n  - Only adds new records and updates those that have changed.\n  - Unchanged records are left untouched.\n  - Maximizes efficiency by reducing unnecessary updates.\n\nSelect the strategy that best fits your use case.",
                        "default": "deltaUpdates"
                    },
                    "dataUpdatesPrimaryDatasetFields": {
                        "title": "Dataset fields to uniquely identify dataset items (only relevant when dataUpdatesStrategy is `upsert` or `deltaUpdates`)",
                        "type": "array",
                        "description": "This array contains fields that are used to uniquely identify dataset items, which helps to handle content changes across different runs.\n\nFor instance, in a web content crawling scenario, the `url` field could serve as a unique identifier for each item.",
                        "default": [
                            "url"
                        ],
                        "items": {
                            "type": "string"
                        }
                    },
                    "enableDeltaUpdates": {
                        "title": "Enable incremental updates for objects based on deltas (deprecated)",
                        "type": "boolean",
                        "description": "When set to true, this setting enables incremental updates for objects in the database by comparing the changes (deltas) between the crawled dataset items and the existing objects, uniquely identified by the `datasetKeysToItemId` field.\n\n The integration will only add new objects and update those that have changed, reducing unnecessary updates. The `datasetFields`, `metadataDatasetFields`, and `metadataObject` fields are used to determine the changes.",
                        "default": true
                    },
                    "deleteExpiredObjects": {
                        "title": "Delete expired objects from the database",
                        "type": "boolean",
                        "description": "When set to true, delete objects from the database that have not been crawled for a specified period.",
                        "default": true
                    },
                    "expiredObjectDeletionPeriodDays": {
                        "title": "Delete expired objects from the database after a specified number of days",
                        "minimum": 0,
                        "type": "integer",
                        "description": "This setting allows the integration to manage the deletion of objects from the database that have not been crawled for a specified period. It is typically used in subsequent runs after the initial crawl.\n\nWhen the value is greater than 0, the integration checks if objects have been seen within the last X days (determined by the expiration period). If the objects are expired, they are deleted from the database. The specific value for `deletedExpiredObjectsDays` depends on your use case and how frequently you crawl data.\n\nFor example, if you crawl data daily, you can set `deletedExpiredObjectsDays` to 7 days. If you crawl data weekly, you can set `deletedExpiredObjectsDays` to 30 days.",
                        "default": 30
                    },
                    "performChunking": {
                        "title": "Enable text chunking",
                        "type": "boolean",
                        "description": "When set to true, the text will be divided into smaller chunks based on the settings provided below. Proper chunking helps optimize retrieval and ensures accurate and efficient responses.",
                        "default": true
                    },
                    "chunkSize": {
                        "title": "Maximum chunk size",
                        "minimum": 1,
                        "type": "integer",
                        "description": "Defines the maximum number of characters in each text chunk. Choosing the right size balances between detailed context and system performance. Optimal sizes ensure high relevancy and minimal response time.",
                        "default": 2000
                    },
                    "chunkOverlap": {
                        "title": "Chunk overlap",
                        "minimum": 0,
                        "type": "integer",
                        "description": "Specifies the number of overlapping characters between consecutive text chunks. Adjusting this helps maintain context across chunks, which is crucial for accuracy in retrieval-augmented generation systems.",
                        "default": 0
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```
