Elasticsearch integration

Overview

Elasticsearch tool provides full CRUD access to Elasticsearch documents and indices. Document operations: create (with auto-ID or explicit ID), delete, get, getAll (search), update. Index operations: create, delete, get, getAll (list all indices via _aliases). Bulk mode available for document create/update/delete — flushes every 50 items using NDJSON /_bulk endpoint. getAll with returnAll=true and a sort field uses point-in-time (PIT) + search_after pagination to page beyond 10,000 hits. getAll without sort uses a single request with size=10000. simple=true flattens _source fields to the top level. Auth: HTTP Basic Auth on every request using username + password from the elasticsearchApi credential.

Node Type

Action — processes input items and produces output

Input / Output

Direction	Port(s)
Input	Input
Output	Output, Error

Credentials

This tool requires Elasticsearch API credentials. See the Credentials Guide for setup instructions.

Resources

Resource	Value
Document	document
Index	index

Operations

Operation	Value	Description
Create	create	Create a document or index.
Delete	delete	Delete a document or index.
Get	get	Get a document or index.
Get Many	getAll	Get many documents or indices.
Update	update	Update a document. Available for the document resource only.

Parameters

Document: Create

Parameter	Type	Required	Default	Description
Index ID	string	Yes	—	ID of the index to add the document to.
Data to Send	options	No	defineBelow	How to send data to Elasticsearch.
				Options: defineBelow (set the value for each destination field), autoMapInputData (use when node input properties match destination field names)
Inputs to Ignore	string	No	—	Comma-separated list of input properties to skip. Leave empty to send all properties. (shown when Data to Send is autoMapInputData)
Fields to Send	fixedCollection	No	{}	The document fields to write, one row per field. (shown when Data to Send is defineBelow)
— Field Name	string	No	—	The document field name.
— Field Value	string	No	—	The value to store in that field.
Additional Fields	collection	No	{}	Optional settings for the write.
— Document ID	string	No	—	ID to assign to the document. If omitted, Elasticsearch auto-generates one.
— Routing	string	No	—	Target primary shard.
— Timeout	string	No	1m	Period to wait for active shards.
Options	collection	No	{}	Request behavior.
— Bulk Create	boolean	No	false	Whether to use the bulk operation to create documents.
— Pipeline ID	string	No	—	ID of the pipeline to use to preprocess incoming documents.
— Refresh	options	No	false	Refresh behavior after the operation.
				Options: true, wait_for, false

Document: Delete

Parameter	Type	Required	Default	Description
Index ID	string	Yes	—	ID of the index containing the document to delete.
Document ID	string	Yes	—	ID of the document to delete.
Options	collection	No	{}	Request behavior.
— Bulk Delete	boolean	No	false	Whether to use the bulk operation to delete the document.
— Refresh	options	No	false	Refresh behavior for the delete operation.
				Options: true, wait_for, false
— Routing	string	No	—	Target primary shard.

Document: Get

Parameter	Type	Required	Default	Description
Index ID	string	Yes	—	ID of the index containing the document to retrieve.
Document ID	string	Yes	—	ID of the document to retrieve.
Simplify	boolean	No	true	Whether to return a simplified version of the response with _source fields at the top level.
Options	collection	No	{}	Which parts of the document to return.
— Source Excludes	string	No	—	Comma-separated list of source fields to exclude from the response.
— Source Includes	string	No	—	Comma-separated list of source fields to include in the response.
— Stored Fields	boolean	No	false	Whether to retrieve fields stored in the index rather than _source.

Document: Get Many

Parameter	Type	Required	Default	Description
Index ID	string	Yes	—	ID of the index to search.
Return All	boolean	No	false	Whether to return all results or only up to a given limit.
Limit	number	No	50	Max number of results to return. (shown when Return All is false)
Simplify	boolean	No	true	Whether to return simplified documents with _source fields at the top level.
Options	collection	No	{}	Query, sorting and search behavior.
— Query	string	No	—	Elasticsearch Query DSL as a JSON string. Example: {“match_all”:{}}
— Sort	string	No	—	Comma-separated field:direction pairs. When set with returnAll=true, PIT pagination is used.
— Source Excludes	string	No	—	Comma-separated list of source fields to exclude.
— Source Includes	string	No	—	Comma-separated list of source fields to include.
— Allow No Indices	boolean	No	true	Whether to allow no matching indices.
— Allow Partial Search Results	boolean	No	true	Whether to return partial results on shard failures.
— Explain	boolean	No	false	Whether to return score computation detail.
— Ignore Unavailable	boolean	No	false	Whether to ignore missing or closed indices.
— Routing	string	No	—	Target primary shard.
— Search Type	options	No	query_then_fetch	How distributed term frequencies are calculated.
				Options: query_then_fetch, dfs_query_then_fetch
— Timeout	string	No	1m	Period to wait for active shards.
— Track Total Hits	number	No	10000	Number of hits to count accurately.
— Version	boolean	No	false	Whether to return document version as part of a hit.

Document: Update

Parameter	Type	Required	Default	Description
Index ID	string	Yes	—	ID of the index containing the document to update.
Document ID	string	Yes	—	ID of the document to update.
Data to Send	options	No	defineBelow	How to send data to Elasticsearch.
				Options: defineBelow, autoMapInputData
Inputs to Ignore	string	No	—	Comma-separated list of input properties to skip. (shown when Data to Send is autoMapInputData)
Fields to Send	fixedCollection	No	{}	The document fields to change, one row per field. (shown when Data to Send is defineBelow)
— Field Name	string	No	—	The document field name.
— Field Value	string	No	—	The value to store in that field.
Options	collection	No	{}	Request behavior.
— Bulk Update	boolean	No	false	Whether to use the bulk operation to update documents.
— Refresh	options	No	false	Refresh behavior after the operation.
				Options: true, wait_for, false
— Routing	string	No	—	Target primary shard.

Index: Create

Parameter	Type	Required	Default	Description
Index ID	string	Yes	—	ID (name) of the index to create.
Additional Fields	collection	No	{}	Optional settings applied when the index is created.
— Aliases	string	No	—	Index aliases as a JSON object.
— Mappings	string	No	—	Field mappings for the index as a JSON object.
— Settings	string	No	—	Configuration options for the index as a JSON object.
— Master Timeout	string	No	1m	Period to wait for a connection to the master node.
— Timeout	string	No	30s	Period to wait for a response.
— Wait for Active Shards	string	No	1	Number of shard copies that must be active before proceeding.

Index: Delete

Parameter	Type	Required	Default	Description
Index ID	string	Yes	—	ID of the index to delete.

Index: Get

Parameter	Type	Required	Default	Description
Index ID	string	Yes	—	ID of the index to retrieve.
Additional Fields	collection	No	{}	Optional settings for the lookup.
— Allow No Indices	boolean	No	true	Whether to allow no matching indices.
— Expand Wildcards	options	No	all	Type of index that wildcard expressions can match.
				Options: all, open, closed, hidden, none
— Flat Settings	boolean	No	false	Whether to return settings in flat format.
— Ignore Unavailable	boolean	No	false	Whether to ignore missing indices.
— Include Defaults	boolean	No	false	Whether to return all default settings.
— Local	boolean	No	false	Whether to retrieve information from the local node only.
— Master Timeout	string	No	1m	Period to wait for a connection to the master node.

Index: Get Many

Parameter	Type	Required	Default	Description
Return All	boolean	No	false	Whether to return all indices or only up to a limit.
Limit	number	No	50	Max number of indices to return. (shown when Return All is false)

All Operations

Parameter	Type	Required	Default	Description
Max Concurrency	number	No	5	Maximum number of items to process concurrently. Accepts 1–20.

Output Data

This node replaces the item JSON rather than merging into it — the output item carries the Elasticsearch response and nothing else. Binary data on the input item is forwarded.

Resource / Operation	Output
Document create	One item per input item, carrying Elasticsearch’s index response — _index, _id, result, _version, _shards.
Document get	One item per input item, carrying the document.
Document update	One item per input item, carrying Elasticsearch’s update response.
Document delete	One item per input item, carrying Elasticsearch’s delete response.
Document getAll	Fans out — one output item per hit. A search that matches nothing produces no output items at all, so guard the branch downstream.
Index create	One item per input item, carrying id (the index name) plus the acknowledgement fields Elasticsearch returned.
Index get	One item per input item, carrying id plus that index’s aliases, mappings and settings.
Index delete	One item per input item, carrying success: true.
Index getAll	Fans out — one item per index, each carrying a single indexId.

Simplify. With Simplify on (the default) each document is flattened to its _id plus the _source fields at the top level, so a hit reads like the document you stored:

{
  "_id": "MSQZ4okBLtVQ6-4lXPTd",
  "title": "Quarterly report",
  "status": "published"
}

Turn Simplify off to receive Elasticsearch’s raw hit objects, with _index, _score and the document body still nested under _source.

Bulk mode. Turning on Bulk Create, Bulk Update or Bulk Delete queues items and sends them to the _bulk endpoint in batches of 50 rather than one request per item. Each output item then carries that document’s own per-action result from the bulk response, and a document that the bulk response reports as failed is routed the same way as any other item error.

Reference results downstream by expression, e.g. {{ $json._id }} or {{ $json.indexId }}.

Example Configuration

Index a document with named fields:

{
  "type": "elasticsearch",
  "parameters": {
    "resource": "document",
    "operation": "create",
    "indexId": "articles",
    "dataToSend": "defineBelow",
    "fieldsUi": {
      "fieldValues": [
        { "fieldId": "title", "fieldValue": "{{ $json.title }}" },
        { "fieldId": "body", "fieldValue": "{{ $json.body }}" },
        { "fieldId": "status", "fieldValue": "published" }
      ]
    },
    "additionalFields": {
      "documentId": "{{ $json.slug }}"
    },
    "options": {
      "refresh": "wait_for"
    }
  }
}

Bulk-index every incoming item, mapping its properties straight to fields:

{
  "type": "elasticsearch",
  "parameters": {
    "resource": "document",
    "operation": "create",
    "indexId": "events",
    "dataToSend": "autoMapInputData",
    "inputsToIgnore": "_error,_metadata",
    "options": {
      "bulkOperation": true
    }
  }
}

Search with Query DSL, capped at 50 hits:

{
  "type": "elasticsearch",
  "parameters": {
    "resource": "document",
    "operation": "getAll",
    "indexId": "articles",
    "returnAll": false,
    "limit": 50,
    "simple": true,
    "options": {
      "query": "{\"match\":{\"status\":\"published\"}}",
      "_source_includes": "title,status,createdAt"
    }
  }
}

Page through an entire index — a Sort field with Return All on switches the node to point-in-time pagination, which is how you get past 10,000 hits:

{
  "type": "elasticsearch",
  "parameters": {
    "resource": "document",
    "operation": "getAll",
    "indexId": "events",
    "returnAll": true,
    "simple": true,
    "options": {
      "query": "{\"match_all\":{}}",
      "sort": "createdAt:asc",
      "track_total_hits": 100000
    }
  }
}

Read one document by ID:

{
  "type": "elasticsearch",
  "parameters": {
    "resource": "document",
    "operation": "get",
    "indexId": "articles",
    "documentId": "{{ $json.slug }}",
    "simple": true
  }
}

Partially update a document:

{
  "type": "elasticsearch",
  "parameters": {
    "resource": "document",
    "operation": "update",
    "indexId": "articles",
    "documentId": "{{ $json._id }}",
    "dataToSend": "defineBelow",
    "fieldsUi": {
      "fieldValues": [
        { "fieldId": "status", "fieldValue": "archived" }
      ]
    },
    "options": {
      "refresh": "true"
    }
  }
}

Delete a document:

{
  "type": "elasticsearch",
  "parameters": {
    "resource": "document",
    "operation": "delete",
    "indexId": "articles",
    "documentId": "{{ $json._id }}"
  }
}

Create an index with mappings and settings:

{
  "type": "elasticsearch",
  "parameters": {
    "resource": "index",
    "operation": "create",
    "indexId": "articles",
    "additionalFields": {
      "mappings": "{\"properties\":{\"title\":{\"type\":\"text\"},\"status\":{\"type\":\"keyword\"}}}",
      "settings": "{\"number_of_shards\":1,\"number_of_replicas\":1}",
      "wait_for_active_shards": "1"
    }
  }
}

List every index:

{
  "type": "elasticsearch",
  "parameters": {
    "resource": "index",
    "operation": "getAll",
    "returnAll": true
  }
}

Error Handling

Mode	Behavior
stop	Halts workflow on first error
continue	Skips failed items, passes successful ones through
errorPort	Routes failed items to Error output port

Tips

This tool interacts with Elasticsearch documents and indices. It is the right tool when you need to perform create, delete, get, getAll, or update operations on documents, or create, delete, get, or getAll operations on indices. It produces the results of these Elasticsearch API calls, including document content, index information, or operation confirmations.