Filters on Dripline Documentation

JS

Wed, 16 Sep 2020 22:38:02 +0200

LLM

Fri, 06 Mar 2026 00:00:00 +0000

LLM

This filter sends the received Message to a Large Language Model (LLM) and propagates the response. It uses any-llm-go to support multiple LLM providers through a unified interface.

Both the prompt and system_prompt parameters support templates, allowing you to dynamically compose prompts using the main field and any extra fields of the incoming Message.

Supported Providers

OpenAI, Anthropic, Ollama, DeepSeek, Groq, Mistral, Gemini, llama.cpp, Llamafile.

Parameters

Parameter	Type	Default	Description
model	STRING		(required) The model name to use (e.g. `"gpt-4"`, `"claude-3-opus-20240229"`, `"llama3"`)
prompt	STRING		(required) The user prompt sent to the LLM. Supports templates with `Message` fields
system_prompt	STRING	empty	An optional system prompt. Supports templates with `Message` fields
provider	STRING	“openai”	The LLM provider to use: `openai`, `anthropic`, `ollama`, `deepseek`, `groq`, `mistral`, `gemini`, `llamacpp`, `llamafile`
api_key	STRING	empty	API key for the provider (required for cloud providers like OpenAI, Anthropic, etc.)
api_url	STRING	empty	Custom base URL for the API endpoint (useful for proxies or self-hosted instances)
target	STRING	“main”	The field of the `Message` where the LLM response will be stored. Use `"main"` to replace the message content, or any other name to set an extra field
temperature	FLOAT	0.7	Sampling temperature for the model (higher values produce more random output)
max_tokens	INT	1024	Maximum number of tokens to generate in the response

... | llm(model="gpt-4", prompt="Summarize: {{ .main }}", api_key="sk-...", provider="openai") | ...

Output

The LLM response text is placed in the field specified by target (default: main). The following extra fields are set on the output Message:

Sat, 10 Feb 2024 23:02:24 +0200

This filter allows you to download files received from the Telegram feeder or send messages.
Note: it can be used only in a rule with the Telegram Feeder.

Parameters

The following parameters are required from this filter:

Parameter	Type	Default	Description
action	STRING	“send_message”	action to perform: “send_message”, “download_file”
to	STRING	""	used with action `send_message`, it has to contain the recipient of the message: username, @username, phone number with the country code (supports templates)
to_chatid	STRING	empty	the recipient of the message will be a chat
filename	STRING	""	specified the path of where to store the downloaded file: `msg_filename` in the extra will contain the name of the file (supports templates)
text	STRING	""	the text of the file (supports templates)

Each action can use different parameters:

XLS

Thu, 16 Nov 2023 20:08:45 +0200

XLS

This filter allows you to extract all the rows from a Excel file.
Based on the qax-os/excelize library.

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be the `main` or and extra field)
filename	STRING	empty	the filename of the XLS file to parse (supports templates)

... | xls(target="{{ .extra_field }}") | ...

The filename field override the target. They are mutually exclusive, so you can specify only one of them.

Output

The filter produces one Message for each row of the XLS file.

RateLimit

Tue, 31 Jan 2023 18:50:23 +0200

RateLimit

This filter allows you to set a rate limit on the messages that can go through it. So for example if we don’t want to limit the number of messages in a pipe to 5 messages per second we just to set the parameter rate to 5.

Parameters

Parameter	Type	Default	Description
rate	STRING	“0”	how many event per second you want to have as rate limiter

... | ratelimit(rate="5") | ...

Output

The filter will slow down the rate of the messages as specified on the parameter rate.

Json

Sun, 29 Jan 2023 18:57:01 +0200

Json

This filter is used to extract information from a JSON doc received through the Message.
It is possible to use XPath query for JSON specifying a selector to search in the doc and extract data (it uses jsonquery).

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be main or an extra field)
selector	STRING	""	the selector to find the data in the JSON

... | json(selector="id", target="doc") | ...

Output

The filter will generate one or more Messages. It is possible to use more than 1 time this filter.
The field fulltext will contain the original target string.

Html

Tue, 19 Jul 2022 23:38:02 +0200

Html

This filter is used to extract information from an HTML page received through the Message.
Like jQuery for JS, we can set a selector to find in the page, extract text from the tags and the html content (we are using goquery library under the hood).

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be main or an extra field)
selector	STRING	""	the selector to find in the HTML page
get	STRING	“html”	what do we want to retrieve on the tags found in the selected one: `html`, `text`, `attr`
attr	STRING	""	if get is `attr` you can define what attr name it should extract

... | html(selector=".link", get="attr", attr="href") | ...

Output

The filter will generate one or more Messages. It is possible to use more than 1 time this filter.
The field fulltext will contain the original target string.

Elasticsearch

Sat, 12 Mar 2022 15:40:23 +0100

ElasticSearch

This filter writes on ElasticSearch all the message it receives as input and return as output the document ID.
You can also specify what message’s field it should use as input.

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for filter’s output (it could be main or an extra field)

... | elasticsearch(target="input_field") | ...

Output

The main field will contain the docID returned by ElasticSearch.

File

Sat, 12 Mar 2022 15:33:42 +0100

File

This filter get as input the path of a local file, read it and return the content back to the pipeline.

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for filter’s output (it could be main or an extra field)

... | file(target="file_content") | ...

Output

The output will contain the content of the read file. Using the target parameter you can specify the output field’s name.

Striptag

Thu, 01 Jul 2021 15:35:53 +0200

StripTag

This filter is used to remove all the HTML tags from a string.

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be main or an extra field)

... | striptag(target="main") | ...

Output

The main of the output Message will have the text extracted from the target config, stripped by all the HTML tags.

Slack

Mon, 12 Oct 2020 19:38:02 +0200

Slack

This filter allows you to send files, messages and download files from Slack.
It can be used alone or with the Slack Feeder to create a simple Slack bot for events API.

Parameters

The following parameters are required from this filter:

Parameter	Type	Default	Description
action	STRING	“send_message”	action to perform: “send_message”, “send_file”, “download_file”, “user_info”
token	STRING	""	Slack bot Token

Each action can use different parameters:

action = send_message

Parameter	Type	Default	Description
to	STRING	""	channel ID or User ID that should receive the message (supports templates)
target	STRING	“main”	if `text` is not used you can choose which field of `Message` to use as text
text	STRING	""	you can define the text of the message to send (supports templates)
blocks	BOOL	false	if true you can use the Slack template blocks (block builder)

action = send_file

Parameter	Type	Default	Description
to	STRING	""	channel ID or User ID that should receive the message (supports templates)
target	STRING	“main”	if `filename` is not specified you can choose which field of `Message` has to be used as file content to send
filename	STRING	""	path of the file to send (supports templates)

action = download_file

Parameter	Type	Default	Description
url	STRING	""	it should contain the Slack private url for the file download (supports templates)
target	STRING	“urlprivate”	if `url` is not specified you can choose which field of `Message` contains the Slack private url
filename	STRING	""	path of where to save the downloaded file. If not specified the file content will be inserted in the `main` field (supports templates)

action = user_info

Parameter	Type	Default	Description
target	STRING	“user”	this field has to contain the USERID

In the output Message you can find all the user information returned by Slack:

Cache

Wed, 16 Sep 2020 22:38:02 +0200

Cache

This filter introduces a simple cache mechanism in the rule. It is a TTL based cache and it can have a local visibility (cache is only visible to the current filter) or a global visibility (cache shared across ALL the rules).
If the target of the Message as input has been cached before, and his TTL is not expired, it will be dropped and not propagated to the next filter.
Otherwise if the target of the Message is new to the cache, it is inserted in it and propagated to the next filter.

Changed

Wed, 16 Sep 2020 22:38:02 +0200

Changed

This filter is similar to the cache. It can only stop the propagation of the Message across the lane, but only if the target of the received Message is different from the previous one.

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be main or and extra field)

... | changed(target="original_author") | ...

Output

The output is not being changed. This filter can only stop or not the propagation of the Message.

Echo

Wed, 16 Sep 2020 22:38:02 +0200

Echo

This filter prints the Message on the logs. It is mostly used to debug the rules.

Parameters

Parameter	Type	Default	Description
extra	BOOL	“false”	print also all the extra fields

... | echo(extra="false") | ...

Output

The output is not being changed. This filter prints the received Message and send it to the next filter in the rule.

Examples

Soon…

Format

Wed, 16 Sep 2020 22:38:02 +0200

Format

This filter is used to format the received Message. It is based on
Go templates and it can load templates from the template_path
directory specified in the configuration file. All Sprig functions are available.

Parameters

Parameter	Type	Default	Description
type	STRING	“text”	specify the type of template to use : `"text"` or `"html"`
template	STRING	empty	a template could be specified directly here, instead of load it from file (supports templates)
file	STRING	empty	load the template from file (supports templates)
target	STRING	“main”	the field of the Message that should be used for the filter (it could be main or an extra field)

In the template is allowed to use all the fields of the received Message: main or extra.

Hash

Wed, 16 Sep 2020 22:38:02 +0200

Hash

This filter is used to search or extract hashes from a Message.
Supported types of hashes are:

MD5
SHA1
SHA256
SHA512

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be main or an extra field)
extract	BOOL	“false”	if `"true"` the main field of the output Message will be the extracted hash
md5	BOOL	“true”	if `"false"` md5 hashes will be ignored
sha1	BOOL	“true”	if `"false"` sha1 hashes will be ignored
sha256	BOOL	“true”	if `"false"` sha256 hashes will be ignored
sha512	BOOL	“true”	if `"false"` sha512 hashes will be ignored

... | hash(target="description", extract="true") | ...

Output

If the extract parameter is “false”, the received Message will be propagated only if at least one hash is found in it.
Otherwise, if extract is “true” and the Message contains one or more hashes, the main field of the propagated Message will contain only the extracted hash.

Http

Wed, 16 Sep 2020 22:38:02 +0200

HTTP

This filter allows you to send HTTP requests. When a Message arrives to the filter, we can decide if use the main field of the Message as URL on the request, or its content for the HTTP data.
This behaviour can be handled with the parameters.

Parameters

Parameter	Type	Default	Description
url	STRING	empty	URL of the web page. It is possible use templates to use fields of the `Message`
download_to	STRING	empty	path of where to download the file. It is possible use templates to use fields of the `Message`
text_only	BOOL	“false”	if “true” it removes all the tags from the body response
method	STRING	“GET”	HTTP method to use on the request
headers	JSON	empty	Headers to use in the request
data	JSON	empty	POST fields to send with the request (it’s not possible to use in combination with `rawData`)
rawData	STRING	empty	raw body of the request (it’s not possible to use in combination with `data`)
status	STRING	empty	the filter will propagate the Message only if the returned status has the specified value
cookies	STRING	empty	Path of the JSON file containing the cookies to use

... | http(url="{{ .main }}", cookies="exported.json", headers="{\"Content-type\": \"application/json\"}") | ...

Output

If the request was successful, the output Message will have the main field set to the HTTP body response. If the status is set, and the response http status is different from it, the Message will be dropped.

Mail

Wed, 16 Sep 2020 22:38:02 +0200

Mail

This filter allows you to send an e-mail. When a Message arrives to the filter, we can decide if use the main field of the Message as URL on the request, or its content for the HTTP data.
This behaviour can be handled with the parameters.

Parameters

Parameter	Type	Default	Description
body	STRING	empty	the body of the e-mail (supports templates)
username	STRING	empty	username for the host authentication
password	STRING	empty	password for the host authentication
host	STRING	empty	host server used to send the e-mail
port	STRING	empty	port of the host server
fromAddr	STRING	empty	source e-mail address
fromName	STRING	empty	source name address
to	STRING	empty	destination e-mail address (supports multi-destination, comma separated)
subject	STRING	empty	subject field of the e-mail to send
use_auth	BOOL	“false”	if “true” the sendmail server will receive the credentials specified in `username` and `password` fields

... | http(url="{{ .main }}", cookies="exported.json", headers="{\"Content-type\": \"application/json\"}") | ...

Every default’s value can be set in the configuration, creating a section with the name of theFilter/Feeder.
more info

Output

The input Message is always propagated to the next filter without changes.

MIME type

Wed, 16 Sep 2020 22:38:02 +0200

MIME type

This filter allows you to detect the MIME type of a file and its extension.
Based on the gabriel-vasile/mimetype library.

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be the `main` or and extra field)
filename	STRING	empty	the filename of the file to detect (supports templates)

... | mime(target="{{ .extra_field }}") | ...

The filename field override the target. They are mutually exclusive, so you can specify only one of them.

Output

The propagated Message will contain the mimetype’s string in the main field and the extension in the extra field mimetype_ext.

Number

Wed, 16 Sep 2020 22:38:02 +0200

Number

This filter allows you to treat a string from the input Message as numeric value and apply some operator on it (for example, check if this field contains a number greater than X).

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be main or and extra field)
op	STRING	""	Compare operator to use for the numeric value (">", “>=”, “<”, “<=”, “!=”, “==”)
value	STRING	""	It has to be a numeric value and it is the number to use for the comparison

... | number(target="num_field", op=">=", value="44") | ...

Output

If the comparison is verified the received Message will be propagated.

Override

Wed, 16 Sep 2020 22:38:02 +0200

Override

This filter allows you to change a field of a Message, before sending it to the next filter. Templates with Sprig functions can be used.

Parameters

Parameter	Type	Default	Description
name	STRING	empty	name of the field to change (supports templates)
value	STRING	empty	new value to assign to the Message’s field specified (supports templates)

... | override(name="description", value="{{ .title }}") | ...

Output

The propagated Message will be identical to the original, with only the specified field changed.

Pdf

Wed, 16 Sep 2020 22:38:02 +0200

Pdf

This filter allows you to extract plain text from a PDF file.
Based on the ledongthuc/pdf library.

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be the `main` or and extra field)
filename	STRING	empty	the filename of the PDF file to parse (supports templates)

... | pdf(target="{{ .extra_field }}") | ...

The filename field override the target. They are mutually exclusive, so you can specify only one of them.

Output

The propagated Message contains the plain text of the input PDF file (fulltext will be set to the file name received as input).

Random

Wed, 16 Sep 2020 22:38:02 +0200

Random

This filter is used to inject an extra field with a random number in the propagated Message.

Parameters

Parameter	Type	Default	Description
output	STRING	“main”	the field of the propagated `Message` that will contain the random number
min	STRING	“0”	the min value of the extracted number is `min`
max	STRING	“999999”	the max value of the extracted number is `max`

... | random(output="random_field", min="9", max="90") | ...

Output

The output Message will be equal to the input, but it will also include the new random field.

System

Wed, 16 Sep 2020 22:38:02 +0200

System

This filter allows you to exec a command on the host machine. The received Message can be used to create the command to launch.
It supports templates with Sprig functions.

Parameters

Parameter	Type	Default	Description
cmd	STRING	empty	command line to exec for each received Message (supports templates)

... | system(cmd="echo '{{ .author }} wrote {{ .main }}' >> logs.txt") | ...

Output

The propagated Message will contain the output of the command if it is provided, and it is not failed.

Text

Wed, 16 Sep 2020 22:38:02 +0200

Text

This filter searches or extracts strings from the received Message. It can be used with a regular expression or a simple string.
If the string is found, the condition is matched and the Message is propagated to the next filter.

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be main or and extra field)
regexp	BOOL	false	the pattern field is a regular expression
extract	BOOL	“false”	if “true” the `main` field of the propagated Message will contain the extracted string (it can be used only if `regexp` parameter is set true)
pattern	STRING	empty	specifies the pattern that should be matched on the Message to check the condition

... | text(target="description", pattern="(#[^\\s]+)", regexp="true", extract="true") | ...

Output

If the extract parameter is “false”, the received Message will be propagated only if the specified pattern is matched in the target field of the Message.
Otherwise if extract is “true” (only regexp can be used in this case), and one or more strings matches with the pattern, the main field of the propagated Message will contain only the matched string.

Url

Wed, 16 Sep 2020 22:38:02 +0200

Url

This filter is used to search or extract URLs from a Message.
Currently supported types of URLs are:

http/s
ftp

Parameters

Parameter	Type	Default	Description
target	STRING	“main”	the field of the Message that should be used for the filter (it could be main or an extra field)
http	BOOL	“true”	if “false”, `http` scheme urls are ignored
https	BOOL	“true”	if “false”, `https` scheme urls are ignored
ftp	BOOL	“true”	if “false”, `ftp` scheme urls are ignored
extract	BOOL	“true”	if “true”, the `main` field of the propagated Message will contain the found URL

... | url(extract="true") | ...

Output

If the extract parameter is “false”, the received Message will be propagated only if at least one URL is found in it.
Otherwise, if extract is “true” and the Message contains one or more URLs, the main field of the propagated Message will contain only the extracted URLs.