Skip to main content

webhook.http

Trigger an HTTP request for every record.

Description​

A processor that sends an HTTP request to the specified URL, retries on error and saves the response body and, optionally, the response status.

A status code over 500 is regarded as an error and will cause the processor to retry the request. The processor will retry the request according to the backoff configuration.

Configuration parameters​

version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "webhook.http"
        settings:
          # Maximum number of retries for an individual record when backing off
          # following an error.
          # Type: float
          backoffRetry.count: "0"
          # The multiplying factor for each increment step.
          # Type: float
          backoffRetry.factor: "2"
          # The maximum waiting time before retrying.
          # Type: duration
          backoffRetry.max: "5s"
          # The minimum waiting time before retrying.
          # Type: duration
          backoffRetry.min: "100ms"
          # Headers to add to the request, use `headers.*` to specify the header
          # and its value (e.g. `headers.Authorization: "Bearer key"`).
          # Type: string
          headers.*: ""
          # Specifies the body that will be sent in the HTTP request. The field
          # accepts a Go [templates](https://pkg.go.dev/text/template) that's
          # evaluated using the
          # [opencdc.Record](https://pkg.go.dev/github.com/conduitio/conduit-commons/opencdc#Record)
          # as input. By default, the body is empty.
          # To send the whole record as JSON you can use `{{ toJson . }}`.
          # Type: string
          request.body: ""
          # Deprecated: use `headers.Content-Type` instead.
          # Type: string
          request.contentType: ""
          # Method is the HTTP request method to be used.
          # Type: string
          request.method: "GET"
          # URL is a Go template expression for the URL used in the HTTP
          # request, using Go [templates](https://pkg.go.dev/text/template). The
          # value provided to the template is
          # [opencdc.Record](https://pkg.go.dev/github.com/conduitio/conduit-commons/opencdc#Record),
          # so the template has access to all its fields (e.g. `.Position`,
          # `.Key`, `.Metadata`, and so on). We also inject all template
          # functions provided by [sprig](https://masterminds.github.io/sprig/)
          # to make it easier to write templates.
          # Type: string
          request.url: ""
          # Specifies in which field should the response body be saved.
          # For more information about the format, see [Referencing
          # fields](https://conduit.io/docs/using/processors/referencing-fields).
          # Type: string
          response.body: ".Payload.After"
          # Specifies in which field should the response status be saved. If no
          # value is set, then the response status will NOT be saved.
          # For more information about the format, see [Referencing
          # fields](https://conduit.io/docs/using/processors/referencing-fields).
          # Type: string
          response.status: ""
          # Whether to decode the record key using its corresponding schema from
          # the schema registry.
          # Type: bool
          sdk.schema.decode.key.enabled: "true"
          # Whether to decode the record payload using its corresponding schema
          # from the schema registry.
          # Type: bool
          sdk.schema.decode.payload.enabled: "true"
          # Whether to encode the record key using its corresponding schema from
          # the schema registry.
          # Type: bool
          sdk.schema.encode.key.enabled: "true"
          # Whether to encode the record payload using its corresponding schema
          # from the schema registry.
          # Type: bool
          sdk.schema.encode.payload.enabled: "true"

Examples​

Send a request to an HTTP server​

This example shows how to use the HTTP processor to send a record's .Payload.After field as a string to a dummy HTTP server that replies back with a greeting.

The record's .Payload.After is overwritten with the response. Additionally, the example shows how to set a request header and how to store the value of the HTTP response's code in the metadata field http_status.

Configuration parameters​

version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "webhook.http"
        settings:
          backoffRetry.count: "0"
          backoffRetry.factor: "2"
          backoffRetry.max: "5s"
          backoffRetry.min: "100ms"
          headers.content-type: "application/json"
          request.body: "{{ printf "%s" .Payload.After }}"
          request.method: "GET"
          request.url: "http://127.0.0.1:54321"
          response.body: ".Payload.After"
          response.status: ".Metadata["http_status"]"

Record difference​

Before
After
1
{
1
{
2
  "position": "cG9zLTE=",
2
  "position": "cG9zLTE=",
3
  "operation": "update",
3
  "operation": "update",
4
-
  "metadata": null,
4
+
  "metadata": {
5
+
    "http_status": "200"
6
+
  },
5
  "key": null,
7
  "key": null,
6
  "payload": {
8
  "payload": {
7
    "before": null,
9
    "before": null,
8
-
    "after": "world"
10
+
    "after": "hello, world"
9
  }
11
  }
10
}
12
}




Send a request to an HTTP server with a dynamic URL​


This example shows how to use the HTTP processor to use a record's .Payload.After.name field in the URL path,
send it to a dummy HTTP server, and get a greeting with the name back.


The response will be written under the record's .Payload.After.response.


Configuration parameters​


version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "webhook.http"
        settings:
          backoffRetry.count: "0"
          backoffRetry.factor: "2"
          backoffRetry.max: "5s"
          backoffRetry.min: "100ms"
          request.method: "GET"
          request.url: "http://127.0.0.1:54321/{{.Payload.After.name}}"
          response.body: ".Payload.After.response"


Record difference​


Before
After
1
{
1
{
2
  "position": "cG9zLTE=",
2
  "position": "cG9zLTE=",
3
  "operation": "create",
3
  "operation": "create",
4
  "metadata": null,
4
  "metadata": null,
5
  "key": null,
5
  "key": null,
6
  "payload": {
6
  "payload": {
7
    "before": null,
7
    "before": null,
8
    "after": {
8
    "after": {
9
-
      "name": "foo"
9
+
      "name": "foo",
10
+
      "response": "aGVsbG8sIGZvbyE="
10
    }
11
    }
11
  }
12
  }
12
}
13
}


scarf pixel conduit-site-docs-using-processors