Skip to main content

json.encode

Encodes a specific field from structured data to JSON raw data (string).

Description​

The processor takes data from the target field, encodes it into a JSON value and stores the encoded value in the target field.

This processor is only applicable to fields under .Key, .Payload.Before and .Payload.After, as they can contain structured data.

Configuration parameters​

version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "json.encode"
        settings:
          # Field is a reference to the target field. Only fields that are under
          # `.Key` and `.Payload` can be encoded.
          # For more information about the format, see [Referencing
          # fields](https://conduit.io/docs/using/processors/referencing-fields).
          # Type: string
          field: ""
          # Whether to decode the record key using its corresponding schema from
          # the schema registry.
          # Type: bool
          sdk.schema.decode.key.enabled: "true"
          # Whether to decode the record payload using its corresponding schema
          # from the schema registry.
          # Type: bool
          sdk.schema.decode.payload.enabled: "true"
          # Whether to encode the record key using its corresponding schema from
          # the schema registry.
          # Type: bool
          sdk.schema.encode.key.enabled: "true"
          # Whether to encode the record payload using its corresponding schema
          # from the schema registry.
          # Type: bool
          sdk.schema.encode.payload.enabled: "true"

Examples​

Encode nested field to JSON​

This example takes a record containing a map in .Payload.Before.foo and converts it into a raw JSON string.

Configuration parameters​

version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "json.encode"
        settings:
          field: ".Payload.Before.foo"

Record difference​

Before
After
1
{
1
{
2
  "position": null,
2
  "position": null,
3
  "operation": "snapshot",
3
  "operation": "snapshot",
4
  "metadata": null,
4
  "metadata": null,
5
  "key": null,
5
  "key": null,
6
  "payload": {
6
  "payload": {
7
    "before": {
7
    "before": {
8
-
      "foo": {
8
+
      "foo": "{\"baz\":\"bar\",\"before\":{\"data\":4,\"id\":3}}"
9
-
        "baz": "bar",
10
-
        "before": {
11
-
          "data": 4,
12
-
          "id": 3
13
-
        }
14
-
      }
15
    },
9
    },
16
    "after": null
10
    "after": null
17
  }
11
  }
18
}
12
}




Encode record key to JSON​


This example takes a record containing structured data in
.Key and converts it into a raw JSON string.


Configuration parameters​


version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "json.encode"
        settings:
          field: ".Key"


Record difference​


Before
After
1
{
1
{
2
  "position": null,
2
  "position": null,
3
  "operation": "create",
3
  "operation": "create",
4
  "metadata": null,
4
  "metadata": null,
5
-
  "key": {
5
+
  "key": "{\"tables\":[\"table1,table2\"]}",
6
-
    "tables": [
7
-
      "table1,table2"
8
-
    ]
9
-
  },
10
  "payload": {
6
  "payload": {
11
    "before": null,
7
    "before": null,
12
    "after": null
8
    "after": null
13
  }
9
  }
14
}
10
}


scarf pixel conduit-site-docs-using-processors