Skip to main content

json.encode

Encodes a specific field from structured data to JSON raw data (string).

Description

The processor takes data from the target field, encodes it into a JSON value and stores the encoded value in the target field.

This processor is only applicable to fields under .Key, .Payload.Before and .Payload.After, as they can contain structured data.

Configuration parameters

version: 2.2
pipelines:
- id: example
status: running
connectors:
# define source and destination ...
processors:
- id: example
plugin: "json.encode"
settings:
# Field is a reference to the target field. Only fields that are under
# `.Key` and `.Payload` can be encoded.
# For more information about the format, see [Referencing
# fields](https://conduit.io/docs/using/processors/referencing-fields).
# Type: string
field: ""
# Whether to decode the record key using its corresponding schema from
# the schema registry.
# Type: bool
sdk.schema.decode.key.enabled: "true"
# Whether to decode the record payload using its corresponding schema
# from the schema registry.
# Type: bool
sdk.schema.decode.payload.enabled: "true"
# Whether to encode the record key using its corresponding schema from
# the schema registry.
# Type: bool
sdk.schema.encode.key.enabled: "true"
# Whether to encode the record payload using its corresponding schema
# from the schema registry.
# Type: bool
sdk.schema.encode.payload.enabled: "true"

Examples

Encode nested field to JSON

This example takes a record containing a map in .Payload.Before.foo and converts it into a raw JSON string.

Configuration parameters

version: 2.2
pipelines:
- id: example
status: running
connectors:
# define source and destination ...
processors:
- id: example
plugin: "json.encode"
settings:
field: ".Payload.Before.foo"

Record difference

Before
After
1
{
1
{
2
  "position": null,
2
  "position": null,
3
  "operation": "snapshot",
3
  "operation": "snapshot",
4
  "metadata": null,
4
  "metadata": null,
5
  "key": null,
5
  "key": null,
6
  "payload": {
6
  "payload": {
7
    "before": {
7
    "before": {
8
-
      "foo": {
8
+
      "foo": "{\"baz\":\"bar\",\"before\":{\"data\":4,\"id\":3}}"
9
-
        "baz": "bar",
10
-
        "before": {
11
-
          "data": 4,
12
-
          "id": 3
13
-
        }
14
-
      }
15
    },
9
    },
16
    "after": null
10
    "after": null
17
  }
11
  }
18
}
12
}

Encode record key to JSON

This example takes a record containing structured data in .Key and converts it into a raw JSON string.

Configuration parameters

version: 2.2
pipelines:
- id: example
status: running
connectors:
# define source and destination ...
processors:
- id: example
plugin: "json.encode"
settings:
field: ".Key"

Record difference

Before
After
1
{
1
{
2
  "position": null,
2
  "position": null,
3
  "operation": "create",
3
  "operation": "create",
4
  "metadata": null,
4
  "metadata": null,
5
-
  "key": {
5
+
  "key": "{\"tables\":[\"table1,table2\"]}",
6
-
    "tables": [
7
-
      "table1,table2"
8
-
    ]
9
-
  },
10
  "payload": {
6
  "payload": {
11
    "before": null,
7
    "before": null,
12
    "after": null
8
    "after": null
13
  }
9
  }
14
}
10
}

scarf pixel conduit-site-docs-using-processors