Skip to main content

json.decode

Decodes a specific field from JSON raw data (string) to structured data.

Description​

The processor takes JSON raw data (string or []byte) from the target field, parses it as JSON structured data and stores the decoded structured data in the target field.

This processor is only applicable to fields under .Key, .Payload.Before and .Payload.After, as they can contain structured data.

Configuration parameters​

version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "json.decode"
        settings:
          # Field is a reference to the target field. Only fields that are under
          # `.Key` and `.Payload` can be decoded.
          # For more information about the format, see [Referencing
          # fields](https://conduit.io/docs/using/processors/referencing-fields).
          # Type: string
          field: ""
          # Whether to decode the record key using its corresponding schema from
          # the schema registry.
          # Type: bool
          sdk.schema.decode.key.enabled: "true"
          # Whether to decode the record payload using its corresponding schema
          # from the schema registry.
          # Type: bool
          sdk.schema.decode.payload.enabled: "true"
          # Whether to encode the record key using its corresponding schema from
          # the schema registry.
          # Type: bool
          sdk.schema.encode.key.enabled: "true"
          # Whether to encode the record payload using its corresponding schema
          # from the schema registry.
          # Type: bool
          sdk.schema.encode.payload.enabled: "true"

Examples​

Decode record key as JSON​

This example takes a record containing a raw JSON string in .Key and converts it into structured data.

Configuration parameters​

version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "json.decode"
        settings:
          field: ".Key"

Record difference​

Before
After
1
{
1
{
2
  "position": null,
2
  "position": null,
3
  "operation": "create",
3
  "operation": "create",
4
  "metadata": null,
4
  "metadata": null,
5
-
  "key": "{\"after\":{\"data\":4,\"id\":3}}",
5
+
  "key": {
6
+
    "after": {
7
+
      "data": 4,
8
+
      "id": 3
9
+
    }
10
+
  },
6
  "payload": {
11
  "payload": {
7
    "before": null,
12
    "before": null,
8
    "after": null
13
    "after": null
9
  }
14
  }
10
}
15
}




Decode nested field as JSON​


This example takes a record containing a raw JSON string in
.Payload.Before.foo and converts it into a map.


Configuration parameters​


version: 2.2
pipelines:
  - id: example
    status: running
    connectors:
      # define source and destination ...
    processors:
      - id: example
        plugin: "json.decode"
        settings:
          field: ".Payload.Before.foo"


Record difference​


Before
After
1
{
1
{
2
  "position": null,
2
  "position": null,
3
  "operation": "snapshot",
3
  "operation": "snapshot",
4
  "metadata": null,
4
  "metadata": null,
5
  "key": null,
5
  "key": null,
6
  "payload": {
6
  "payload": {
7
    "before": {
7
    "before": {
8
-
      "foo": "{\"before\":{\"data\":4,\"id\":3},\"baz\":\"bar\"}"
8
+
      "foo": {
9
+
        "baz": "bar",
10
+
        "before": {
11
+
          "data": 4,
12
+
          "id": 3
13
+
        }
14
+
      }
9
    },
15
    },
10
    "after": null
16
    "after": null
11
  }
17
  }
12
}
18
}


scarf pixel conduit-site-docs-using-processors