json.decode
Decodes a specific field from JSON raw data (string) to structured data.
Description
The processor takes JSON raw data (string
or []byte
)
from the target field, parses it as JSON structured data and stores the decoded
structured data in the target field.
This processor is only applicable to fields under .Key
, .Payload
.Before and
.Payload.After
, as they can contain structured data.
Configuration parameters
- YAML
- Table
version: 2.2
pipelines:
- id: example
status: running
connectors:
# define source and destination ...
processors:
- id: example
plugin: "json.decode"
settings:
# Field is a reference to the target field. Only fields that are under
# `.Key` and `.Payload` can be decoded.
# For more information about the format, see [Referencing
# fields](https://conduit.io/docs/using/processors/referencing-fields).
# Type: string
field: ""
# Whether to decode the record key using its corresponding schema from
# the schema registry.
# Type: bool
sdk.schema.decode.key.enabled: "true"
# Whether to decode the record payload using its corresponding schema
# from the schema registry.
# Type: bool
sdk.schema.decode.payload.enabled: "true"
# Whether to encode the record key using its corresponding schema from
# the schema registry.
# Type: bool
sdk.schema.encode.key.enabled: "true"
# Whether to encode the record payload using its corresponding schema
# from the schema registry.
# Type: bool
sdk.schema.encode.payload.enabled: "true"
Name | Type | Default | Description |
---|---|---|---|
field | string | null | Field is a reference to the target field. Only fields that are under
For more information about the format, see Referencing fields. |
sdk.schema.decode.key.enabled | bool | true | Whether to decode the record key using its corresponding schema from the schema registry. |
sdk.schema.decode.payload.enabled | bool | true | Whether to decode the record payload using its corresponding schema from the schema registry. |
sdk.schema.encode.key.enabled | bool | true | Whether to encode the record key using its corresponding schema from the schema registry. |
sdk.schema.encode.payload.enabled | bool | true | Whether to encode the record payload using its corresponding schema from the schema registry. |
Examples
Decode record key as JSON
This example takes a record containing a raw JSON string in
.Key
and converts it into structured data.
Configuration parameters
- YAML
- Table
version: 2.2
pipelines:
- id: example
status: running
connectors:
# define source and destination ...
processors:
- id: example
plugin: "json.decode"
settings:
field: ".Key"
Name | Value |
---|---|
field | .Key |
Record difference
Before | After | ||||
1 | { | 1 | { | ||
2 | "position": null, | 2 | "position": null, | ||
3 | "operation": "create", | 3 | "operation": "create", | ||
4 | "metadata": null, | 4 | "metadata": null, | ||
5 | - | "key": "{\"after\":{\"data\":4,\"id\":3}}", | 5 | + | "key": { |
6 | + | "after": { | |||
7 | + | "data": 4, | |||
8 | + | "id": 3 | |||
9 | + | } | |||
10 | + | }, | |||
6 | "payload": { | 11 | "payload": { | ||
7 | "before": null, | 12 | "before": null, | ||
8 | "after": null | 13 | "after": null | ||
9 | } | 14 | } | ||
10 | } | 15 | } |
Decode nested field as JSON
This example takes a record containing a raw JSON string in
.Payload.Before.foo
and converts it into a map.
Configuration parameters
- YAML
- Table
version: 2.2
pipelines:
- id: example
status: running
connectors:
# define source and destination ...
processors:
- id: example
plugin: "json.decode"
settings:
field: ".Payload.Before.foo"
Name | Value |
---|---|
field | .Payload.Before.foo |
Record difference
Before | After | ||||
1 | { | 1 | { | ||
2 | "position": null, | 2 | "position": null, | ||
3 | "operation": "snapshot", | 3 | "operation": "snapshot", | ||
4 | "metadata": null, | 4 | "metadata": null, | ||
5 | "key": null, | 5 | "key": null, | ||
6 | "payload": { | 6 | "payload": { | ||
7 | "before": { | 7 | "before": { | ||
8 | - | "foo": "{\"before\":{\"data\":4,\"id\":3},\"baz\":\"bar\"}" | 8 | + | "foo": { |
9 | + | "baz": "bar", | |||
10 | + | "before": { | |||
11 | + | "data": 4, | |||
12 | + | "id": 3 | |||
13 | + | } | |||
14 | + | } | |||
9 | }, | 15 | }, | ||
10 | "after": null | 16 | "after": null | ||
11 | } | 17 | } | ||
12 | } | 18 | } |