You are reading the documentation for an outdated Corteza release. 2023.9 is the latest stable Corteza release.

Source Decoder

The source decoding layer takes in different data formats (such as json and yaml) and outputs a set of resource structures.

The source decoder is usually the first layer but we can also omit it if we construct resource structures manually.

The generated resources are then used in data shaping, dependency graph, and the resource encoder.

Table 1. A list of predefined decoders:

YAML decoder

The YAML decoder knows how to work with files that are in the YAML format.

All of the Corteza provisionings are in the YAML format, so all of the system resources are supported here. Take a look at the CRM configuration for an example.

CSV decoder

The CSV decoder knows how to work with files that are in the CSV format.

We consider CSV data as unstructured as we are not able to extract most of the contextual information required by system resources.

The CSV decoder provides a generic ResourceDataset structure, that we use in data shaping to construct the actual resource.

JSON decoder

The JSON decoder knows how to work with files that are in the JSON format.

Currently, we only really support JSONL (lines) format to import records.

We consider JSONL data as unstructured as we are not able to extract most of the contextual information required by system resources.

The JSON decoder provides a generic ResourceDataset structure, that we use in data shaping to construct the actual resource.

directory decoder

The directory decoder knows how to work with a series of files nested under an arbitrary file structure.

The directory decoder attempts to determine the source type, and then uses the pre-defined decoders to do the actual decoding.