Customizing Record Storage

The redefined Corteza record storage now enables you to customize the way records are stored under the hood. Customizing record storage enables you to easily integrate external data as if it was created through Corteza, offload datasets to dedicated databases, or larger datasets to specialized data centres*.

To store records into a different connection, you’ll need to create it first.

By default, Corteza stores all records (along with their values) into a single table, which should suffice for most basic use-cases. If your use case doesn’t require storing records in dedicated storage, feel free to skip this section.

The schema changes, suggested by Corteza, may not be the most optimal and may change through future releases. Prefer to make necessary database alterations manually if you prefer to have full control over what happens.

When you customize the record storage, in cases where the used database can not store records for this module (to exemplify; the table or column does not exist), Corteza suggests a set of changes you can make to properly store the data.

Annotated image

You can either resolve these alterations directly via Corteza using the suggested change by clicking on the Resolve or Automatically Resolve button.

Annotated image

Alternatively, you can do the changes directly in the database and dismiss the alteration by clicking on the Dismiss button.

Annotated image

Configuration

The record storage configuration options are available in the module edit page under the data store tab.

Annotated image

Connection

Basic connection parameters define what connection Corteza should use when storing the module’s records and where in the connection the data should be stored — such as what table or collection.

Changing the connection will reset the sensitivity level configuration on the module and fields if the sensitivity level is incompatible with the new connection.

Annotated image

Database connection

The database connection specifies what connection Corteza should use when handling the module’s records. By default, Corteza uses the primary connection.

Collection or database table name

The collection or database table name specifies where the module’s records are stored inside the connection.

If the identifier is omitted, the connection-defined identifier is used. If the connection doesn’t provide a default, the system default compose_recrd is used.

For some scenarios, Corteza can create the appropriate structures if they don’t already exist in the connection (such as a new table in an RDBMS database).

DevNote expand notes when tables can are automatically created.

You can use the following placeholders:
  • {{module}}: the {{module}} placeholder is replaced with the module’s handle or the ID if the handle is not defined.

  • {{namespace}}: the {{namespace}} placeholder is replaced with the namespace’s slug or the ID if the slug is not defined.

Module Field Mapping

The module field mapping and encoding parameters specify which module fields should be stored and how they should be stored. Refer to the field encoding codecs section for details on the available value encoding codecs.

Annotated image

Map module field

The map module field checkbox allows you to choose whether Corteza should store the module field’s values or not. If the checkbox is unchecked, Corteza will drop the values.

Generally, you’ll want to store all of the module fields. If a field is no longer needed, you should remove it from the module definition.

Annotated image

Codec

The encoding codec defines what coded Corteza should use when dealing with the field’s values. Refer to the field encoding codecs section for details.

Annotated image

Codec configuration

The codec configuration provides the required configuration parameters to the selected encoding codec. Refer to the field encoding codecs section for details.

Annotated image

System Fields

The system module field mapping and encoding parameters allow you to specify which module fields should be stored and how they should be stored. Refer to the field encoding codecs section for details on the available value encoding codecs.

System fields, for the most part, provide metadata about the record. If needed, you can omit specific system fields from being stored. Omitting system fields can come in handy when connecting to existing databases whose structure doesn’t define all supported system fields.

Annotated image

Map module field

The map module field checkbox allows you to choose whether Corteza should store the module field’s values or not. If the checkbox is unchecked, Corteza will drop the values.

Generally, you’ll want to store all of the system module fields, but you can omit the ones your custom schema does not support.

Omitting system fields like the id and the timestamps may lead to unexpected behaviour. We suggest you keep all of the system fields.

Annotated image

Codec

The encoding codec defines what coded Corteza should use when dealing with the field’s values. Refer to the field encoding codecs section for details.

Annotated image

Codec configuration

The codec configuration provides the required configuration parameters to the selected encoding codec. Refer to the field encoding codecs section for details.

Annotated image

Field Encoding Codecs

The field encoding codec defines how the record value for the corresponding field should be stored and retrieved later. The available encoding codecs are listed in the table below with an explanation, configuration, and use cases.

JSON

The JSON field codec encodes the record value into a JSON object. The location of the JSON object (relative to the connection’s storage, such as the table column) is defined by the identifier provided alongside the encoding codec definition.

This encoding strategy is used by default for user-defined module fields.

The encoding codec is useful when you wish to retrieve values from JSON objects.

Currently, Corteza only supports the following JSON structure (future versions plan to expand the flexibility):
{
  "identifier": [(1)
    ...(2)
  ]
}
1 The value identifier should match the module field’s name.
2 One or more values are stored for the record. Values are provided as an array to allow the field to go from a single to a multi-value field.

Alias

The alias field codec encodes the record value as a stand-alone value under a custom identifier. The location (relative to the connection’s storage, such as the table column) is defined by the identifier provided alongside the encoding codec definition.

The encoding codec is useful when you retrieve values from a dedicated location that can have a different identifier to the module field, such as a table column or collection attribute.

Column

The column field codec encodes the record value as a stand-alone value using the same identifier as the module field. The location (relative to the connection’s storage, such as the table column) is defined by the module field’s name.

This encoding strategy is used for system-defined module fields.

The encoding codec is useful when you wish to retrieve values from a dedicated location with the same identifier as the module field, such as a table column or collection attribute.

Troubleshooting

Module Issues

In case the module is miss configured or the underlying connection reports any errors, a list of identified issues is presented in the module edit screen. If there are any issues, a new issues tab is displayed.

Annotated image