You are reading the documentation for an outdated Corteza release. 2024.9 is the latest stable Corteza release.

Syncing structures

The process of determining what structures on the origin node, is exposed to it’s destination node. The origin node has full control over what data the destination node is allowed to access. This can be as simple as allowing access to only specific modules; and as complex as allowing access to only specific module fields.

The two nodes must be paired prior to this. See Node pairing.

Exposing structures

Firstly, in order to perform any data sharing, the origin node must define what structures (in our case modules and fields) the destination node can access. This can be performed via the Corteza Low-Code administration panel, or directly via the API.

Besides the module itself, the origin node must also specify what fields the destination node is allowed to access.

API

Expose module

Used variables
# Base url for the federation api
$BASE_URL

# JWT of the user
$JWT

# Node id of the destination node
$NODE_ID

# Federation module id
$MODULE_ID
Example request
curl -X PUT "$BASE_URL/federation/nodes/$NODE_ID/modules/$MODULE_ID" \
  -H "Authorization: Bearer $JWT"
  -H "Content-Type: application/json" \
  --data "[{
      \"name\":\"LinkedIn\",
      \"label\":\"LinkedIn Url\",
      \"kind\":\"Url\",
      \"is_multi\":0
    }]";
Example response
{
    "response": {
        "moduleID": "122709113267335170",
        "handle": "Account",
        "name": "Account",
        "createdAt": "2019-12-18T17:45:15Z",
        "updatedAt": "2020-05-26T13:29:36Z",
        "fields": [
            {
                "name": "LinkedIn",
                "label": "LinkedIn Url",
                "kind": "Url",
                "is_multi": 0
            }
        ]
    }
}

See exposed module

Used variables
# Base url for the federation api
$BASE_URL

# JWT of the user
$JWT

# Node id of the destination node
$NODE_ID
Example request
curl -X GET "$BASE_URL/federation/nodes/$NODE_ID/modules/$MODULE_ID/exposed" \
  -H "Authorization: Bearer $JWT";
Example response
{
    "response": {
        "moduleID": "122709113267335170",
        "handle": "Account",
        "name": "Account",
        "createdAt": "2019-12-18T17:45:15Z",
        "updatedAt": "2020-05-26T13:29:36Z",
        "fields": [
            {
                "kind": "Url",
                "name": "LinkedIn",
                "label": "LinkedIn",
                "isMulti": false,
            },
            {
                "kind": "String",
                "name": "Phone",
                "label": "Phone",
                "isMulti": false,
            }
        ]
    }
}

Remove exposed module

Used variables
# Base url for the federation api
$BASE_URL

# JWT of the user
$JWT

# Node id of the destination node (?exposed) or the origin node (?shared)
$NODE_ID

# Federation module id
$MODULE_ID
Example request
curl -X DELETE "$BASE_URL/federation/nodes/$NODE_ID/modules/$MODULE_ID" \
  -H "authorization: Bearer $JWT";
Example response

Change exposed module fields

Used variables
# Base url for the federation api
$BASE_URL

# JWT of the user
$JWT

# Node id of the destination node
$NODE_ID

# Federation module id
$MODULE_ID
Example request
curl -X PUT "$BASE_URL/federation/nodes/$NODE_ID/modules/$MODULE_ID/exposed" \
  -H "Authorization: Bearer $JWT"
  -H "Content-Type: application/json" \
  --data "[{
      \"name\":\"LinkedIn\",
      \"label\":\"LinkedIn\",
      \"kind\":\"Url\",
      \"is_multi\":1
    }, {
      \"name\":\"Phone\",
      \"label\":\"Phone\",
      \"kind\":\"String\",
      \"is_multi\":0
    }]";
Example response
{
    "response": {
        "moduleID": "122709113267335170",
        "handle": "Account",
        "name": "Account",
        "createdAt": "2019-12-18T17:45:15Z",
        "updatedAt": "2020-05-26T13:29:36Z",
        "fields": [
            {
                "kind": "Url",
                "name": "LinkedIn",
                "label": "LinkedIn",
                "isMulti": 0,
            },
            {
                "kind": "String",
                "name": "Phone",
                "label": "Phone",
                "isMulti": 0,
            }
        ]
    }
}

Store

Exposed modules

Table 1. federation_module_exposed
Column Type Description

ID

uint64

federation module id

rel_node

uint64

node id (destination node id - who are we sharing to)

rel_compose_module

uint64

module id on source node

fields

json

list of fields

Syncing structures

When the two nodes are paired and the origin node has exposed some structures, the actual structure sync can occur.

Outline of structure sync process on the origin node.
Figure 1. Outline of structure sync process on the origin node.
Outline of structure sync process on the destination node.
Figure 2. Outline of structure sync process on the destination node.
Destination node requests changed structures

Origin node provides a set of endpoints that the destination node can use to fetch shared structures.

$TOKEN_B is the token that was generated during the handshake and is used to authenticate the user on the Origin node (the one who shares the data) by the Destination node.
Used variables
# Base url for the federation api
$BASE_URL

# JWT of the user
$JWT

# Node id of the destination node
$NODE_ID

# Node B auth token
$TOKEN_B
Example request
curl -X GET "$BASE_URL/federation/exposed/modules" \
  -H "Authorization: Bearer $TOKEN_B";
Example response
{
    "response": {
        "filter": {
            "query": "after=1600109447",
            "page": 1,
            "perPage": 20,
            "count": 97,
        },
        "set": [
            {
                "type": "GET",
                "rel": "Account",
                "href": "$BASE_URL/federation/exposed/modules/$MODULE_ID?after=1600109447"
            },
            {
                "type": "GET",
                "rel": "Contact",
                "href": "$BASE_URL/federation/exposed/modules/$MODULE_ID?after=1600109447"
            }
        ]
    }
}
Destination node fetches updated structures

The destination node requests structure definitions based on the above provided set of endpoints. The origin node provides the structure definition with respect to the fields that the destination node is allowed to access.

$TOKEN_B is the token that was generated during the handshake and is used to authenticate the user on the Origin node (the one who shares the data) by the Destination node.
Used variables
# Base url for the federation api
$BASE_URL

# JWT of the user
$JWT

# Node id of the origin node
$NODE_ID

# Node B auth token
$TOKEN_B
Example request
curl -X GET "$BASE_URL/federation/exposed/modules/$MODULE_ID" \
  -H "Authorization: Bearer $TOKEN_B";
Example response
{
    "response": {
        "moduleID": "122709113267335170",
        "handle": "Account",
        "name": "Account",
        "createdAt": "2019-12-18T17:45:15Z",
        "updatedAt": "2020-05-26T13:29:36Z",
        "fields": [
            {
                "kind": "Url",
                "name": "LinkedIn",
                "label": "LinkedIn",
                "isMulti": false,
            },
            {
                "kind": "String",
                "name": "Phone",
                "label": "Phone",
                "isMulti": false,
            }
        ]
    }
}
Destination node updates its store

Once the structure sync is finished, the destination node writes the structures to the store and updates the nodes status.

This step doesn’t create any system resources, such as records, on the destination node. This is performed later in the data sync.

Store

Shared modules

Table 2. federation_module_shared
Column Type Description

ID

uint64

federation module id

handle

varchar(200)

Module handle

name

varchar(64)

Module name

rel_node

uint64

node id (source node id - who is sharing with us)

xref_module

uint64

federation module id on source node (id in federation_module_exposed)

fields

json

list of fields

Field mapping

Field mapping allows the destination node to determine what fields from the origin node should map into what fields on the destination node. This allows some flexibility when it comes down to datamodel definitions on both origin and destination nodes. This can be performed via the Corteza Low-Code administration panel, or directly via the API.

API

Set field mapping for a module

Used variables
# Base url for the federation api
$BASE_URL

# JWT of the user
$JWT

# Node id of the destination node
$NODE_ID

# Federation module id
$MODULE_ID
Example request
curl -X PUT "$BASE_URL/federation/nodes/$NODE_ID/modules/$MODULE_ID/mapped" \
  -H "Authorization: Bearer $JWT"
  -H "Content-Type: application/json" \
  --data "[{
      \"origin\":{
          \"name\":\"LinkedIn\",
          \"kind\":\"Url\",
          \"is_multi\":0
        },
        \"destination\":{
            \"name\":\"Social\",
            \"kind\":\"String\",
            \"is_multi\":0
        }
    }]";
Example response
{
    "response": {
        "moduleID": "122709113267335170",
        "handle": "Account",
        "name": "Account",
        "createdAt": "2019-12-18T17:45:15Z",
        "updatedAt": "2020-05-26T13:29:36Z",
        "mapping": [
            {
                "origin": {
                    "name": "LinkedIn",
                    "kind": "Url",
                    "is_multi": 0
                },
                "destination": {
                    "name": "Social",
                    "kind": "String",
                    "is_multi": 0
                }
            }
        ]
    }
}

Get field mapping for a module

Used variables
# Base url for the federation api
$BASE_URL

# JWT of the user
$JWT

# Node id of the destination node
$NODE_ID

# Federation module id
$MODULE_ID
Example request
curl -X GET "$BASE_URL/federation/nodes/$NODE_ID/modules/$MODULE_ID/mapped" \
  -H "Authorization: Bearer $JWT";
Example response
{
    "response": {
        "moduleID": "122709113267335170",
        "handle": "Account",
        "name": "Account",
        "createdAt": "2019-12-18T17:45:15Z",
        "updatedAt": "2020-05-26T13:29:36Z",
        "mapping": [
            {
                "origin": {
                    "name": "LinkedIn",
                    "kind": "Url",
                    "is_multi": 0
                },
                "destination": {
                    "name": "Social",
                    "kind": "String",
                    "is_multi": 0
                }
            }
        ]
    }
}

Remove field mapping for a module

Used variables
# Base url for the federation api
$BASE_URL

# JWT of the user
$JWT

# Node id of the destination node (?exposed) or the origin node (?shared)
$NODE_ID

# Federation module id
$MODULE_ID
Example request
curl -X DELETE "$BASE_URL/federation/nodes/$NODE_ID/modules/$MODULE_ID" \
  -H "authorization: Bearer $JWT";
Example response

Store

Module mapping

Table 3. federation_module_mapping
Column Type Description

federation_module_id

uint64

id from federation_module_exposed

compose_module_id

uint64

existing module

field_mapping

json

json field mappings, ex: [{ source: 'node_A_module_field_7', dest: 'node_B_module_field_2', transform: 'string' }]

Example

Phase I - on Origin node

First phase is exposing the desired modules for a specific node (to the Destination), so the structure mapping on the Destination and then the data sync can be done.

compose_module

id handle name

161250629010849793

Account

Account

compose_module_field

id kind name label is_multi

161250629061509121

String

Phone

Phone

0

161250629027758081

Url

LinkedIn

LinkedIn

0

161250629044666369

String

Description

Description

0

federation_node

id name

1

Origin server

2

Destination server

federation_module_exposed

id rel_node rel_compose_module field_mapping

11

2

161250629010849793

[{"name":"Phone","kind":"String","is_multi":0}]

Phase II - on Destination node

There are 2 phases in the phase II. First the module info from Origin is saved. After that we can do the mapping. The modules on the Destination need to be created beforehand.

compose_module

id handle name

261250629010849755

Account_federated

Account (federated from Origin)

compose_module_field

id kind name label is_multi

161250629061509121

String

Mobile

Mobile

0

161250629044666369

String

Desc

Description

0

federation_node

id name

1

Our server

2

Misc server (some other server)

3

Origin server in this example (from phase I)

1. Fetch and save the module info

federation_module_shared

id handle name rel_node xref_module field_mapping

22

Account

Account

3

11

[{"name":"Phone","kind":"String","is_multi":0}]

2. Mapping finished, modules created

The sharing of modules info from Origin is added, that is enough information for us to handle mapping from UI. We can now pick the fields from the field_mapping column that we need and store it into the mapping table.

federation_module_mapping

federation_module_id compose_module_id fields

22

261250629010849755

[{"origin":{"name":"Phone","kind":"String","is_multi":0},"destination":{"name":"Mobile","kind":"String","is_multi":0}}]