Corteza Architecture overview

Mayor component or services in Corteza
  • frontend applications (WebApps)

  • backend API (Server)

  • automation script execution engine (Corredor)

Auxiliary services
  • log storage

  • error tracking

  • email sending

  • object storage

  • database

Frontend applications
  • Auth handles authentication flows, sign-up, login, password recovery, …​

  • Admin is used for instance administration, settings, user overview, general permission management

  • Low Code is a low-code development platform and allows creation of record-absed business applications

  • Messaging collaboration platform that allows teams to collaborate more efficiently and communicate safely with other organizations or customers

  • One is a unified workspace to access and run third-party web and Corteza applications. Centralized access management from a single console enables administrative control over who can see or access applications

Technical Requirements

Corteza is a server application that can be accessed through a browser on a workstation, laptop or mobile device.

For testing and exploring Corteza you do not need an external, externally accessible software. Corteza can deploy on a local machine.

Client software

Table 1. Supported web browsers and versions

Chrome

v78+

Firefox

v68+

Safari

v12+

Edge

v44+

iOS

iPhone 5s devices and later with iOS 12+

Android

Android 7+

Server

Software

Server Operating System

Any Docker-compatible operating system (Linux-based OS is recommended, preferably Ubuntu LTS)

HTTP server

Web application and API should be placed behind a firewall and/or a HTTP reverse proxy. The responsibility of this proxy is to forward traffic to Corteza services and do SSL termination. Most of the popular HTTP servers have this capability.

Corteza provides configuration guide for Nginx Reverse Proxy and for LetsEncrypt companion container for nginx-prox.

Nginx is not required, you can use HTTP server you are familiar with. Use provided examples for Nginx as a reference on how to configure it.

Docker

Minimum version for Docker Engine is 1.12.x

Database Software

Percona Server for MySQL 8.x (on host, managed or provided by Corteza through Docker image)

Corteza supports Docker and Docker Compose based setup and deployment. You should familiarize yourself with both Docker and Docker Compose.

Hardware

Number of Users

Processor

Memory

1 - 500

1 vCPU/cores

2 GB RAM

500 - 1000

2 vCPU/cores

4 GB RAM

1000 - 2000

4 vCPU/cores

8 GB RAM

2000 - 5000

8 vCPU/cores

16 GB RAM

5000+

16 vCPU/cores

32 GB RAM

Storage

Table 2. Estimated storage per user per month

Description

Recommended Storage

Low volume

Messaging: sending text messages and links (without file upload)

Low Code: creating plain text records (without file upload)

5 MB/user/month

Medium volume

Messaging: sending text messages, links and occasionally sending documents and images

Low Code: attaching documents and images to records

25 MB/user/month

High volume

Messaging: uploading large files on a daily basis

Low Code: attaching a high number of documents and images to records on a daily basis

100 MB/user/month

Annual storage calculation
  • Base storage (operating system) = 10 GB

  • Recommended storage per user per month based on the matrix above (multiplied by 12 to get yearly amount)

  • Average number of users in a year

  • Safety factor of 2.0 is recommended

Formula
total = (nr. of users * recommended storage * 12 * safety factor) + base storage
Example calculation for 30 medium usage users:
Recommended storage = (30 * 25 MB * 12 * 2) + 10 GB = 28 GB / year

Network

Corteza does need some access to outside services, depending on configuration and customization of the deployment.

SMTP servers should be configured and accessible if deployment uses email address confirmation on sign-in, password recovery or other kinds of email notifications.

HTTP traffic should be allowed if there OAuth2 or OIDC sign-up is enabled or there are custom scripts that access external URLs.

Domain, hostname(s)

Setup with 2 domains (api + app) is the most common one:

  • hostname for Web application

  • hostname for API server

If you already own a domain, you can add two or more hosts and use them to access your Corteza instance.

Testing locally

Alternatively, when doing local-only demo/test deployment, you can use our 127.0.0.1 wildcard alias *.local.cortezaproject.org. Wildcard alias provides a simple way to get any domain under local.cortezaproject.org (api.local.cortezaproject.org for example) to point to localhost.

Warning
You can not use local.cortezaproject.org for public Corteza deployments. It will always point to your local machine.

HTTPS/SSL Certificates

Corteza supports any kind of SSL certificate from all certificate authorities (CA). You can use any existing or new SSL certificate from any commercial SSL certificate provider, self-signed or provided by service like Let’s Encrypt.

If you do not use any other HTTP server, we highly recommend setup with Nginx Reverse Proxy and LetsEncrypt companion container for nginx-prox. It provides fully automated and simplest way to secure Corteza or any other web application you host.

Installing Corteza

Corteza with Docker Compose

Set up Docker

If Docker is already set up on the machine where you want to use Corteza, you can skip this section. If you are using a Docker version below 18.0, I strongly encourage you to update.

If you’re not sure whether you have Docker, open your console or terminal and enter:
docker -v

If the response is "command not found," download and install a Docker community edition for desktop or server or cloud that fits your environment.

Configuration files

Very base Corteza service configuration for Docker Compose consists of one or two files: obligatory docker-compose.yaml and .env file.

Environment file (.env)

The format for environment files is simple and clean with one KEY=VALUE per line. File usually has the name .env and is placed in the same directory as an application. In Docker Compose context it is placed in the same directory as docker-compose.yaml configuration file.

Environment file is used on three levels:
  1. Configuring Docker Compose itself. Implicit, as .env file in the same directory as docker-compose.yaml)

  2. Utilize values for docker-compose configuration. Implicit, as .env file in the same directory as docker-compose.yaml)

  3. Passed to configured services. Environment file(s) must be explicitly referenced by each service (env_file: [ .env ]) You can use .env or one or more environment files. See Delaying API execution.

# Docker images version (1)
VERSION=latest (2)
  1. Comment line

  2. Key and value

docker-compose.yaml file

This file describes service, network and storage configuration in a human and machine readable format.

Full Docker Compose file reference documentation is available on docs.docker.com.

Note

YAML (a recursive acronym for "YAML Ain’t Markup Language") is a human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted.

Environmental variables from .env can be utilized to make docker-compose.yaml file more compact, modular and simpler to change. You can define a variable (like VERSION) and then use it inside yaml file (as ${VERSION}`).

Basic setup for local demo

.env
# Version of Corteza Docker images
VERSION=latest

# Database connection
DB_DSN=corteza:change-me@tcp(db:3306)/corteza?collation=utf8mb4_general_ci

############################################################
# Only part of an documentation example

# In case you have other services running on your localhost,
# change these two numbers to an available port no.
LOCAL_DEMO_SPA_PORT=8080
LOCAL_DEMO_API_PORT=8081
docker-compose.yaml
version: '2.1'

services:
  db:
    image: percona:8.0
    restart: on-failure
    environment:
      # To be picked up by percona image when creating the database
      # Must match with DB_DSN settings inside .env
      MYSQL_DATABASE:      corteza
      MYSQL_USER:          corteza
      MYSQL_PASSWORD:      change-me
      MYSQL_ROOT_PASSWORD: change-me-too
    healthcheck: { test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"], timeout: 20s, retries: 10 }

  server:
    image: cortezaproject/corteza-server:${VERSION}
    restart: on-failure
    env_file: [ .env ]
    environment:
      # Informing Corredor where it he contact us
      CORREDOR_API_BASE_URL_SYSTEM:    "http://127.0.0.1:${LOCAL_DEMO_API_PORT}/system-settings"
      CORREDOR_API_BASE_URL_MESSAGING: "http://127.0.0.1:${LOCAL_DEMO_API_PORT}/messaging"
      CORREDOR_API_BASE_URL_COMPOSE:   "http://127.0.0.1:${LOCAL_DEMO_API_PORT}/compose"
    depends_on: { db: { condition: service_healthy } }
    # Binds internal port 80 to port LOCAL_DEMO_API_PORT on localhost
    ports: [ "127.0.0.1:${LOCAL_DEMO_API_PORT}:80" ]

  corredor:
    image: cortezaproject/corteza-server-corredor:${VERSION}
    restart: on-failure
    env_file: [ .env ]

  webapp:
    image: cortezaproject/corteza-webapp:${VERSION}
    restart: on-failure
    environment:
      # Monolith server in the backend, all services can be found under one base URL
      MONOLITH_API: 1
      # Configure web application with API location
      API_BASEURL:  "127.0.0.1:${LOCAL_DEMO_API_PORT}"
    # Binds internal port 80 to port LOCAL_DEMO_SPA_PORT on localhost
    ports: [ "127.0.0.1:${LOCAL_DEMO_SPA_PORT}:80" ]
Note
Some of the configuration lines in the provided YAML examples and files are written in a single line for brevity and simpler enabling/disabling (commenting-out)

Create an empty directory, with .env and docker-compose.yaml files and copy contents from the examples above. Some operating systems do not like files that start with a dot so make sure .env file is properly named.

Start all services (database, server, corredor, webapp)
docker-compose up -d
Running docker-compose ps should produce something like:
      Name                    Command                  State               Ports
-----------------------------------------------------------------------------------------
basic_corredor_1   docker-entrypoint.sh node  ...   Up             80/tcp
basic_db_1         /docker-entrypoint.sh mysqld     Up (healthy)   3306/tcp, 33060/tcp
basic_server_1     /bin/corteza-server serve-api    Up             127.0.0.1:8081->80/tcp
basic_webapp_1     /entrypoint.sh                   Up             127.0.0.1:8080->80/tcp

You can see 4 services up and running. 2 of them are accessible on localhost (ports 8080 and 8081).

Direct your browser to http://localhost:8080 (use another port if you changed value for LOCAL_DEMO_SPA_PORT). On first visit, you should be redirected to /auth where you can login, sign up, etc.

Create your account through the sign-up form. Corteza detects if the database is empty and auto-promotes first user to administration role.

Stopping and removing containers and date. Do not ask for confirmation, stop containers if running and remove volumes
docker-compose rm --force --stop -v (1)

Other useful docker-compose commands:

View container output (logs) and follow output and output 20 lines from each service
docker-compose logs --follow --tail 20 (1)

Nginx Proxy

Note
Nginx Proxy setup is not needed for local testing. Instructions from Basic setup for local demo are enough to get local demo up & running.

Nginx Proxy (docker image jwilder/nginx-proxy) is auto-configurable reverse-proxy that routes traffic from your public IP to containers on the host LetsEncrypt Nginx Proxy Companion (docker image jrcs/letsencrypt-nginx-proxy-companion) handles the automated creation, renewal and use of Let’s Encrypt certificates for proxyed Docker containers.

Warning
In the following instructions, we assume you don’t have anything similar set up on your current environment. If you have other means to provide traffic forwarding and/or SSL certificate handling, proceed with caution!

Please see Nginx Proxy and LetsEncrypt Nginx Proxy Companion Github pages to

How it works & benefits
  1. Both images mount /var/run/docker.sock (read-only) and listen to docker events (when containers start or stop)

  2. Containers (like Corteza server, and fronted application) that are exposed publicly no longer have to publish their ports on public IP

  3. No complicated firewall or network forwarding rules are needed

  4. Containers MUST (also) be on the same network as nginx-proxy (in the examples we’re using network named proxy)

  5. Nginx Proxy detects VIRTUAL_HOST on each container that comes online. Then it auto-generates configuration, reloads itself and starts forwarding HTTP traffic to that container

  6. LetsEncrypt companion detects LETSENCRYPT_HOST and starts certificate creation process with LE. It also reconfigures nginx-proxy, adds certificates and enables redirection from HTTP to HTTPS

docker-compose.yaml
version: '3.5'

services:
  nginx-proxy:
    image: jwilder/nginx-proxy
    container_name: nginx-proxy
    restart: always
    networks:
      - proxy
    ports:
      - "80:80"
      - "443:443"
    labels:
      - "com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy"
    volumes:
      - ./certs:/etc/nginx/certs
      - ./htpasswd:/etc/nginx/htpasswd
      - ./vhost.d:/etc/nginx/vhost.d
      - ./html:/usr/share/nginx/html
      - ./custom.conf:/etc/nginx/conf.d/custom.conf:ro
      - /var/run/docker.sock:/tmp/docker.sock:ro

  nginx-letsencrypt:
    image: jrcs/letsencrypt-nginx-proxy-companion
    container_name: nginx-letsencrypt
    restart: always
    depends_on:
      - nginx-proxy
    volumes:
      - ./certs:/etc/nginx/certs
      - ./vhost.d:/etc/nginx/vhost.d
      - ./html:/usr/share/nginx/html
      - /var/run/docker.sock:/var/run/docker.sock:ro

# Create network if it does not exist
networks: { proxy: { name: proxy } }
Add custom.conf Nginx configuration file next to docker-compose.yaml
# Make sure we can upload at least 200Mb files
client_max_body_size    200M;

# Add other custom configs.

#
Start both services
docker-compose up -d
Running docker-compose ps should produce something like:
      Name                     Command               State                    Ports
-----------------------------------------------------------------------------------------------------
nginx-letsencrypt   /bin/bash /app/entrypoint. ...   Up
nginx-proxy         /app/docker-entrypoint.sh  ...   Up      0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp

Relaying inbound email to Corteza with Postfix

This quick how-to show what and where to modify in Postfix' configuration files to enable forwarding emails to Corteza.

Change /etc/postfix/main.cf:
virtual_alias_maps = pcre:/etc/postfix/virtual_alias
Add virtual alias to /etc/postfix/virtual_alias:
# Catch-all for corteza.domain.tld and redirect it to corteza_sink mailbox
/.+@corteza\.domain\.tld$/ corteza_sink
Update virtual-alias map/db file and restart postfix
postmap /etc/postfix/virtual_alias
postfix reload
Add entry to /etc/aliases
corteza_sink: "| https://api.your-corteza-instance.tld/system/system/sink?content-type=email&expires=&method=POST&origin=postfix&sign=6280d530ae74f1f9c55e4dd362c9ef2094221287'"

This forwards mail for specific mailbox a curl command. Curl command then pushes that raw email to sink endpoint on Corteza API.

See email automation setup for in information how to create a signed URL

Update aliases
newaliases

Test changes

You can verify if configuration changes have desired effect by sending an email to the configured address or with a simple command line:

echo "hello corteza"|mail -s 'hello' test@corteza.domain.tld

It’s best if you try this from a different machine than the one running postfix

This should produce a new entry in your mail log (usually `/var/log/mail.log) where you can see information about received email

Log example:
postfix/smtpd[23155]: connect from some-host.tld[xxx.xxx.xxx.xxx]
postfix/smtpd[23155]: 277AF5C1B78: client=some-host.tld[xxx.xxx.xxx.xxx]
postfix/cleanup[23159]: 277AF5C1B78: message-id=<b808218e-ce41-6cbf-cb4f-be2b4cf8f776@crust.tech>
postfix/qmgr[14490]: 277AF5C1B78: from=<sender@some-host.tld>, size=1476, nrcpt=1 (queue active)
postfix/smtpd[23155]: disconnect from some-host.tld[xxx.xxx.xxx.xxx] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
postfix/local[23160]: 277AF5C1B78: to=<corteza_sink@my-server>, orig_to=<demo@corteza.domain.tld>, relay=local, delay=0.67, delays=0.03/0.01/0/0.62, dsn=2.0.0, status=sent (delivered to command:  curl --data-binary @- 'https://api.your-corteza-instance.tld/system/sink?content-type=email&expires=&method=POST&origin=postfix&sign=6280d530ae74f1f9c55e4dd362c9ef2094221287'')
postfix/qmgr[14490]: 277AF5C1B78: removed

Forward logs to ELK

Corteza server can log requests, responses, errors and other events as JSON. By default, this logs are outputted to Docker container console. You can configure your (docker-compose) services to forward these logs to an external service.

We’ve prepared cortezaproject/elk docker image with the necessary setup that can consume and

Using GELF docker logging driver and GELF ELK server

The following example assumes you’ve taken configuration from cortezaproject/elk without any changes. You should also extend defined services in your docker-compose.yml with networks and logging section:

# ...
    networks: [ elk ] # or add elk to networks you already have
    logging:
      driver: gelf
      options:
        gelf-address: "tcp://elk:12201"
# ...

Setup

Corteza Command Line Interface (CLI)

Corteza comes with command line interface tool. With it, you can manually or automatically change applications settings, add users, roles, and run the API. You are using the same binary that can start the backend service and the API.

Depending on built flavour, you can use a different entry:
  • Microservice build produce corteza-server-system, corteza-server-messaging and corteza-server-compose

  • Monolith build produces corteza-server binary with system, messaging, compose subcommands

Using the CLI tool

Important
At least basic Docker and Docker Compose knowledge is required for most Corteza setup and management.
Executing with Docker Compose (when service is started)
docker-compose exec <service name> help
Executing with docker (when container is running)
docker exec -it <container name> help
Running with Docker Compose (when service is not started)
docker run -it --rm <container name> help
Executing with Docker (when container is not running)
docker run -it --rm [docker run options] <container name> help
Note
Running and executing with Docker Compose should be easier and simpler because all required options and settings for container are packed in the Docker Compose project configuration file (docker-compose.yaml)
Start HTTP Server with REST API
Usage:
  corteza-server serve-api [flags]

Flags:
  -h, --help   help for serve-api
Provision tasks
Usage:
  corteza-server provision [command]

Available Commands:
  configuration    Create permissions & resources
  migrate-database Run database migration scripts

Flags:
  -h, --help   help for provision

Use "corteza-server provision [command] --help" for more information about a command.
Create permissions & resources
Usage:
  corteza-server provision configuration [flags]

Flags:
  -h, --help   help for configuration
Run database migration scripts
Usage:
  corteza-server provision migrate-database [flags]

Flags:
  -h, --help   help for migrate-database
External authentication
Usage:
  corteza-server system auth [command]

Available Commands:
  auto-discovery     Auto discovers new OIDC client
  jwt                Generates new JWT for a user
  test-notifications Sends samples of all authentication notification to receipient

Flags:
  -h, --help   help for auth

Use "corteza-server system auth [command] --help" for more information about a command.
Auto discovers new OIDC client
Usage:
  corteza-server system auth auto-discovery [name] [url] [flags]

Flags:
      --enable            Enable this provider and external auth
  -h, --help              help for auto-discovery
      --skip-validation   Skip validation
Sends samples of all authentication notification to receipient
Usage:
  corteza-server system auth test-notifications [recipient] [flags]

Flags:
  -h, --help   help for test-notifications
Generates new JWT for a user
Usage:
  corteza-server system auth jwt [email-or-id] [flags]

Flags:
  -h, --help   help for jwt
Export system resources
Usage:
  corteza-server system export [flags]

Flags:
  -h, --help          help for export
  -p, --permissions   Export system permissions
  -s, --settings      Export settings
Import
Usage:
  corteza-server system import [flags]

Flags:
  -h, --help   help for import
Role management
Usage:
  corteza-server system roles [command]

Available Commands:
  useradd     Add user to role

Flags:
  -h, --help   help for roles

Use "corteza-server system roles [command] --help" for more information about a command.
Add user to role
Usage:
  corteza-server system roles useradd [role-ID-or-name-or-handle] [user-ID-or-email] [flags]

Flags:
  -h, --help   help for useradd
Settings management
Usage:
  corteza-server system settings [command]

Available Commands:
  delete      Set value (raw JSON) for a specific key (or by prefix)
  export      Import settings as JSON to stdout or file
  get         Get value (raw JSON) for a specific key
  import      Import settings as JSON from stdin or file
  list        List all
  set         Set value (raw JSON) for a specific key

Flags:
  -h, --help   help for settings

Use "corteza-server system settings [command] --help" for more information about a command.
Set value (raw JSON) for a specific key (or by prefix)
Usage:
  corteza-server system settings delete [keys, ...] [flags]

Flags:
  -h, --help            help for delete
      --prefix string   Filter settings by prefix
Import settings as JSON to stdout or file
Usage:
  corteza-server system settings export [file] [flags]

Flags:
  -h, --help   help for export
Get value (raw JSON) for a specific key
Usage:
  corteza-server system settings get [key to get, ...] [flags]

Flags:
  -h, --help   help for get
Import settings as JSON from stdin or file
Usage:
  corteza-server system settings import [file] [flags]

Flags:
  -h, --help   help for import
List all
Usage:
  corteza-server system settings list [flags]

Flags:
  -h, --help            help for list
      --prefix string   Filter settings by prefix
Set value (raw JSON) for a specific key
Usage:
  corteza-server system settings set [key to set] [value] [flags]

Flags:
  -h, --help   help for set
Sink
Usage:
  corteza-server system sink [command]

Available Commands:
  signature   Creates signature for sink HTTP endpoint

Flags:
  -h, --help   help for sink

Use "corteza-server system sink [command] --help" for more information about a command.
Creates signature for sink HTTP endpoint
Usage:
  corteza-server system sink signature [flags]

Flags:
      --content-type string   Content type (optional)
      --expires string        Date of expiration (YYYY-MM-DD, optional)
  -h, --help                  help for signature
      --method string         HTTP method that will be used (default "GET")
      --origin string         Origin of the request (arbitrary string, optional)
User management
Usage:
  corteza-server system users [command]

Available Commands:
  add         Add new user
  list        List users
  password    Change password for user

Flags:
  -h, --help   help for users

Use "corteza-server system users [command] --help" for more information about a command.
Add new user
Usage:
  corteza-server system users add [email] [flags]

Flags:
  -h, --help          help for add
      --no-password   Create user without password
List users
Usage:
  corteza-server system users list [flags]

Flags:
  -h, --help   help for list
Change password for user
Usage:
  corteza-server system users password [email] [flags]

Flags:
  -h, --help   help for password
Specify one ("modules", "pages", "charts", "permissions") or more resources to export
Usage:
  corteza-server compose export [flags]

Flags:
  -h, --help               help for export
      --namespace string   Export namespace resources (by ID or string)
  -p, --permissions        Export system permissions
  -s, --settings           Export settings
Import
Usage:
  corteza-server compose import [flags]

Flags:
  -h, --help               help for import
      --namespace string   Import into namespace (by ID or string)
Export Messaging resources
Usage:
  corteza-server messaging export [flags]

Flags:
  -h, --help          help for export
  -p, --permissions   Export system permissions
  -s, --settings      Export settings
Import
Usage:
  corteza-server messaging import [flags]

Flags:
  -h, --help   help for import

API Server

Provisioning

Provisioning is step after server is started and API becomes available. It allows you to directly influence how a fresh Corteza instance is configured on first run. This entire process can be disabled by setting env. variables PROVISION_MIGRATE_DATABASE and PROVISION_CONFIGURATION to false.

Table 3. Control provisioning procedure
Type Default value Description

PROVISION_MIGRATE_DATABASE

boolean

true

Controls if database migration (creation of tables, changes of schema between versions) should be done before each service is started.

It’s recommended to keep this setting on. Disabling it on a running server prevents migration between version udpates

PROVISION_CONFIGURATION

boolean

true

Runs various auto-setup procedures and creates resources:

- creates default permission rules and roles - default channels are created - default applications are created - compose resources (namespaces, modules, charts,…​) - settings auto-discovery

Table 4. Provision authentication settings:
Type Default value Description

PROVISION_SETTINGS_AUTH_EXTERNAL_ENABLED

auth.external.enabled

boolean

true

Is OAuth2 enabled or disabled

OAuth2 flow redirection URL.

PROVISION_SETTINGS_AUTH_EXTERNAL_REDIRECT_URL

auth.external.redirect-url

string

searches env-variables (DOMAIN, LETSENCRYPT_HOST, VIRTUAL_HOST, HOSTNAME, HOST) and uses additional info (monolith, api-base-url) to calculate the value

PROVISION_SETTINGS_AUTH_EXTERNAL_SESSION_STORE_SECRET

auth.external.session-store-secret

string

random 64 char string

generated 64 char long string if missing.

Is session cookie "secure" flag used (if yes, cookie can only be access over HTTPS).

PROVISION_SETTINGS_AUTH_EXTERNAL_SESSION_STORE_SECURE

auth.external.session-store-secure

bool

false

If HTTPS is used for external auth redirection url, value is set to true.

PROVISION_SETTINGS_AUTH_FRONTEND_URL_BASE

auth.frontend.url.base

string

Where the frontend SPA is located. Serves as base for generating other auth.frontend.url…​ variables.

PROVISION_SETTINGS_AUTH_FRONTEND_URL_PASSWORD_RESET

auth.frontend.url.password-reset

string

Where the frontend SPA is located, the password reset form.

Where the frontend SPA is located, password email confirmation page.

PROVISION_SETTINGS_AUTH_FRONTEND_URL_EMAIL_CONFIRMATION

auth.frontend.url.email-confirmation

string

auth.frontend.url.base is used as base URL

PROVISION_SETTINGS_AUTH_FRONTEND_URL_REDIRECT

auth.frontend.url.redirect

string

Where the frontend SPA is located. User will be redirected here on successful external authentication. Auto discovery uses auth.frontend.url.base as base URL

PROVISION_SETTINGS_AUTH_EMAIL_FROM_ADDRESS

auth.mail.from-address

string

to-be-configured@example.tld

Email address used for sending auth emails (password reset, email confirmation)

Name used for sending auth emails (password reset, email confirmation)

PROVISION_SETTINGS_AUTH_EMAIL_FROM_NAME

auth.mail.from-name

string

Corteza Team (to-be-configured)

Is internal auth enabled? Enable this to allow users to use all (enabled) internal features (sign-up, log in…​.)

auth.internal.signup

PROVISION_SETTINGS_AUTH_INTERNAL_ENABLED

boolean

true

PROVISION_SETTINGS_AUTH_INTERNAL_SIGNUP_ENABLED

auth.internal.signup.enabled

boolean

true

Is internal sign-up enabled? Enable this to allow users to register if you do not have external authentication providers.

PROVISION_SETTINGS_AUTH_INTERNAL_SIGNUP_EMAIL_CONFIRMATION_REQUIRED

auth.internal.signup-email-confirmation-required

boolean

false

Is email confirmation required for internal sign-ups? Enabled on auto-discovery if server has email capabilities (SMTP_HOST variable is set)

PROVISION_SETTINGS_AUTH_INTERNAL_PASSWORD_RESET_ENABLED

auth.internal.password-reset.enabled

boolean

false

Is password reset enabled for internal account? Auto-discovery enables this if server has email capabilities (SMTP_HOST variable is set)

Table 5. Provision OIDC providers:
Type Default value Description

PROVISION_OIDC_PROVIDER

string

Registers all given providers on start. Provide a list of space delimited provider pairs (<name> <provider-url> or <name> <provider-url> <name-2> <provider-url-2>).

The provider is auto-discovered only if it does not exist (match by name).

Also, make sure that your redirect URL (auth.external.redirect-url) is properly. and PROVISION_SETTINGS_AUTH_EXTERNAL_REDIRECT_URL.

PROVISION_SETTINGS_AUTH_EXTERNAL_REDIRECT_URL

string

Sets value for auth.external.redirect-url setting.

This should be set to: https://api.your-corteza-instance.tld/system/auth/external/%s/callback

Table 6. Provision other external provider
Type Default value Description

PROVISION_SETTINGS_AUTH_EXTERNAL_GITHUB

string

Github’s app credentials: <key> <secret>

PROVISION_SETTINGS_AUTH_EXTERNAL_FACEBOOK

string

Facebook’s app credentials: <key> <secret>

PROVISION_SETTINGS_AUTH_EXTERNAL_GPLUS

string

Google’s app credentials: <key> <secret>

PROVISION_SETTINGS_AUTH_EXTERNAL_LINKEDIN

string

LinkedIn’s app credentials: <key> <secret>

PROVISION_SETTINGS_AUTH_EXTERNAL_OIDC

string

OIDC provider settings <name> <issuer> <key> <secret>

Server options

Type Default value Description

CORREDOR_ENABLED

bool

true

Enable/disable Corredor integration

CORREDOR_ADDR

string

corredor:80

Hostname and port of the Corredor gRPC server

CORREDOR_LOG_ENABLED

bool

false

Log communication with Corredor

CORREDOR_MAX_BACKOFF_DELAY

duration

1 minute

Max delay for backoff on connection

CORREDOR_API_BASE_URL_SYSTEM

string

Instructions passed to Corredor on every call - where Corteza API is located.

CORREDOR_API_BASE_URL_MESSAGING

string

Instructions passed to Corredor on every call - where Corteza API is located.

CORREDOR_API_BASE_URL_COMPOSE

string

Instructions passed to Corredor on every call - where Corteza API is located.

DB_DSN

string

corteza:corteza@tcp(db:3306)/corteza?collation=utf8mb4_general_ci

Database connection string <username>:<password>@(<host>:<port>)/<dbname>?collation=utf8mb4_general_ci

DB_LOGGER

bool

false

Log SQL queries

DB_MAX_TRIES

int

100

Max number of connection retries

DB_CONN_ERR_DELAY

duration

5 seconds

How long do we wait between retries

DB_CONN_TIMEOUT

duration

1 minute

For how long do we try to connect

GRPC_SERVER_NETWORK

string

tcp

Network to use for gRPC

GRPC_SERVER_ADDR

string

:50051

Where do we listen for gRPC connections

GRPC_CLIENT_BACKOFF_DELAY

duration

1 minute

Max delay for backoff on connection

GRPC_CLIENT_LOG

bool

false

Log gRPC communication

HTTP_ADDR

string

:80

IP & port for HTTP server

HTTP_LOG_REQUEST

bool

false

Log requests

HTTP_LOG_RESPONSE

bool

false

Log responses

HTTP_ENABLE_VERSION_ROUTE

bool

false

Enable /version route

HTTP_ENABLE_DEBUG_ROUTE

bool

false

Enable /debug route

HTTP_METRICS

bool

false

Enable (prometheus) metrics

HTTP_METRICS_NAME

string

corteza

Name for metrics endpoint

HTTP_METRICS_USERNAME

string

metrics

Username for metrics endpoint

HTTP_METRICS_PASSWORD

string

(random)

Password for metrics endpoint

HTTP_REPORT_PANIC

bool

Report panic to Sentry

HTTP_CLIENT_TSL_INSECURE

bool

false

Allow insecure (invalid, expired TSL/SSL cert)

HTTP_CLIENT_TIMEOUT

bool

30 seconds

Default timeout for clients

AUTH_JWT_SECRET

string

(random)

Secret used for signing JWT tokens

AUTH_JWT_EXPIRY

duration

1 month

Expiration time

MONITOR_INTERVAL

duration

5 minutes

Output (log) interval for monitoring

PROVISION_MIGRATE_DATABASE

bool

true

Migrate database (if needed) on server start

PROVISION_CONFIGURATION

bool

true

Import configuration (only on empty database) on server start

SENTRY_DSN

string

Set to enable Sentry client

SENTRY_DEBUG

bool

false

Print out debugging information

SENTRY_ATTACH_STACKTRACE

bool

false

Attach stacktraces

SENTRY_SAMPLE_RATE

float32

Sample rate for event submission (0.0 - 1.0, defaults to 1.0)

SENTRY_MAX_BREADCRUMBS

int

Maximum number of breadcrumbs

SENTRY_SERVERNAME

string

Set reported Server name

SENTRY_RELEASE

string

(current version)

Set reported Release

SENTRY_DIST

string

Set reported distribution

SENTRY_ENVIRONMENT

string

Set reported environment

SMTP_HOST

string

localhost:25

SMTP_PORT

int

SMTP_USER

string

SMTP_PASS

string

SMTP_FROM

STORAGE_PATH

string

var/store

Where do we store uploaded files

MINIO_ENDPOINT

string

MINIO_SECURE

bool

true

MINIO_ACCESS_KEY

string

MINIO_SECRET_KEY

string

MINIO_SSEC_KEY

string

MINIO_BUCKET

string

MINIO_STRICT

bool

false

Delaying API execution

With WAIT_FOR* Delaying API execution, waiting for external (HTTP) services.

This might aid you in complex setup where another service should be running and accessible before Corteza is ready.

Type Default value Description

WAIT_FOR

duration

0

Delays API startup for the amount of time specified (10s, 2m…​).

This delay happens before service (WAIT_FOR_SERVICES) probing.

WAIT_FOR_STATUS_PAGE

bool

true

Show temporary status web page.

WAIT_FOR_SERVICES

string

Space delimited list of hosts and/or URLs to probe. Host format: host or host:443 (port will default to 80).

Services are probed in parallel.

WAIT_FOR_SERVICES_TIMEOUT

duration

1m

Max time for each service probe.

WAIT_FOR_SERVICES_PROBE_TIMEOUT

duration

30s

Timeout for each service probe.

WAIT_FOR_SERVICES_PROBE_INTERVAL

duration

5s

Interval between service probes.

Corredor Server

Environmental variables used by Corredor and API Server when connected to and communicating with Corredor.

Type Default value Description

CORREDOR_ADDR

string

corredor:80

This setting is used by both, Corredor and API Server.

For Corredor server: where is server listening on

For API server: where can Corredor server be accessed.

Used by Corredor and API server.

CORREDOR_ENABLED

bool

false

This is a setting for API server, will Corredor be used for server automation?

Used by Corredor and API server.

CORREDOR_MAX_BACKOFF_DELAY

duration

1m

Connection timeout (from API server to Corredor)

Used by API server.

CORREDOR_API_BASE_URL_COMPOSE

string

Location of the compose API (example: https://api.your-corteza-instance.tld/compose)

Used by API server.

CORREDOR_API_BASE_URL_MESSAGING

string

Location of the messagign API (example: https://api.your-corteza-instance.tld/messagig)

Used by API server.

CORREDOR_API_BASE_URL_SYSTEM

string

Location of the system API (example: https://api.your-corteza-instance.tld/system)

Used by API server.

CORREDOR_LOG_ENABLED

boolean

corredor

This setting is used by both, Corredor and API Server.

For Corredor service: where is service listening on (gRPC)

For API server: where can Corredor service be accessed.

Used by Corredor and API server.

CORREDOR_LOG_LEVEL

bool

false

If set to true, API server will log communication with Corredor and Corredor will log incoming requests.

Used by Corredor and API server.

CORREDOR_LOG_LEVEL

string

info

Defaults to trace when CORREDOR_DEBUG is true, defines amount of log information outputed.

Used by Corredor and API server.

CORREDOR_LOG_PRETTY

boolean

false

Are events logged in one-line JSON or formatted to ease development?

Used by Corredor.

CORREDOR_DEBUG

boolean

false

Corredor will log even more information

Used by Corredor.

Email automation: processing inbound email

Corteza supports email automation and processing of incoming email messages. This can be achieved through local email service like [postfix] or 3rd party provider that forward received emails through webhooks.

Flow:
  1. Email is received by internal or external system

  2. Email is forwarded to sink API endpoint

  3. Corteza sink service extracts header and body data from received email

  4. onReceive triggers are filtered for a match (trigger can be configured to match specific headers like sender or subject)

  5. Automation script is called

Preparing (signing) sink URL

Signing the URL (the sign=…​ parameter and value) is created as a combination of all parameters and Corteza’s secret string. This signature should be kept secret as it is effectively same as password that allows access to Corteza.

Example:
docker-compose exec server system sink signature --method POST --origin postfix --content-type email
Command will output a line that looks like this:
/sink?content-type=email&expires=&method=POST&origin=postfix&sign=6280d530ae74f1f9c55e4dd362c9ef2094221287
Table 7. Parameters:
Parameter Description

method

must match the request method

origin

arbitrary string, can be used to describe

content-type

used to set the processor for the data inputed

expires

can be used to sign link with expiration date.

Validation
echo "
From: 
To: 
Subject: hello
Message-ID: <1234@local.machine.example>

Ola Corteza!
" | curl -i --data-binary @- "https://api.your-corteza-instance.tld/system/sink?content-type=email&expires=&method=POST&origin=postfix&sign=6280d530ae74f1f9c55e4dd362c9ef2094221287'"

This command must return 200 OK response.

Backup and restore

We recommend doing regular backups of database and uploaded files. Also, it is highly recommended to do a database backup before you upgrade to newer version if Corteza

Database

We recommend use of mysqldump tool to backup your Corteza database. Tool can be found inside db container (percona:8.0 image).

Warning

Do not try to copy raw database files to perform backup. It might lead to corrupted backup data.

Database dump command:
docker-compose exec -T \
    --env MYSQL_PWD=your-password db \
    mysqldump your-db-name --add-drop-database -u your-username > dump.sql

This will dump entire database and place it in dump.sql file. If you want to use a different tool to create your backup you will need to connect to the container or publish mysql ports.

Tip

You can backup your data without shutting down Corteza server.

Database restore

Database restore command:
docker-compose exec -T \
    --env MYSQL_PWD=your-password db \
    mysql your-db-name -u your-username < dump.sql
Important

We recommend that Corteza server is shut-down until restore procedure is done.

Uploaded files

Without object storage service like Min.io (see [Object storage with Minio]), uploaded files are stored directly on the file-system. Corteza server is storing data to /data directory (if not configured diffrently with *_STORAGE_PATH environmental variables).

You can use anyone of the standard file management tools to make backup copy of the files

Compressing files with tar command:
tar -cjf foo.tar.bz2 data/server/

Restoring files

Uncompromising files from archive with tar command:
tar -xjf foo.tar.bz2