Extensions
This section covers the system behind extensions. If you’re interested in extension development, refer to the integrator guide. |
Overview
Components
- Corteza Corredor Server
-
A Node.js service written in TypeScript. It is the heart of the system as it parses and serves automation scripts. It also executes server-scripts.
- Corteza Server
-
The Corteza server is responsible for automation script execution. On its own, Corredor server is unable to do any code execution; it must be invoked either via Corteza server or manually via gRCP protocol.
- Corredor Client
-
Each Corteza webapp that allows client-script execution implements this. The Corredor Client is responsible for client-script registration and their execution. Automation scripts are received from the Corteza Server.
Corredor Server
Corredor server is mainly responsible for:
-
Loading, processing and serving automation scripts,
-
server-script execution.
Server-scripts file structure
/server-scripts (1)
/... (2)
1 | Root folder for all server scripts (under each search path). |
2 | Undefined file structure; can be defined as needed. |
Client-scripts file structure
/client-scripts (1)
/auth (2)
/... (7)
/admin (3)
/... (7)
/compose (4)
/... (7)
/messaging (5)
/... (7)
/shared (6)
/... (7)
1 | Root folder for all client scripts (under each search path). |
2 | Defines a bundle for Corteza Auth. |
3 | Defines a bundle for Corteza Admin. |
4 | Defines a bundle for Corteza Low Code. |
5 | Defines a bundle for Corteza Messaging. |
6 | Reserved directory for any shared logic, such as custom libraries, assets, … |
7 | Undefined file structure; can be defined as needed. |
Automation script processing
Flow outline:
- Scan extensions defined in the .env
-
Go over all extensions defined in
the CORREDOR_EXT_SEARCH_PATHS
.env variable. Extensions are separated by:
.
Note that all of the following steps are performed for each of the extensions. |
- Dependency management
-
Load all requested dependencies by the given extension.
Dependencies are scoped to the extension that has requested them. If two extensions wish to use the same dependency (axios for example) they both should define it. |
- Parse server-scripts
-
Go over each script file defined inside the
server-scripts
sub-tree. Parse the script and extract all the important bits such as dependencies, triggers, iterators and the exec function itself.
At the moment server scripts are not bundled. |
- Parse client-scripts
-
Go over each script file defined inside the
client-scripts
sub-tree. Parse the script and extract all the important bits such as dependencies, triggers, iterators and the exec function itself. Create a per-app bundle containing relevant automation scripts, dependencies and other relevant bits. Refer to Client-script bundling for details about creating bundles.
Bundling is done using webpack. |
- Prepare a list of available scripts
-
This allows us to inspect what scripts were loaded from inside web-apps. It also simplifies the process of determining available automation scripts from inside client-script bundles.
Failing automation scripts are also included in this list but provide a descriptive error. This allows easier debugging. |
Automation script naming
Corredor determines automation scripts name on-the-fly from the extension and it’s relative path. This simplifies the script’s definition as we don’t need to worry about unique names and possible conflicts between extensions. Script name is defined as:
/${path to script}/${file name}:${export name}(1)(2)(3)
1 | Path to the script file, excluding the search path. |
2 | The script’s file name. |
3 | Used export name; this will normally be default . |
The base search path of the extension is excluded from the script’s name. This allows easier reusability across different systems that don’t define the same file hierarchy. |
Client-script bundling
Client-scripts are bundled using webpack. Bundles help us assure consistent code execution between different user-agents and allow the use of external node dependencies.
The bundling process creates multiple bundles, one for each available application (Auth, Admin, Low Code, …). This simplifies the process of registering client-scripts on the client-side.
Bundle name is determined from the extensions file-structure:
|
The bundling process creates a boot loader file for each available application.
The file consists of { name, triggers, security }
JSON objects for each available automation script.
The Webpack is then executed over each boot loader file to create a bundle for the given application.
If the application doesn’t define any automation scripts the bundling process is omitted.
gRPC server
Corredor server allows communication via the gRPC server via the gRPC protocol via the following services:
-
Server scripts with list and exec procedures,
-
client scripts with list and bundle procedures.
Refer to protobuf service definition for the details on the definitions.
You can use BloomRPC or similar when working solely on the Corredor server to remove the need for the Corteza server. |
Server-script execution
Corredor server is and should not be able to perform any code execution on its own. Any execution is and should be invoked either by the Corteza server or manually via gRPC protocol. |
Watching for changes
When the Corredor server is running in development mode or is configured so via the .env variables it uses file watchers to detect changes to the extensions and dependency definition files. When a change occurs, the server wil restart the entire Automation script processing flow.
Corteza Server
Corteza server is mainly responsible for:
-
Providing endpoints for fetching scripts and script execution,
-
scheduling deferred automation scripts,
-
requesting script execution on the Corredor server.
The following automation scripts are executed directly by the Corteza server:
|
Script endpoints
- [APPLICATION NAME]/automation/list
-
Provides a list of available automation scripts for the given Corteza application.
- [APPLICATION NAME]/automation/{bundle}-{type}.{ext}
-
Provides the client-script automation bundle for the given Corteza application.
If the given application doesn’t have it’s own bundle, the endpoint returns a 404 not found. |
- [APPLICATION NAME]/automation/trigger
-
Requests explicit server-script execution on the Corredor server.
- system/sink
-
Provides an endpoint that server-scripts are able to listen on and perform operations for.
Event bus
Any executable automation script is registered on the event bus. Whenever any part of the system might require a script execution, it dispatches an event on the event bus and any registered script that conforms to the requirements is executed on the Corredor server.
The script registration flow is as follows:
- Fetch available scripts from the Corredor server
-
This provides a full automation script list, including invalid scripts (those that didn’t compile successfully) — these are excluded.
- Prepare a lightweight script representation
-
Define a lightweight
handler
struct for easier execution determination and script identification. Execution is performed by the handler function that allows a bit more flexibility on how to handle each script type.
Note that the iterators handler function is a bit more complex than the rest. |
- Register the script on the event bus
-
Register the above simplified automation script on the event buss.
Now when it comes to script execution, this is invoked by dispatching an event on the event bus. To outline the flow:
- Dispatch the event on the event bus
-
An event is a generic interface that allows the event bus to determine what scripts should be executed.
- Execute the scripts
-
Iterate and execute any automation script that conforms to the dispatched event. This can be done either synchronously or asynchronously.
Server side script execution
If the result of an automation script is an |
- Explicit
-
Explicit scripts are invoked by the front-end applications or manually via the API. When invoked, an event is constructed and dispatched via the event bus.
This is also true for sink script execution. |
- Implicit
-
These are triggered as a side-effect of another operation, such as record creation, user registration, password change, … Each part of the system that performs some operation that may invoke script execution generates an event and dispatches it on the event bus.
The same is also true for the Scheduler. |
- Iterator
-
Iterator script execution is invoked like any other deferred script. The important difference is that a new instance is executed for every resource that matches the trigger’s constraints.
If 10 records match the provided filter, the script will be executed 10 times.
System sink
System sink allows the implementation of custom endpoints on the Corteza server. This can be used for things like webhooks.
-
intercept any HTTP request on the
/sink
endpoint, -
basic request validation (provided required parameters),
-
validate request against the sink’s signature,
-
process request based on:
-
Content-Type
or any other HTTP header, -
remote address (IP),
-
request method and path,
-
username and password (HTTP basic auth),
-
GET (query string) parameters,
-
POST parameters,
-
-
generate an event and dispatch it via the event bus.
Scheduler
The scheduler system is responsible for triggering deferred scripts. At it’s core, scheduler is a "simple" ticker that dispatches periodical event on the event bus.
The interval is "hard-coded" to 1minute but that can be made configurable in the future. |
There is no mechanism in place that would prevent the automation scripts to overlap. For example:
This could be improved in the future. |
Corredor Client
Corredor client is mainly responsible for:
-
Script registration on the event bus,
-
registering available UI hooks,
-
script execution; either locally or on the server.
Event bus
Any executable automation script is registered on the event bus. Whenever any part of the system might require a script execution, it dispatches an event on the event bus and any registered script that conforms to the requirements is executed on the Corredor server.
The event bus concept follows the same pattern as the Corteza server does. |
- Fetch available script bundle from the Corteza server
-
This provides available client-script automation scripts defined for the given application. This allows us to register available client-scripts.
- Fetch available scripts from the Corteza server
-
This provides a full automation script list, including invalid scripts (those that didn’t compile successfully) — these are excluded. This allows us to register explicit server-scripts.
- Prepare a lightweight script representation
-
Define a lightweight
Handler
object for easier execution determination and script identification. Execution is performed by the handler function that allows a bit more flexibility on how to handle each script type. - Register the script on the event bus
-
Register the above simplified automation script on the event buss.
Now when it comes to script execution, this is invoked by dispatching an event on the event bus. To outline the flow:
- Dispatch the event on the event bus
-
An event is an object that allows the event bus to determine what automation scripts should be executed and their execution context/arguments.
- Execute the scripts
-
Iterate and execute any automation script that conforms to the dispatched event. This can be done either synchronously or asynchronously.
A big difference between server-side and client-side execution is that client-side arguments are provided as references and not as copied objects. Any change done to the original object is reflected to the original object. |
UI hooks
A UI hook defines a component that represents an explicit automation script — either a client-script or a server-script. UI hooks mainly define the component properties — what element to render, styling, what script should be executed, additional context, …
It is up to the web-app to render these components and dispatch events on the event bus when needed.
Script execution
Corredor client is able to interact with some UI elements such as |
- Explicit
-
Explicit scripts are invoked by the user by explicitly invoking the above mentioned UI hooks. The web-app creates an event and dispatches it on the event bus. The handler function requests the script’s execution on the Corteza server.
An event caused by an explicit script will provide the script name that should be executed. Implicit events do not. |
- Implicit
-
These are triggered as a side-effect of another operation, such as form submission, view render, validation error, … Each part of the system that performs some operation that may invoke script execution generates an event and dispatches it on the event bus.