Enmasse overview - GitOps for Zato configuration

Export and import environment configuration as YAML across dev, test and production.

Overview

Enmasse is GitOps for Zato - you use YAML to export or import all of your server definitions, such as REST, GraphQL, SQL connections, scheduler and security definitions, or any other object. The file name for such imports and exports is "enmasse.yaml", so when working with Zato you talk about "importing enmasse" or "exporting enmasse". But remember, "enmasse" is just a YAML config file, straightforward to use.

The enmasse workflow is this:

  • During development, developers use the Dashboard GUI to define new elements, e.g. it is usually convenient to fill out a form to have a new REST channel available.

  • When a solution, or an incremental part of it, is ready for wider testing, an enmasse file is exported from the development server to import it to a test server.

  • Afterwards, the same enmasse export file can be used to configure production servers.

  • The workflow loop now returns to developers who keep using Dashboard to add new server elements.

  • All of the export and import steps can be part of CI/CD pipelines that automate the provisioning of Zato environments

Anything that can be declared with the Dashboard GUI can be also exported or imported with enmasse. For instance, in the screenshot below a new REST channel is being defined.

Now, enmasse can be used to export such an API key and REST channel to YAML and you will be able to import it later on.

security:
  - name: My API Key
    is_active: True
    type: apikey

channel_rest:
  - name: My REST Channel
    service: demo.input-logger
    url_path: /api/my/rest/channel

Such export files can be stored in a git repository. Iteratively, developers add new definitions to the file based on the current state of their work. Ultimately, the file will contain each definition, e.g. each REST channel and other connection types, that are needed to configure a particular project or solution.

  • To prevent any potential leakage of sensitive information, enmasse will not export any passwords from servers to the YAML file. Yet, it is always possible to add passwords to your enmasse files via environment variables, as described later in this chapter, so that you will be still able to recreate fully each definition with passwords where they are needed.

  • An enmasse file contains configuration only, it does not contain services that the configuration may depend on. Any such services need to be deployed before enmasse imports your configuration file.

  • You do not need to create an object in Dashboard before it can be added to an enmasse file. With many similar REST channels or security definitions, differing only in small details, create one in Dashboard, export it to see what its enmasse format looks like, and write the remaining definitions in the file directly.

How to export enmasse

  • In your Dashboard, click System → Config → Export enmasse and your server configuration will be exported to enmasse.yaml.

How to import enmasse

  • In your Dashboard, click System → Config → Import enmasse and select the file to import.


  • To automate the import process, mount your enmasse file under /opt/hot-deploy/enmasse/enmasse.yaml inside the container - best using the blueprint project described fully in the DevOps guide

Automatic imports

Two kinds of automatic imports exist, and together they mean that no environment ever needs a manual import:

  • At startup - a file mounted under /opt/hot-deploy/enmasse/enmasse.yaml is imported each time the container starts, so a freshly created container comes up with all its channels, connections and security definitions already in place

  • On change - enmasse files inside a mounted project - any file whose name contains enmasse and ends in .yaml or .yml - are imported automatically whenever they change, as part of hot deployment, so editing the YAML in git and letting the mount propagate it is a complete deployment mechanism on its own

  • If you notice that enmasse takes longer than a few seconds to complete - in the Dashboard, when you see the "Importing" spinner for a longer time - this indicates that one of your services is missing. For instance, your enmasse definition says that a REST channel should be created for service "api.my-service2" but this service is not currently deployed, or perhaps there's a typo in its name, and enmasse will keep waiting for such a service. Deploy the expected service or correct its name and enmasse will continue, or alternatively, import the corrected enmasse file again.

Environment variables

When enmasse runs, it can read environment variables and replace selected parts of a file to be imported based on what is found in the environment.

Passwords and other types of credentials are what environment variables are usually used for but it is possible to use them with any values in enmasse files.

A value is read from the environment when the whole value is a reference - either with the "Zato_Enmasse_Env." prefix, as in the example below which reads a password for this security definition from a variable called "My_Token_Password" which must exist in the same system where enmasse runs ..

security:
  - name: My.Token
    username: api
    type: bearer_token
    password: Zato_Enmasse_Env.My_Token_Password

.. or with the ${VAR} form, which reads the variable of that exact name:

security:
  - name: My.Token
    username: api
    type: bearer_token
    password: ${My_Token_Password}

Both forms replace the whole value only - a reference cannot be embedded inside a longer string.

Environment variables are read from the standard places, such as ~/.bashrc of the "zato" user inside the container, or from what you exported to a starting Docker container explicitly. Consult the full DevOps tutorial for more information how to pass environment variables to your Zato containers.

Splitting a file into includes

One enmasse file can pull in others through the include key - useful when one team owns security definitions and another owns channels, or when a shared base is reused across environments.

include:
  - security.yaml
  - channels/rest.yaml

scheduler:
  - name: My.Job
    service: my.service
    job_type: interval_based
    minutes: 30
  • Relative paths are resolved against the directory of the file that includes them
  • Included files can have include keys of their own - the mechanism is recursive
  • A file that includes itself, directly or through a chain, is an error and the import stops
  • All the included files are merged into one configuration before anything is imported

Workflow again

  • Use the Dashboard to add your definitions, such as REST channels, etc.
  • Click System → Config → Export enmasse to export them once or more
  • Keep adding them to your git, along with your environment variables for passwords
  • Configure your container to read the enmasse file when it's starting up (check the DevOps guide for details)
  • Alternatively, import enmasse manually by clicking System → Config → Import enmasse

Enmasse reference

All available options for each object type are described in the enmasse reference page.

Read more

Learn more

Production checklist What to confirm before an environment takes real traffic
Audit log Every request, response and error, and what you can do with them afterwards
Alerting Rules over the audit log, and the seven ways an alert reaches you
Health checks Have Zato ping your outgoing connections instead of waiting for traffic to fail
Logging What is written where, and how to ship it to Splunk, Loki, Elastic or Datadog