Multi-server environments
Scale horizontally and stay available behind a load balancer.
A multi-server environment has more than one Docker Quickstart container running, with a total of more than one server. An external load-balancer distributes the incoming traffic to one or more servers.
This is in contrast with single-server environments that have one Zato container with one server.
When to use multi-server environments
Use multi-server environments if you would like to:
- Scale your environment horizontally, by adding more servers
- Implement HA (High Availability), by adding standby or active servers
Architectural considerations
If you have more than one server in your environment, consider that all of them run independently of each other, which means that:
If you use publish/subscribe, note that message queues are not shared across servers and each server will effectively act as a separate message broker. It means that you should use pub/sub on only one server in the whole environment, unless you specifically want for each server to be an independent message broker.
If you use the scheduler, it must be started in only one container in the whole environment. All jobs will always run in this container's server. It also means that one-time tasks scheduled dynamically should only be created by that one server.
If you don't use either pub/sub or the scheduler, you can add multiple servers with an external load-balancer in front of them without any further considerations.
What load-balancer to use
Most of the mainstream load-balancers will work well with Zato. The most commonly used ones are HAProxy, Envoy, NetScaler and AWS ELB.
Active/standby environments
- In an environment of this type, there's one server that actively processes incoming requests. If you use pub/sub, all messages will be processed on this server. If you use the scheduler, all tasks run on this one server as well.
- Another server acts as a mirror copy but its container is not started and it doesn't accept any requests yet.
There are two ways to implement the standby server:
- Don't keep the host running - rather, start it only when it's required for the standby to take over the traffic, and then start a container on the host.
- Keep running the host on which a container with the standby server will run, but don't start the container until it's required.
Which way to choose will depend on your preferences and requirements. Consider that:
- If the standby host is not started, it will take a bit more time to first start the host, and then run the container on it. Yet, the startup time of the host is often much shorter than the startup time of the container itself, so it may be that this won't be a concern.
- If the standby host is started, it will always consume processing resources, which incurs costs. At the same time, the time to switch to the standby server will be slightly lower because the host will be already running, so only a container will have to be started.
In either case, the container with the standby server is not started until it's needed. It's only a question whether the host should be always running in the background or not.
It's recommended to choose the first method initially - not keeping the standby host running - and switch to the other one if you decide that it will be beneficial in your case.
The very act of switching the traffic from the active server to the standby one will be implemented using your own provisioning scripts and CI/CD pipelines.
Each Zato container will give you monitoring endpoints that let you decide whether the components inside the container are running or whether the standby server should be started.
For instance - your monitoring tool will be pinging the active server's monitoring endpoint once in 30 seconds. If 3 pings are missed, it will trigger a CI/CD script. The script will stop a container with the first server, and then the same script will start a new container on the mirror host, turning it into a new active server, then the script will notify your monitoring product, and the load-balancer in front of the servers, with information about which server is the active one and which is the standby now.
Active/active environments
In an environment of this type, there's more than one server that actively processes incoming requests. If you use the scheduler, you need to configure the containers to start the scheduler on only one of them.
Multiple servers handle incoming requests, which is what provides the high availability, and more servers mean more throughput.
By default, a starting container will start the scheduler. To prevent it, pass the
Zato_Start_Scheduler=Falseenvironment variable when the container is starting. If you don't set this flag, or if you set it to True, the container's server will start the scheduler.For instance - if you have 5 servers, and you set the scheduler's flag to False on 4 of them, only the fifth one will start the scheduler.
Zato_Start_Scheduler=False also stops running the alerting sweeps and the outgoing connection health checks, because both are scheduled jobs. That is the intended split - the one container that runs the scheduler runs them for the whole environment.There are no limits as to how many active servers you can have in an environment.
Servers don't need to have direct TCP connectivity enabled between them. It means that each can run behind a firewall, with a load-balancer in front of them all, and each server can potentially be installed in a different geographic region.
Note that the server running the scheduler doesn't need to accept API requests from the load-balancer. If it suits your requirements better, you can just treat it as a server dedicated to that task, without the overhead of handling incoming requests.
Before the environment takes real traffic, go through the production checklist.
See also
| Feature | What it does |
|---|---|
| Single-server environments | The default production shape the multi-server one builds on |
| Production checklist | What to confirm before real traffic arrives |
| Docker Quickstart | The image every container in the environment runs on |