When a Python library cannot run in the server
A library that needs its own interpreter, run outside the server and called as if it were local.
Every now and then a Python library cannot run inside the server process - it needs a different Python version, it conflicts with the server's own dependencies, or it simply misbehaves when sharing a process with anything else.
Such a library can still be a first-class connection type. The Connector SDK runs it in an interpreter of your choosing - any Python on the machine - and your connector calls it as if it were local. Unlike the foreign-runtime case, there is nothing to write around the library - the part that hosts it ships with Zato and the module it hosts is plain Python.
The module with your library
The module is ordinary Python with module-level functions - no Zato imports, no protocol, nothing to subclass:
# -*- coding: utf-8 -*-
# Anything the library needs
import the_library_that_cannot_run_in_the_server
def ping():
return 'pong'
def transform(text):
return the_library_that_cannot_run_in_the_server.process(text)
The connector module
The connector starts the library's interpreter with start_process and talks to it with RunnerClient - each call names a function of your module:
# -*- coding: utf-8 -*-
# stdlib
import time
# Zato
from zato.common.sdk import Connector, Field
from zato.common.sdk import runner
from zato.common.sdk.runner import RunnerClient
# How long to wait for the runner process to start accepting connections, in seconds.
_startup_timeout = 15
class TextProcConnector(Connector):
""" Wraps a Python library that cannot live in the server process - the stock runner runs
the library's module in a clean interpreter and this connector talks to it over a local socket.
"""
type = 'textproc'
# Configuration schema - which interpreter runs the runner and which module the runner exposes.
python_path = Field.Text()
module_path = Field.Text()
def create_client(self) -> 'RunnerClient':
# Run the stock runner as a supervised helper process, in a clean interpreter -
# the runner depends on the standard library only, so any interpreter can run it by path.
command = [self.config.python_path, runner.__file__, '{port}', self.config.module_path]
process = self.start_process(command)
client = RunnerClient('127.0.0.1', process.port)
# The interpreter needs a moment before the runner accepts connections.
deadline = time.monotonic() + _startup_timeout
while True:
try:
client.call('ping')
except OSError:
if time.monotonic() > deadline:
raise Exception(f'The runner did not start within {_startup_timeout}s')
time.sleep(0.2)
else:
break
return client
def ping(self, client:'RunnerClient') -> 'None':
client.call('ping')
def transform(self, text:'str') -> 'str':
return self.client.call('transform', text)
In the code above:
python_pathdecides which interpreter runs the library - any Python on the machine will do, no Zato installation is needed on that side- The library's process is supervised like any helper started with
start_process- if it dies, the platform rebuilds the connection, which starts a new one RunnerClient.callsends one JSON object per line and gets one back - an exception raised by your module arrives on this side as an exception too
Creating a definition
Definitions are managed with enmasse under a key derived from the connector's type - custom_ plus the type name:
custom_textproc:
- name: My TextProc
python_path: /opt/python312/bin/python
module_path: /opt/company/textproc_module.py