Python Microsoft Fabric - Notebooks
Spark notebooks run on demand with parameters, monitored and cancelled from Python services.
Notebooks are Fabric's analytics and machine learning execution engine - Spark code that transforms data, trains models and computes results. Running them on demand from your services is how analytics becomes part of your integrations - an order arrives, a notebook recomputes the forecast. You create a Fabric connection in the Dashboard and every notebook is available to your services.
Finding notebooks
Notebooks are workspace items of the Notebook type.
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class ListNotebooks(Service):
input = 'workspace_id'
def handle(self):
# Get the connection by its Dashboard name
conn = self.microsoft.fabric['My Fabric']
# List only the notebook items
response = conn.list_items(self.request.input.workspace_id, 'Notebook')
notebooks = []
for item in response['value']:
notebooks.append({
'id': item['id'],
'name': item['displayName'],
})
self.response.payload = {'notebooks': notebooks}
Running a notebook on demand
conn.run_job with the RunNotebook job type starts a notebook - the Spark session spins up, the code runs, and the job's status tracks its progress.
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class RecomputeForecast(Service):
input = 'workspace_id', 'notebook_id'
def handle(self):
conn = self.microsoft.fabric['My Fabric']
# Start the notebook
conn.run_job(self.request.input.workspace_id, self.request.input.notebook_id, 'RunNotebook')
self.response.payload = {'status': 'started'}
Passing parameters
Notebooks accept parameters through the job's execution payload - the notebook reads them in its parameter cell, which is how one notebook serves many scenarios.
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class RecomputeRegionForecast(Service):
input = 'workspace_id', 'notebook_id', 'region'
def handle(self):
conn = self.microsoft.fabric['My Fabric']
# Parameters the notebook's parameter cell will receive
payload = {
'executionData': {
'parameters': {
'region': {'value': self.request.input.region, 'type': 'string'},
'horizon_days': {'value': '30', 'type': 'string'},
}
}
}
conn.run_job(self.request.input.workspace_id, self.request.input.notebook_id, 'RunNotebook', payload)
self.response.payload = {'status': 'started', 'region': self.request.input.region}
Monitoring a notebook run
conn.get_job returns the status of a job instance - NotStarted, InProgress, Completed, Failed or Cancelled - together with its timing information.
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class GetNotebookRunStatus(Service):
input = 'workspace_id', 'notebook_id', 'job_id'
def handle(self):
conn = self.microsoft.fabric['My Fabric']
job = conn.get_job(
self.request.input.workspace_id,
self.request.input.notebook_id,
self.request.input.job_id,
)
self.response.payload = {
'status': job['status'],
'started': job['startTimeUtc'],
}
Cancelling a run
A run that is no longer needed - or one that is stuck - can be cancelled with conn.cancel_job.
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class CancelNotebookRun(Service):
input = 'workspace_id', 'notebook_id', 'job_id'
def handle(self):
conn = self.microsoft.fabric['My Fabric']
conn.cancel_job(
self.request.input.workspace_id,
self.request.input.notebook_id,
self.request.input.job_id,
)
self.response.payload = {'status': 'cancelled'}
Collecting results
The standard pattern is for the notebook to write its results to a lakehouse - a Delta table or a file - and for your service to read them from there once the job completes, either through the OneLake data plane or through a SQL connection to the lakehouse's SQL endpoint.