Fabric notebooks and pipelines
Run a notebook and read what it wrote, run a pipeline and refresh a report, and let Fabric read your own systems.
This page will show you how to work with Microsoft Fabric notebooks and pipelines in Python:
- Running a notebook and reading its results
- Running a pipeline and refreshing a report
- Letting a notebook read your own systems
In Fabric nomenclature, running a notebook, running a pipeline and refreshing a report's data are all jobs.
Such a job runs in Fabric as a background task which may potentially take minutes to complete, so there are two functions to use when working with jobs.
conn.run_jobstarts the job and returns its IDconn.wait_for_jobwaits until the job is done, raising an exception if it failed
Connection and IDs
The Fabric connection and the workspace and lakehouse IDs are the same as in the tutorial.
You'll also need the ID of the item to run (e.g. of a job), which is in the browser's address bar when the item is open in Fabric:
| Item | In the address bar after | Used on this page |
|---|---|---|
| Notebook | /synapsenotebooks/ | ba7b1e23-b680-4fe7-a982-2a979b125604 |
| Pipeline | /pipelines/ | bf4868aa-05c8-4213-ab4d-8dcbe3189767 |
| A report's semantic model | /datasets/ | e040a801-73bd-4b01-b036-d490c7122941 |
Running a notebook and reading its results
In this case, a notebook writes to the reminder_candidates table and a service reads what the notebook wrote.
The service below will:
- Start the notebook
- Wait until it is done
- Read the table the notebook wrote with
conn.query
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class SendReminders(Service):
name = 'fabric.send-reminders'
def handle(self):
workspace_id = '262ddd3d-3d0d-4495-b80f-6dddda1e23a1'
lakehouse_id = '73811955-064d-481a-a7e9-f9c563124f9e'
notebook_id = 'ba7b1e23-b680-4fe7-a982-2a979b125604'
# Get a Fabric connection ..
conn = self.microsoft.fabric['Zato Fabric']
# .. start the notebook ..
job_id = conn.run_job(workspace_id, notebook_id, 'RunNotebook')
# .. wait until it is done ..
conn.wait_for_job(workspace_id, notebook_id, job_id)
# .. read the table it wrote ..
sql = """
select appointment_id, location, starts_at
from reminder_candidates
order by starts_at
"""
rows = conn.query(workspace_id, lakehouse_id, sql)
# .. log each row ..
for row in rows:
appointment_id = row['appointment_id']
location = row['location']
self.logger.info(f'Reminder -> {appointment_id} at {location}')
# .. and tell our caller how many there were.
self.response.payload = {'reminders': len(rows)}
After invoking the service you'll see:
Running a pipeline and refreshing a report
Here, a pipeline writes the occupancy table that a Power BI report reads, and the report's data is refreshed once the pipeline is done.
The service below will:
- Run the pipeline and wait for it
- Refresh the report's semantic model and wait for it too
- Return when both are done
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class RefreshOccupancyReport(Service):
name = 'fabric.refresh-occupancy-report'
def handle(self):
workspace_id = '262ddd3d-3d0d-4495-b80f-6dddda1e23a1'
pipeline_id = 'bf4868aa-05c8-4213-ab4d-8dcbe3189767'
dataset_id = 'e040a801-73bd-4b01-b036-d490c7122941'
# Get a Fabric connection ..
conn = self.microsoft.fabric['Zato Fabric']
# .. run the pipeline and wait for it ..
job_id = conn.run_job(workspace_id, pipeline_id, 'Pipeline')
pipeline = conn.wait_for_job(workspace_id, pipeline_id, job_id)
# .. refresh the report's data and wait for that too ..
job_id = conn.run_job(
workspace_id,
dataset_id,
'DefaultSemanticModelRefresh')
refresh = conn.wait_for_job(workspace_id, dataset_id, job_id)
# .. and tell our caller when each of them ended.
self.response.payload = {
'pipeline_ended': pipeline['endTimeUtc'],
'refresh_ended': refresh['endTimeUtc'],
}
After invoking the service you'll see:
If the pipeline fails, conn.wait_for_job raises an exception with Fabric's reason and the refresh does not start.
Letting a notebook read your own systems
In this example, a notebook needs data from a database in your own network, so it will invoke a REST API channel exposing a service to that notebook.
The service will in turn look up the data inside your on-prem database and return it to the notebook.
# -*- coding: utf-8 -*-
# Zato
from zato.server.service import Service
class GetAppointments(Service):
name = 'fabric.get-appointments'
input = 'since'
def handle(self):
# The query to run ..
sql = """
select appointment_id, location, starts_at, status
from appointments
where updated_at >= :since
order by starts_at
"""
# .. the value for it, from our input ..
input = self.request.input
params = {
'since': input.since,
}
# .. get a connection to the local database ..
conn = self.out.sql['Appointments DB']
# .. run the query ..
rows = conn.execute(sql, params)
# .. and return the rows to our caller.
self.response.payload = rows
Now in the notebook, a cell can invoke the channel and load the rows into a DataFrame:
# In a Microsoft Fabric notebook
import requests
url = 'https://api.example.com/api/appointments'
headers = {'X-API-Key': '<the key>'}
params = {'since': '2026-08-31T00:00:00Z'}
response = requests.get(url, headers=headers, params=params)
appointments = response.json()
frame = spark.createDataFrame(appointments)
display(frame)
See also
| Page | What it covers |
|---|---|
| Your first Fabric integration | The app registration, the connection and where the IDs come from |
| Fabric API - Jobs | conn.run_job, conn.wait_for_job, conn.get_job and conn.cancel_job, job types and statuses |
| Writing data to Fabric | The nightly load the pipeline runs after |