OpenLineage Export

Publishes this workspace's past lineage runs to any OpenLineage server, so Marquez, DataHub and Atlan read them.

What you can do

The Lineage hub knows which stages of your pipeline read which data and wrote which artefacts, for every run on the audit trail. This extension translates that into OpenLineage run events and posts them to a server you configure.

In the lineage ecosystem you become a source rather than a competitor: the graph the editor already holds shows up in the tool your organisation already uses.

Getting started

1. Open Settings → OpenLineage Export.
2. Put your server's base address in openlineage.url — http://localhost:5000 for a local Marquez.
3. If the server needs a key, store it in the SECRETS panel and put the entry's name in openlineage.apiKey.secretRef. The key itself never goes into settings, which travel with the workspace.
4. Run OpenLineage: Export Lineage Runs from the command palette.

Pass dryRun to build the events and see what would be sent without sending anything. It needs no server, which is the point of it.

What gets published

One editor run becomes one pipeline run plus one run per stage, filed under the pipeline as its parent. Each stage's reads are the run's inputs and its writes are the outputs, with the engine's content digest riding as the datasetVersion facet.

Where the editor's trace id is already a UUID it is the OpenLineage run id, so an audit trail entry and a lineage run are the same string.

What it will not do

A pipeline that has never run exports nothing. OpenLineage is a log of runs, and the shape it provides for a graph that never ran is refused by Marquez. The export says so rather than inventing a run to carry the graph.

Re-exporting is safe. Run ids are derived from the run rather than minted, so pressing Export twice — or running it on every push — updates the same runs instead of duplicating them.

Requires

The Lineage extension, which is where the graph comes from, and the desktop app, because the server is reached through the host's own HTTP surface.