SEC 301 · Multi-Tenant Isolation

Every course in this branch so far ran the retriever for one customer. This one runs it for two, and the two are a trap: Northgate Motors and Westford Autos both sell the Kessel Vento, and they service it differently. Ask each how often the Vento needs its oil changed and Northgate says every 20,000 km and Westford says every 15,000. Put both dealerships' documents in one vector index, the cheap and common design, ask Westford's question, and the index answers with Northgate's document. The answer is a real Vento service interval, the wrong dealership's, and nothing about it looks wrong.

That is the subject: not the obvious leak where one tenant reads a document about a car nobody else stocks, but the plausible leak where two tenants share a thing and the wrong answer is well-formed and confident. The course carries the tenant through five layers, retrieval, cache, logs, metrics and the eval set, and finds that two of them are commonly guarded and three leak in silence. The material is Northgate's 52 documents from the wave one courses and a second dealership built from them by transformation, with one model kept the same on purpose. Retrieval uses a local embedding model; everything else is numpy and pandas, and every number is one you recompute in a cell.

How this course teaches

Lesson 1 is a tour: it asks the shared Vento question as Westford, receives Northgate's document, and filters the index to fix it, then names the four layers the filter does not reach. The eight lessons after it are graded work, each built the same way, and nine of their cells are yours.

• A prediction you commit to before the cell runs. It is graded on the reasoning, not the guess, and being wrong here is the point.
• Warmups: a one line blank or a two to four line exercise under the theory it practices, each with a four rung hint ladder behind it, where the last rung explains and still does not hand over the code.
• An exercise that is broken when you open it.
• A diagnose cell: code that runs, prints a confident and plausible answer, and is wrong. Something below it refuses the answer by computing the same thing a second way, so nothing is taken on trust.
• A challenge that ends in a record or a sentence you write. The tutor grades the sentence, which means a green tick you earned for the wrong reason can be taken back.

No cell in this course passes in the state it ships. That is deliberate, and it is checked mechanically before the course is published.

The particular danger of this subject is a boundary that holds only where someone is watching. The index leaks a plausible wrong answer, and a filter fixes it. The cache serves one tenant's answer to another, and a key field fixes it. Then the log exposes a tenant's rows with no column to scope on, a dashboard hides one tenant's regression inside a healthy average, and an eval gate ships a failing tenant in a pooled score, and none of those three produces a wrong answer a customer would report. Every diagnose cell is one of those, and every cross check is the second computation that refuses it.

What you will be able to do

• Ask the one question two tenants answer differently and receive the other one's answer from a shared index.
• Thread a tenant through a call path and name the two places it is reconstructed from content or defaulted rather than passed.
• Choose between a filtered shared index and a per-tenant index on correctness, cost and failure mode, knowing the filter raises recall rather than costing it.
• Add the tenant to a cache key and show it both removes wrong hits and gives the smaller tenant a cache of its own.
• Find a tenant's documents quoted in a log with no tenant column, and scrub the bodies to identifiers.
• Bring one tenant's hidden regression back into view by grouping the metric, and alarm on the worst tenant rather than the average.
• Split a pooled eval set, gate each tenant against the bar, and floor each tenant's case count.
• Write a cross-tenant probe that passes on the fixed system and fails on a reintroduced leak, and say why it must test shared material.

The lessons

1. A plausible wrong answer. One shared index over two dealerships. Westford's Vento question returns Northgate's document at rank one, an answer of 20,000 km when Westford's own is 15,000, and 38 of 78 question-tenant pairs leak the same way. A tenant filter fixes the retrieval and names the four layers it does not reach.

2. Carrying an identity. A correct path passes the tenant to the index filter. Two reconstruction sites break it: a retriever that guesses the tenant from the question, which can resolve only 1 of 40 questions and is ambiguous on the Vento, and a cache key that defaults the tenant, which fails silently on every omitted call.

3. Isolation at the index. A shared index scores 0.885 recall and leaks on 76 of 78 pairs; filtering it to the asking tenant raises recall to 0.962, recovering six questions, and drops the leak to zero, because the other tenant's near-duplicates were distractors. A per-tenant index reaches the same numbers with a structural, not a runtime, guarantee.

4. The cache, again. A tenant-less key hits 120 times over a two-round replay, 40 correct and 80 crossing between tenants. Adding the tenant drops the crossings to zero and raises the correct hits to 80, because the tenant-less key had been overwriting Westford's cache with Northgate's answers.

5. Logs and errors. Retrieval and cache are correct and the tenant still leaks: a log with no tenant column makes 40 of Northgate's rows readable by a Westford operator, every one of the 78 rows carries a document body that should be an identifier, and the 10 low-confidence rows carry theirs into the error tracker.

6. Metrics that aggregate the label away. A reindex takes Westford to 0.647, below the 0.75 bar, while the traffic-weighted aggregate falls only to 0.926 and stays green, because Westford is 15 percent of the traffic. Grouping by tenant and alarming on the worst tenant brings the regression back.

7. An eval set per tenant. A shared eval file scores 0.815 after the same regression, over the bar, so the gate ships a broken release. Split it and Northgate is 0.975 and Westford 0.647. The case count is a second hidden knob: with the 2 cases a small tenant often has, the pool would read 0.959.

8. The test that would have caught it. A cross-tenant probe passes with zero failures on the fixed system and fails 38 times when the filter is dropped; the cache probe is zero on the tenant key and 40 on the tenant-less key. A probe on tenant-unique material stays green on a broken boundary, so it must test the shared Vento.

9. Count the layers. All five layers leak if the tenant is dropped, two are commonly guarded and three are quiet, so the course's opening hypothesis is corrected: the drop is at three layers, not one, and the quiet three outnumber the loud two. The boundary is defended layer by layer, and counted by an audit rather than a bug report.

What you need

This course runs on Python with numpy, pandas and a local Ollama for the document embeddings. Install the Python packages into the kernel with:

%pip install numpy pandas ollama

Ollama is a separate install from https://ollama.com; start it and pull the embedding model once:

ollama pull nomic-embed-text

The model is a few hundred megabytes and never needs the network again. In IVYX Studio the LLMS panel installs Ollama and pulls the model for you. Both dealerships' corpora are built in the first cell of every lesson and embedded into one index, so each lesson runs on its own.

What to read outside this course

DATA 202 built the vector index this course shares between tenants, and DATA 203 built the cache and the tenant-in-the-key rule that lesson 4 reuses. SEC 201, on permissions and secrets, is the other prerequisite, because a tenant boundary is a permission boundary. Beyond the courses, any platform's multi-tenancy documentation covers the same five layers under different names, and the value of building it here is that you watch each layer leak and measure how much rather than trusting that the framework carried the tenant for you.