CTX 201 · The Context Compiler

Between the retriever and the model there are a few lines nobody writes on purpose: take the top k chunks and join them with a newline. That is a context compiler with no rules, and RAG 202 showed what it costs: the right chunk arrives more often as k grows and the answer is right less often. This course replaces those lines with a function that has rules and prints them. The same five chunks in a different order are a different answer: with the gold chunk first, 18 of 24 pass the key; with it last, 15; the whole top five reversed, 14. The warranty question says 24 months with the 2025 edition first and 12 months with it last.

The model is llama3.2:1b through Ollama at seed 0 and temperature 0, the retriever is RAG 201's routed chunk retriever, the key is RAG 202's, and the token counter is the model's own prompt_eval_count, so nothing new is installed. Every number in the prose was produced by the cell above it, on this machine, and every check is a relation or a band rather than a typed number, because another Ollama build may word an answer differently.

How this course teaches

Lesson 1 is a tour: it puts the same five chunks in front of the model in two orders, sweeps the gold chunk's position, and shows the surprise without explaining it. The eight lessons after it are graded work, each built the same way, and nine of their cells are yours.

• A prediction you commit to before the cell runs. It is graded on the reasoning, not the guess, and being wrong here is the point.
• Warmups: a one line blank or a two to four line exercise under the theory it practices, each with a four rung hint ladder behind it, where the last rung explains and still does not hand over the code.
• An exercise that is broken when you open it.
• A diagnose cell: code that runs, prints a confident and plausible answer, and is wrong. Something below it refuses the answer by computing the same thing a second way, so nothing is taken on trust.
• A challenge that ends in a sentence you write. The tutor grades the sentence, which means a green tick you earned for the wrong reason can be taken back.

No cell in this course passes in the state it ships. That is deliberate, and it is checked mechanically before the course is published.

The particular danger of this subject is a rule that scores well for the wrong reason. A chunk counted alone includes the chat wrapper, so a sum of bare counts is over by 25 a chunk. A rule that keeps the higher scored edition gets the veto's score and keeps the stale edition half the time. A cosine dedupe scores two better than exact dedupe by deleting the 2023 edition where it scored lower, and deletes three part numbers on the way. A direction read at k = 3 ships best last at k = 5 and loses four questions. A budget met on chunk tokens is over on every invoice. A compiler credited with four questions over the plain top eight earned three. A bracket counted as a citation names nothing a third of the time. Five held out questions report a citation as free that 24 price at three. Every diagnose cell is one of those, and every cross check is the second route that refuses it.

What you will be able to do

• Count a prompt the way the generator counts it, and charge the chat wrapper, the template and the question before any passage.
• Rank chunks by score and veto the edition a newer one replaces, and measure that reordering the editions does nothing.
• Remove a sentence already in the context, and refuse a cosine rule with the numbers it deleted.
• Sweep the gold chunk's position at three values of k, read a direction with a margin, and refuse a direction measured at a k you do not ship.
• Cut a candidate list to a budget whole or in part with the overhead charged, and audit the cut on the model's invoice.
• Write a context compiler as one function that returns the context and a decisions table with a row per candidate, and attribute its gain rule by rule with an ablation.
• Have the answer cite its passage by number, check the passage against the key, and price the instruction.
• Compile under a budget, hold the choice to five questions the gold set never saw, and write the decision down with the question it costs.

The lessons

1. Head or tail. The same five chunks, best first and best last: 24 months and 12 months. The sweep at k = 5: 18, 17, 17, 16, 15. Eight questions whose verdict depends on the position. At k = 3 the tail wins.

2. The token budget. prompt_eval_count as the ruler: the same on a second call and with the passages reversed. The wrapper is 25 tokens and the template 22. A quarter of the characters estimates a chunk within a dozen tokens; the word count is under by 30 percent. The whole prompt is within two tokens of its parts. Three chunks counted by bare calls are 75 over.

3. Score, then edition. Two editions in the index, both retrieved for six questions, the older scoring higher on three. Newest first: 18 to 18, no answer changed. The veto: 18 to 20, the battery and the collector car. A rule keyed on the score also scores 20 and keeps the stale edition three times.

4. Deduplication. Ten repeated sentences across 24 contexts, 170 tokens, and no answer changed when they go. The cosine rule at 0.95 makes thirteen merges, six losing a number, the Corsa's schedule into the Nube's at 0.981, and scores 20 by dropping the stale edition by luck.

5. Ordering, measured. 15, 15, 17 at k = 3; 18, 17, 17, 16, 15 at k = 5; 16, 15, 14, 15, 17, 15, 14, 15 at k = 8. A direction with a margin of two. Best last at k = 5 from the k = 3 reading: 14 against 18.

6. The budget cut. The paper cut over budget on 24 of 24 invoices, 209 tokens for a budget of 150. Whole chunks first fit, or a prefix with a trimmed tail, measured word by word; the two answer within three questions of each other at every budget. Arrival rises with the budget and right answers do not.

7. The compiler as a function. Retrieve, veto, dedupe, cut, and a decisions table with a final row per candidate. 21 at 250 against the plain top five's 18; 20 at 300 against the plain top eight's 16 and the same budget cut's 17. The ablation: the veto two, the dedupe one, both three.

8. Citations. By id, 24 answers with brackets and 16 naming a passage; by number, 23 and 19 the gold document. The support check: 19 of 23 named passages hold the key, one right answer is unsupported, two wrong answers are supported. The instruction costs three right answers.

9. Compile for a budget. The gold score and the held out score as functions, a chooser with a refusal on a held out reversal, and the decision as a record with its model. 250 tokens, best first, cited by number: 21, 16 best last, 18 cited on the 24; four, three and four on the five, with the Adler's filter and the old car question trading places.

Requirements

• Python 3.9 or later with ollama, numpy and pandas.
• A running Ollama (https://ollama.com) with two models pulled once:

ollama pull nomic-embed-text
ollama pull llama3.2:1b

• RAG 202, for the generator, the key and the two columns this course measures; RAG 201, for the retriever; PANDAS 101 for the decisions tables.

Every lesson's setup cell rebuilds the corpus, the index and the generator, checks for the server and the models it needs, and says what to run if either is missing. The counts, the sweeps and the scores are this machine's: the generator is deterministic here, another build may word an answer differently, and every check in the course is a band or a relation rather than a typed number.

What to read

Lost in the Middle: How Language Models Use Long Contexts by Liu and others (2023) is the paper lesson 5 is a small version of, and its figures are worth comparing with your own sweep, which will not draw the same curve on a 1B model at five chunks. The Ollama API documentation on the chat endpoint explains prompt_eval_count, the number lesson 2 is built on. RAG 202's lesson 2 is the measurement this course reuses, and its lesson 8 is where the model's preferences, a negation, a quote without an answer, were first seen.