HF 101 · Transformers, off the shelf

PYTORCH 102 ended with a network you built, trained and defended yourself. This course starts from the other end: a model with 66,955,010 parameters that somebody else trained on 67,000 movie review phrases, which you call in three lines and never train at all. The first cell prints POSITIVE 0.9999 for a sentence about a film, and the rest of the course is about what those three lines did, what they refuse to tell you, and where the model that looks so sure is wrong.

Three models run through the seven lessons: the sentiment classifier, a 92 MB sentence encoder whose vectors you compare with the cosine from AI MATH 101, and a 26 MB zero-shot classifier that takes labels you invented a second ago. Every number in the lessons was read off one of them rather than promised about it, and the last lesson puts all three on one support inbox and measures which tool was the right one for each question.

How this course teaches

Every lesson is the same twenty six cells, and nine of them are yours.

• A prediction you commit to before the cell runs. It is graded on the reasoning, not the guess, and being wrong here is the point.
• Warmups: a one line blank or a two to four line exercise under the theory it practices, each with a four rung hint ladder behind it, where the last rung explains and still does not hand over the code.
• An exercise that is broken when you open it.
• A diagnose cell: code that runs, prints a confident and plausible number, and is wrong. Something below it refuses the number by computing the same thing a second way, so nothing is taken on trust.
• A challenge that ends in a sentence you write. The tutor grades the sentence, which means a green tick you earned for the wrong reason can be taken back.

No cell in this course passes in the state it ships. That is deliberate, and it is checked mechanically before the course is published.

The particular danger of this subject is a confident number. The sentiment model prints 0.9994 for a sarcastic complaint and 0.8174 for a Turkish sentence it cannot read, and nothing in either output looks different from a right answer. Most of the diagnose cells are built out of that: a score read as a scale, a batch averaged the wrong way, a mask left out, a word count used as a token budget, a search that returns something when the honest answer is nothing.

What you will be able to do

• Run a pretrained sentiment model in three lines, read its output without looking it up, and predict its verdict on a negated sentence before it answers.
• Name the three stages a pipeline hides, tokenizer, model and post-processing, run each one by hand, and reproduce the pipeline's score to six decimals.
• Predict roughly what a word will cost in tokens, read a list of pieces, and budget a text against a 512 token window with the tokenizer rather than a word count.
• Turn a sentence into a vector of 384 numbers, compare two of them with a cosine you wrote yourself, and build a search over a handful of documents that works with no shared words.
• Apply softmax to raw logits by hand, cool it with a temperature, and measure on 800 labelled reviews what a 0.99 from this model is actually worth.
• Classify with labels the model never trained on, explain the entailment question underneath, read a flat spread as a refusal, and test two label sets against each other.
• Combine all three models on a support inbox, triage four new tickets, and say which model failed on which one.

The lessons

1. The pipeline. pipeline("sentiment-analysis", model=...) and one call. The model handles not bad and did not love and gets I expected to hate it, and I did not. backwards at 0.9969, because it cannot carry a negation across a clause. Swap the two halves of The acting was great but the plot was terrible. and the verdict swaps with them. Three sarcastic complaints come back POSITIVE, two of them above 0.999; two Turkish sentences, one praise and one complaint, both come back NEGATIVE; anything past token 512 is dropped without a word. The diagnose cell takes the lowest score in a batch as the most negative review and gets an undecided POSITIVE at 0.5531.

2. What just happened. The three stages, by hand. Six words become nine ids. The model returns logits of -4.27 and 4.61, which sum to 0.34 and are not probabilities; exp and a division turn them into the pipeline's 0.9999 to every digit. A padded batch without its attention mask moves Fine. from 0.9998 to 0.9931 with no error and no label change, and a softmax down the batch axis produces columns that sum to one and rows that mean nothing.

3. Tokenizers. unbelievability is five tokens, cut at un, ##bel, ##ie, ##va, ##bility, none of them a morpheme. 37 plain words are 43 tokens; 120 copies of one rare word are 602, past the window. This tokenizer lowercases everything and cuts 3.14 and fine-tuning into three pieces each; distilgpt2's keeps case, has no [UNK], and cuts the same word into four pieces bare and two with a space in front.

4. Embeddings are vectors. A plain AutoModel returns one 384 wide vector per token; mean pooling over the mask gives one per sentence. Of fifteen pairs, the closest is I love this phone. and This phone is terrible. at 0.704, above the two cats at 0.650, because similarity is about topic. A search finds The screen flickers when it is cold. for the display goes weird in winter at 0.659 with no word in common. The [CLS] row of this model makes every pair of documents score above 0.72 and returns the wrong one at 0.850.

5. Logits to labels. Softmax of two logits is the sigmoid of their gap. Divide the logits by 5 and every verdict holds while the surest score drops from 0.9999 to 0.855. On 800 held out reviews the model is 89.1% right with a mean score of 0.992; the bucket claiming 0.99 or more is 92.5% right and the bucket claiming 0.90 to 0.99 is 56.5% right, and 54 of the 87 mistakes score above 0.99. Averaging scores instead of positive probabilities reads the same eight reviews as 0.906 positive when the honest number is 0.539.

6. Zero-shot. Labels handed in as text, an entailment model asked This example is sports. once per label, a softmax across the answers. Watering the plants comes back science 0.817 and a cancelled flight business 0.357 with a spread of 0.203, the model's way of saying none of the above. On 200 news headlines with four labels it never trained on it is 85.5% right, and this time the top score tracks the accuracy: 98.8% above 0.9, 61.3% below 0.7. The label set the app is broken routes support tickets seven of ten against bug report's five.

7. A tiny demo. Twelve support tickets, three questions. Ten are NEGATIVE, and the stuck login page is read as praise at 0.9922. Routing by the nearest team name embedding gets all ten labelled tickets against zero-shot's seven, which was measured rather than assumed. The two duplicate pairs sit at 0.672 and 0.616, a search for refund returns a ticket at 0.363 with a gap of 0.014 to the runner up, and ranking complaints by sentiment score puts the top five within 0.0025 of each other. One triage function, four new tickets, and a sentence about which model failed where.

The models and the data

Four models from huggingface.co, each pinned to one commit so that the numbers in the prose stay the numbers you see, and one dataset:

	Size	Used in
distilbert-base-uncased-finetuned-sst-2-english	268 MB	lessons 1, 2, 3, 5, 7
distilgpt2	356 MB	lesson 1 (one worked example) and lesson 3 (tokenizer only)
sentence-transformers/all-MiniLM-L6-v2	92 MB	lessons 4, 7
MoritzLaurer/xtremedistil-l6-h256-zeroshot-v1.1-all-33	26 MB	lessons 6, 7
cornell-movie-review-data/rotten_tomatoes, fancyzhx/ag_news	1 MB, 20 MB	lessons 5, 6

The first run of each lesson downloads what it needs into ~/.cache/huggingface; every later run reads the cache. Every setup cell catches the download error and says so in a sentence. Nothing here needs a GPU: the heaviest cell scores 800 reviews in about a second on a laptop CPU.

Where this course sits

Tenth course of the ivyx AI series and the third of the deep learning wave. PYTORCH 101 is the prerequisite, for tensors, shapes and what a forward pass is. The cosine in lesson 4 is AI MATH 101's, and the habit of not trusting a confident score is ML 102's.

HF 102 is next: the base model these were built from, distilbert-base-uncased, trained on labelled data of your own, held against the off the shelf model from lesson 1, and a written verdict on whether the training was worth it.

Requirements

Python 3 with torch, transformers and datasets, and a network connection the first time each lesson runs.

You install those packages yourself. The list above is a statement of what the lessons import, not an installer: nothing in this product installs a package or checks whether one is importable, so a learner opening this course against their own kernel or a bare virtualenv would otherwise meet an ImportError in the second cell of lesson 1. Every lesson's setup cell catches that and re-raises it carrying the command:

pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install transformers datasets

The index url matters on Linux, where a plain pip install torch resolves to the CUDA build and drags in several gigabytes of nvidia-* wheels that a laptop downloads and cannot use. On the ivyx platform kernel all three packages are already there. The pipelines pick the fastest device they find; lessons that open a model and feed it tensors by hand pin device="cpu" so that the tensors you build live where the model does.