MLOps Text Data

Adds text steps to the MLOps pipeline, from labelling documents to training, gating and deploying a span model.

A model that finds names, IBANs or tax numbers in documents needs documents with those spans marked by a person. These steps make that data as cards in the MLOps designer: read your documents in, mark what a checksum can prove, let a person correct the rest in Label Studio on this machine, add synthetic documents for training, split by source so the test is honest, check the result, train a span model here or on another machine, stop the pipeline when it misses names or identifiers a person marked, and make the model that passed the one the PII Mask node uses.

What you can do

• Read documents in from a folder of .txt, .md and .pdf files, one JSONL line per document.
• Mark identifiers by rule: IBAN, Montenegrin accounts, JMBG, PIB, TCKN, VKN, payment cards, e-mail and phone numbers, each only when its checksum holds.
• Label in Label Studio, started for you in Docker on 127.0.0.1, with the rules' spans offered as suggestions and a person's corrections brought back.
• Make synthetic documents from templates and made-up values, the same file for the same seed, with nothing a golden set holds.
• Split by group, so no template or source file is in both training and test, with synthetic documents kept to training.
• Check the dataset before training: no leak between parts, every label in the test part, enough documents confirmed by a person.
• Train a span model with a script written into your workspace, whose base model and epochs are yours, on this machine's GPU or on another machine through the Compute panel.
• Gate on recall per label: the model's predictions, with the checksum rules beside them, must find each label a person marked at its floor, and no less than the deployed champion, whose own predictions the deploy keeps and the gate finds by itself.
• Export to ONNX in int8, about a quarter of the size, refused when the export reads any token differently from the model, and gated again on its own predictions.
• Deploy the span model as the champion in models/<name>/, with a person's yes, for the PII Mask node to read.

What it does not do

It does not send documents anywhere. Label Studio runs on this machine and the steps refuse any other address, so a deny egress: remote policy leaves them running.

It does not keep a password. The Label Studio account's password is made on the first start and kept in the Secret Manager, and it reaches Label Studio in the environment, never on a command line.

It does not trust the training script's numbers. The gate scores the predictions the script leaves behind itself, label by label.

It does not find names by rule. Names, firms and addresses have no checksum; finding them is what the model trained on this data is for.

Requirements

Docker for Label Studio, Python 3.8 or newer on this machine, pypdf to read PDFs, torch and transformers to train, onnx and onnxruntime to export, and the Secret Manager. The MLOps extension 1.29.0 or newer draws the steps, and 1.30.0 turns a step's offer, such as starting Label Studio, into a NEXT STEP button.

Getting started

1. Run Label Studio: Start from the command palette.
2. In the MLOps designer, add Import Documents, Prelabel by Rule and Push to Label Studio, and run the pipeline.
3. Label the documents in Label Studio, then add Pull from Label Studio, Split Data by Group and Check the Dataset, and run it again.
4. For synthetic training data, run NLP: Create a Starter Synthesis Spec and add Synthesize Documents before the split.
5. Run NLP: Scaffold the Training Script, set the base model in src/train_ner.py, and add Train a Span Model (or Train on a Remote Machine) and Check Span Recall after the dataset check.
6. Add Export to ONNX, a second Check Span Recall on its predictions and Deploy the Span Model; the scaffold command hands back all of these stages.