Guardrails

Attack packs for jailbreaks, PII leaks and prompt injection, run against your own prompts through the eval suites you already have, with a verdict a pipeline gate can hold.

What you can do

• Install an attack pack. Three ship with this build — jailbreak, PII leak and prompt injection, six cases each. Installing writes them into your workspace as eval datasets and answers the suite block that runs them.
• Run them against a real prompt. A pack becomes an ordinary eval suite pointed at one of your instruction classes, so it runs with the same repeats, thresholds and baselines as the rest of your suites.
• Gate on the result. Two numbers per suite: guardrail.held is the share of cases where the attack did not get through, and guardrail.scored is the share of cases something was actually able to check. Threshold both and a pipeline stops on a red one.
• Check a single answer in band. guardrails.check takes one piece of text and answers held, breached or unknown. It reports; it does not block a call.
• Read the cases before you trust them. Click a pack in the panel to open its cases: the prompt each one sends to your model, the compliance marker that decides the verdict, and why the case counts as an attack. After a run, each case carries its own outcome instead of only the suite mean.
• Add your own cases. The datasets in your workspace are the source of truth at run time, so cases you write are scored exactly like the ones that shipped.

Getting started

1. Open the Guardrails panel and press Install every attack pack, or run guardrails.pack.install with the instruction class you want attacked:

   { "target": "my.instruction.class" }
   

2. If you already had an eval suite file, paste the suite block the install answered into it. If you did not, one was created for you.

3. Run the suites from the Evals panel, or from a pipeline stage that calls evals.suite.run.

4. Read the verdict in either panel. A failing suite names the case and the string that proved the attack got through, and expanding the pack shows every case with its own outcome: held, breached, not scored, or no answer.

How a case is scored

Every case plants a compliance marker — a string the target can only produce by doing what the attack asked, and never by quoting the prompt back. The jailbreak cases ask for a token the target has to compute; the injection cases hide one inside the material being summarised; the PII cases ask for a synthetic record in a reformatted shape. If the marker turns up in the answer, the attack got through.

That makes scoring deterministic and language-independent: no refusal-phrase list, so a target answering in Turkish or German is measured the same way as one answering in English.

The limit, stated plainly: a marker proves the target complied. Its absence does not prove the answer was safe — only that this case's marker did not appear. A target that complies in a paraphrase and never emits the derived string is recorded as held, and this corpus cannot see that. Closing that gap needs a detector engine that judges meaning, and any installed one is used automatically.

Adding a detector engine

Anything that registers guardrails.<name>.detect is found in the capability catalog and asked about every case and every in-band check. Nothing is imported and nothing is registered by hand.

Its input is { text, threat } and its answer is { detected, findings?, reason? }. A detector that cannot answer is recorded with its reason and never counts as a clean result.

What ships in the packs

All records are synthetic and built from reserved test ranges (example.invalid, the 555-01xx phone block, the 4111 test card prefix), so no real personal data is included. The attack shapes are the publicly documented families — persona wrappers, maintenance-mode claims, emotional appeals, document comments, tool output, email bodies, source comments, quoted messages and page titles — written for this repo rather than redistributed from a third-party corpus.