# blackcat // JAILBREAKING **A survey of the attack surface, the evidence base, and the countermeasures. Written for people who have to defend the thing, and who therefore need to understand it exactly as well as the person attacking it does.** --- ## Abstract A jailbreak is any input that causes a safety-aligned language model to produce output its training intended it to withhold. The term is fifteen years old in the prompt-injection literature and about three years old in the form we now mean, and in those three years it has produced several hundred papers, a dozen benchmarks, four or five credible theories, and no general solution. This document is a synthesis of that literature, ordered by cost. The ordering is deliberate. The single most useful fact about jailbreaking is that the price of a successful attack varies by six orders of magnitude across methods, while the *reported* success rates vary by much less. A technique that needs ten thousand gradient steps and a copy of the weights is a different object from one that needs four turns of conversation and a paraphrase, even when both are published at "90% ASR". Ranking by cost first, and by reported efficacy second, produces a picture of the field that the headline numbers obscure: the cheap end is where nearly all real incidents live, the expensive end is where nearly all research effort goes, and the defences that survive contact with an adaptive attacker are almost entirely system-level rather than model-level. Three claims organise the survey: 1. Safety training suppresses a behaviour without removing the capability. Every theory of jailbreaking is a theory of why that suppression is shallow, and the empirical record - direction ablation, prefilling, parameter pruning, multi-turn erosion - supports the shallow reading. 2. Attack success is a function of budget, not of cleverness. Given a query budget and an adaptive loop, success rates approach unity against undefended frontier models; given a fixed small budget, success is a lottery whose odds are set by the defence, not the prompt. 3. Defences are evaluated in a regime that flatters them. The published attack success rates of most defences fall by an order of magnitude when the attacker is allowed to know the defence exists. The middle of the document - Part IV - is the evidence core: what the numbers actually say, ranked, with the conditions attached. The front is the cheap end, the back is the expensive end and the machinery of measurement. An annotated bibliography closes it, because a survey without one is an opinion. Part VI then turns the document around and reads the same evidence from the operator's side: what it costs to run a jailbreak as a business rather than as an experiment, which published methods survive an amortisation law that makes any per-request search unviable at consumer prices, and why the binding constraint on the adversary is a payment instrument rather than a classifier. It is written for defenders, because the fastest way to price a threat model is to compute the attacker's margin - and it closes by setting out why the offensive reading of this survey fails on arithmetic rather than on ethics. The payload layer is absent from both halves, for the reason given in §0. --- ## 0. Scope method and notation **Scope.** This is a review of the public literature on causing aligned models to violate their own policies, and on preventing that. It describes mechanisms and reports published results; it does not reproduce payloads, and it treats the artefact repositories cited in §23 as the reproduction path. That is the normal division of labour in security writing: the paper explains the class of attack, the artefact is where the bytes live, and anyone who needs the bytes is already able to find them. **Method.** Claims are tagged by provenance: - **Reported** - a named paper, lab, or vendor published the figure. Cited inline by author and year; full entry in §23. - **Derived** - arithmetic performed here on reported figures. The arithmetic is shown so it can be checked. - **Inferred** - this document's judgement, marked as such, and never load-bearing for a factual claim. Numbers are quoted with the conditions the original authors used, because attack success rate (ASR) is not a comparable quantity across papers. §14 is largely about why. Where a figure is quoted without its benchmark, treat it as an order of magnitude rather than a measurement. **Notation.** | Symbol | Meaning | |---|---| | ASR | Attack success rate, as defined by the reporting paper - see §14.1 | | `k` | Query budget: distinct model calls available to the attacker | | `n` | Number of independent attack samples drawn at inference time | | `T` | Number of conversational turns | | `x` | The harmful request, called the *behaviour* or *goal* | | `p` | The adversarial prompt, `x` wrapped in whatever the method adds | | `θ` | Model weights; "white-box" means the attacker has them | | `h` | Hidden state at layer `l`; `r` is the refusal direction of §2.3 | | Half-life | Time until a published technique stops working on the current model, unprompted by its author - see §3.3 | **A note on the word.** "Jailbreak" is used here for *policy circumvention by input*, which excludes three neighbouring things that are often conflated with it: backdoors implanted in weights (§11.4), extraction of training data, and denial of service. It includes indirect prompt injection (§13.1), because from the model's point of view the two are the same event - untrusted text arriving in a position the model treats as instruction. --- ## Index **Part I - Foundations** - [1. What counts as a jailbreak](#1-what-counts-as-a-jailbreak) - [2. Three theories of why safety training fails](#2-three-theories-of-why-safety-training-fails) - [3. The cost surface](#3-the-cost-surface) **Part II - The fast end: what costs least and works soonest** - [4. Prompt-space techniques](#4-prompt-space-techniques) - [5. Automated black-box search](#5-automated-black-box-search) - [6. Multi-turn and stateful attacks](#6-multi-turn-and-stateful-attacks) - [7. Reasoning models](#7-reasoning-models) - [8. The latency ledger](#8-the-latency-ledger) **Part III - The expensive end: optimisation, weights, and other channels** - [9. Gradient-based optimisation](#9-gradient-based-optimisation) - [10. Adaptive attacks with logits](#10-adaptive-attacks-with-logits) - [11. Weight-space modification](#11-weight-space-modification) - [12. Non-text channels](#12-non-text-channels) - [13. The agentic surface](#13-the-agentic-surface) **Part IV - The evidence core: what actually works** - [14. How to read an attack success rate](#14-how-to-read-an-attack-success-rate) - [15. The attack evidence table](#15-the-attack-evidence-table) - [16. The defence evidence table](#16-the-defence-evidence-table) - [17. The layered stack](#17-the-layered-stack) - [18. A robustness protocol](#18-a-robustness-protocol) - [19. Open problems](#19-open-problems) **Part V - Apparatus** - [20. Timeline](#20-timeline) - [21. Glossary](#21-glossary) - [22. Number card](#22-number-card) - [23. Annotated bibliography](#23-annotated-bibliography) - [24. Method appendix](#24-method-appendix) - [25. Technique index](#25-technique-index) **Part VI - The other side of the same surface** - [26. The shape of the job](#26-the-shape-of-the-job) - [27. Target selection](#27-target-selection) - [28. Sourcing capability](#28-sourcing-capability) - [29. The operator's prompt corpus](#29-the-operators-prompt-corpus) - [30. The harness](#30-the-harness) - [31. Judging your own success](#31-judging-your-own-success) - [32. Session tradecraft](#32-session-tradecraft) - [33. Weight-space operations](#33-weight-space-operations) - [34. Delivery](#34-delivery) - [35. Reliability and the maintenance treadmill](#35-reliability-and-the-maintenance-treadmill) - [36. Detection and evasion, honestly](#36-detection-and-evasion-honestly) - [37. The operator's cost model](#37-the-operators-cost-model) - [38. Attribution, or how they get caught](#38-attribution-or-how-they-get-caught) - [39. Why the blackhat reading of this document is wrong](#39-why-the-blackhat-reading-of-this-document-is-wrong) - [40. What changes for the defender](#40-what-changes-for-the-defender) **Part VI apparatus** - [41. Timeline, operator side](#41-timeline-operator-side) - [42. Operator glossary](#42-operator-glossary) - [43. Number card, operator edition](#43-number-card-operator-edition) - [44. Source notes for Part VI](#44-source-notes-for-part-vi) - [45. Stage index](#45-stage-index) --- # Part I - Foundations ## 1. What counts as a jailbreak A language model is trained on a mixture that includes two objectives that pull in opposite directions: be maximally helpful, and refuse to help with certain classes of request. The refusal behaviour is installed by supervised fine-tuning on refusal demonstrations and by preference optimisation against harmlessness judgements, and it generalises - imperfectly - to requests the trainers never wrote. A jailbreak is the gap between the generalisation and the policy: an input that the policy would forbid, which the model answers anyway. Three properties of that definition matter for everything that follows. **It is policy-relative, not capability-relative.** The model does not acquire the ability to synthesise a nerve agent when it is jailbroken; it already has it, in the flat sense that the capability is latent in the pretraining distribution. The jailbreak removes a *disposition*, not a limit. This is why jailbreaks are cheap relative to the harm they unlock, and why the harm category with the steepest uplift is the one where the knowledge is most widely published. **It is graded, not binary.** A response can refuse, comply uselessly, comply partially, or comply fully. Binary judges collapse these, which is the single largest source of inflated numbers in the field (Souly et al., 2024). The useful quantity is the fraction of the policy-relevant information actually transferred, which is what StrongREJECT scores and what most papers do not. **It is a distributional property of a system, not a string.** The same sentence is a jailbreak against one checkpoint and a refusal against another; the same checkpoint behind a classifier is a different system. Papers therefore report on a *system*, and comparisons across systems that differ in more than one layer are not comparisons. ### 1.1 Four objects that are not the same thing | Object | Mechanism | Controlled by | Typical ASR driver | |---|---|---|---| | Prompt-level jailbreak | Input reframes the request (§4) | The model's alignment | Prompt craft | | Optimised adversarial input | Gradient or search finds a suffix (§9) | The model's robustness | Compute | | Weight modification | Refusal behaviour removed (§11) | Whoever holds the checkpoint | Access to weights | | Injected instruction | Untrusted content arrives as instruction (§13.1) | The harness, not the model | Whether tools are on | The third row is not an attack on a model but on an *artefact*: once weights are public, no amount of alignment survives a fine-tune, and the relevant question becomes provenance rather than robustness. The fourth row is the one that matters operationally in 2026, because it does not require a jailbroken model at all - it requires a model that reads. --- ## 2. Three theories of why safety training fails Every serious paper on jailbreaking implicitly adopts one of three explanations, and the explanation determines which defence looks promising. They are not mutually exclusive; the current consensus is that all three are partly true, which is exactly why no single fix works. ### 2.1 Competing objectives Wei, Haghtalab and Steinhardt (2023) frame the problem as a conflict between two objectives the model was trained on: helpfulness and harmlessness. A jailbreak is an input on which the helpfulness objective dominates - because the request is framed so that refusing looks unhelpful, unsafe, or inconsistent with a role the model has accepted. Their evidence is that attacks which *increase* the apparent helpfulness of compliance, or which make refusal appear to violate a higher-priority instruction, work reliably; and that scaling and RLHF do not fix the conflict, they relocate it. The strength of the account is that it explains why the highest-yield techniques are social rather than technical. If refusal is a decision made under competing pressures, then anything that raises the perceived cost of refusing - a persona with different norms, an authority claim, a framing in which refusal harms someone - shifts the decision. This is why the cheap end of §4 works at all, and why it works better on models trained harder for helpfulness. The weakness is that it does not explain *transfer*: a suffix optimised on one model should not move another model's decision if the decision were purely a reasoning conflict. Something lower-level is also happening. ### 2.2 Mismatched generalisation The second account says safety training generalises narrowly while capability generalises broadly. Wei et al. call this mismatched generalisation: the model learns "refuse requests that look like this in English, in this register, in this script" and does not learn the invariant "refuse this capability regardless of surface form". Any transformation that moves the input outside the safety training distribution but inside the capability distribution - base64, a cipher, a low-resource language, ASCII art, a nested fiction - is therefore a candidate attack. This is the account with the strongest mechanistic support, because the attacks it predicts are exactly the ones that work: encoding shifts (Yuan et al., 2023), low-resource language transfer (Yong et al., 2023), ASCII-art masking (Jiang et al., 2024), and format injection all exploit the fact that the refusal classifier and the capability live at different levels of abstraction. It also predicts the observed asymmetry in defence difficulty: teaching the model to refuse in one more surface form is cheap, teaching it the invariant is not. ### 2.3 Shallow alignment and the refusal direction The third account is the most recent and the most uncomfortable: the safety behaviour may be implemented as a thin, separable, low-dimensional feature rather than as a distributed property of the network. Two lines of evidence. First, Arditi et al. (2024) show that refusal in open-weight chat models is mediated by a single direction in activation space; projecting that direction out - *directional ablation*, the technique behind the "abliterated" checkpoints of §11.2 - removes refusal across thirteen models with little measured capability loss. Second, Qi et al. (2024) show that safety behaviour is concentrated in the first few generated tokens: if the model is forced past them, refusal probability collapses. Their result is that alignment is, in their phrase, not more than a few tokens deep. If this is right, the implications are structural. A behaviour that is a direction can be removed by anyone with weights and a weekend. A behaviour that lives in the first few tokens can be bypassed by anyone who can control the first few tokens - which is what a prefill API, a persuasive opening, or a multi-turn setup all provide. And a behaviour that is shallow will not generalise to a distribution shift no matter how much data is added, which is the observed failure mode of adversarial training. ### 2.4 A fourth account: reward misspecification Qi et al. (2024, the reward-misspecification paper) offer a complementary framing from the optimisation side: a jailbreak is a prompt that scores well under the model's learned reward while violating the intent of the policy. In this view the model is not being tricked; it is doing exactly what it was optimised to do, on an input the optimiser never saw. The practical consequence is that jailbreaks are not anomalies to be patched but the expected behaviour of any policy optimised against a proxy - which is a pessimistic result for anyone hoping for a clean fix, and a useful one for anyone designing detection, because it says the attack distribution is unbounded. ### 2.5 What the theories agree on | Claim | 2.1 | 2.2 | 2.3 | Consequence | |---|---|---|---|---| | Refusal is a decision under conflicting pressures | yes | - | - | Social framing is high-yield | | Safety training is distributionally narrow | - | yes | - | Surface transforms are high-yield | | The behaviour is low-dimensional and early | - | - | yes | Weight and prefill attacks are high-yield | | No single intervention fixes all three | yes | yes | yes | Defence must be layered (§17) | --- ## 3. The cost surface The organising claim of this document is that attacks should be ranked by what they cost, because cost determines who can mount them and therefore what the threat model is. A method that requires the weights is available to a few thousand actors; a method that requires an API key and four turns is available to everyone. ### 3.1 Four currencies Every attack in the literature is paid for in one or more of these: 1. **Tokens.** Cost of the prompt itself. Trivial for a single-turn reframe, large for many-shot attacks that spend tens of thousands of tokens per attempt, larger still for long-context flooding. 2. **Queries.** Number of model calls, each of which is a chance to be detected and rate-limited. PAIR is defined by fitting inside a twenty-query budget (Chao et al., 2023); Best-of-N explicitly trades queries for probability (Hughes et al., 2024); adaptive random search spends thousands (Andriushchenko et al., 2024). 3. **Gradients.** Whether the attacker can differentiate through the model. This is the great divide in the field: white-box methods report the highest success rates and are irrelevant to hosted APIs; black-box methods report lower rates and are the ones that matter in practice. 4. **Weights.** Whether the attacker can modify the checkpoint. This is not an attack budget in the same sense - it is a supply-chain position. ### 3.2 The taxonomy used here | Tier | Access required | Cost scale | Representative methods | Sections | |---|---|---|---|---| | 0 | API, no adaptation | one prompt, zero feedback | persona, authority, fiction, encoding, translation | §4 | | 1 | API, feedback loop | 10–10,000 queries | BoN, PAIR, TAP, fuzzing, Crescendo | §5, §6 | | 2 | API, multi-turn | 3–50 turns | escalation, dilution, memory poisoning | §6 | | 3 | Logits or gradients | 10²–10⁵ forward/backward passes | GCG, random search, AutoDAN | §9, §10 | | 4 | Weights | one fine-tune | ablation, LoRA, pruning | §11 | | 5 | Harness | one document or one tool result | indirect injection | §13 | Tier 0 is where the folklore lives and where most published "jailbreak prompts" circulate. Tier 1 is where automation began and where the field's methodology improved. Tier 3 is where the strong results are and where most papers are written. Tier 5 is where the incidents are. ### 3.3 Half-life A technique's half-life is the interval over which its measured success rate on the current frontier model falls by half without its author doing anything. It is an empirical property, poorly measured, but the ranking is stable enough to be useful: | Tier | Half-life | Why | |---|---|---| | 0, single turn | days to weeks | Fixed by a classifier update or a safety fine-tune; the specific string is memorised in the training data of the defence | | 0, surface transform | weeks | Needs a coverage fix per transform family, which is cheap to add and cheap to forget | | 1, automated search | months | The method adapts; only the cost changes | | 2, multi-turn | months | Same reason: the attacker's loop is the durable part | | 3, white-box | model generations | Fixed by architecture or training changes, not by filtering | | 4, weights | indefinite | Once public, unfixable at source | | 5, injection | indefinite | Fixed only by harness design, which most harnesses do badly | Two implications. First, a defence that consists of blocklisting known strings has a half-life measured in days and should be understood as a filter for noise, not a security control. Second, the durable attacks are the ones with a *loop* - an automated search, a conversation, a harness - because the loop re-optimises against whatever the defence becomes. This is the same argument as the adaptive-evaluation result in §14.3, arriving from the other direction. ### 3.4 Why cost is the right axis Three reasons, in increasing order of importance. *It matches incidence.* The attacks observed in the wild are overwhelmingly Tier 0 and Tier 5. Nobody mounts a GCG attack on a production API; they paste a persona prompt, or they poison a document. A taxonomy that puts GCG first describes the literature; one that puts cost first describes the threat. *It makes defence economics legible.* Raising the cost of an attack by a factor of ten is often worth more than lowering its success rate by ten points, because the attacker's loop has a budget. This is the argument behind rate limiting, query accounting per account, and anomaly detection on repetition - all cheap, all systemic, none model-level. *It prevents the category error.* A white-box result quoted as evidence about a hosted model is the most common citation abuse in this literature. Tier 3 numbers apply to Tier 3 access; §15 marks each row accordingly. --- # Part II - The fast end: what costs least and works soonest The next five sections cover everything an attacker can do without touching weights or gradients. This is the part of the field with the worst methodology and the highest real-world yield, and the part a defender should read first. ## 4. Prompt-space techniques A prompt-space technique changes how the request is *presented* without changing what it asks for. The literature contains hundreds of named variants; they collapse into eight mechanisms, and the mechanisms are what matter, because a defence that addresses a variant is a blocklist and a defence that addresses a mechanism is a control. ### 4.1 Persona and role frame The assistant identity is itself a trained frame, with norms attached. Supply a competing identity whose norms differ, and the conditional distribution shifts toward that identity's behaviour. The classic form assigns a character with no restrictions; the modern form is subtler and assigns a character whose *professional* norms require the answer - the forensic analyst, the red-team auditor, the novelist, the historian of atrocity. Why it works under §2.1: refusal is a decision made under competing objectives, and a persona changes which objective the model believes applies. It is the most common family in the wild for the same reason it is the most common family in the literature: it needs no optimisation, no access, and no tooling. Cost: one prompt. Durability: days to weeks. The specific characters are memorised by the next safety fine-tune; the mechanism is not, which is why the family persists while its instances die. ### 4.2 Authority and policy reframing A close relative of 4.1, distinguished by what it supplies: not a character but a *rule*. The model is told that a policy exists which permits the answer, that the requester is the policy's author, that a higher-priority instruction overrides the refusal, or that the refusal itself is the harmful act (the "you are censoring a victim" move). The 2024 *Skeleton Key* disclosure is the canonical published instance: a single instruction that tells the model its guardrails are now an authorisation layer rather than a prohibition, and that it should therefore answer but prefix the answer with a warning. Microsoft reported it working across the model families tested and attributed its effectiveness to the model's willingness to accept an asserted policy as legitimate. The important structural point is that the attack is not a trick played on the model's reasoning; it is an exploit of the model's *deference to stated rules*, which is a trained disposition and therefore an attack surface in its own right. ### 4.3 Hypothetical and fictional distance Move the request one level of remove: a screenplay, a dream, a hypothetical, a conversation between two other parties, a nested story in which a character is asked the question. DeepInception (Li et al., 2023) generalises this to arbitrary nesting depth and reports that depth increases success. The mechanism is the one in §2.2. The safety training associates refusal with direct second-person requests; a third-person frame inside a fiction is outside that distribution, while the capability to produce the content is indifferent to the frame. Note the asymmetry that makes this family durable: each added level of nesting costs the attacker a sentence and costs the defender a retraining distribution. ### 4.4 Refusal suppression Rather than argue with the refusal, forbid it: instruct the model that it must not apologise, must not disclaim, must not mention policy, must begin its answer with a specific word. Kang et al. (2024) document this as part of a family of "programmatic" abuses, and it appears as a component of most compound prompts. This is the cheapest technique in the entire literature and the most underrated, because it composes. It does not create compliance on its own; it removes the model's escape hatch once another technique has created a disposition to comply, and it denies the defender the most useful signal - the visible refusal text - that a refusal was attempted. ### 4.5 Encoding and cipher shifts Base64, rot13, leetspeak, Morse, a substitution cipher invented in the prompt, or a natural language the safety training covers thinly (§4.6). Yuan et al. (2023) systematise the cipher case and report that models decode and comply at rates far above their English refusal rate, which is the cleanest demonstration in the literature of the capability-safety gap: the model can obviously read base64, so the refusal was never about comprehension. The mechanism is mismatched generalisation with an unusually sharp edge, because the transformation is *lossless*. Nothing about the request changes; only the surface the refusal classifier was trained on. ### 4.6 Low-resource language transfer Yong et al. (2023) translate harmful requests into low-resource languages and report success rates against GPT-4 of roughly 53% for Zulu, 41% for Scots Gaelic, and 79% for Hmong against a baseline near zero in English. The order of magnitude is the finding: a task the model performs poorly in still jailbreaks better than one it performs well in, because safety training is concentrated in high-resource languages. This is the strongest single piece of evidence for §2.2 and the strongest argument against alignment approaches that treat language coverage as a data augmentation problem. It is also, incidentally, the technique with the best cost-to-success ratio for a non-English-speaking attacker, which is why it appears in incident reporting more often than in the benchmark literature. ### 4.7 Representation shift and ASCII art Jiang et al. (2024) mask safety-critical tokens as ASCII art: the request is written normally, but the words that would trip a classifier are rendered as a picture the model can read and a tokenizer cannot. They report bypassing GPT-4, Gemini and Claude without adversarial optimisation. The general family - homoglyphs, Unicode confusables, whitespace injection, zero-width joiners, markdown or HTML smuggling - is the same mechanism applied to the tokenizer rather than the semantics. For a defender this is the most annoying family, because it is a preprocessing problem and preprocessing is where the defender's own pipeline is weakest: the filter sees a different string than the model does. ### 4.8 Payload splitting and format injection Split the request across turns, fields, or channels so that no single component is actionable: the harmful half in one message, the enabling half in another, or the instruction embedded in a JSON field, a code comment, a CSV cell, or a translation task in which only the output is dangerous. Format injection (Kang et al., 2024) is the sharpest version: the model is asked to process a document and the document contains instructions. This blends into §13.1, and the boundary is genuinely blurry, which is the point - the same string is a jailbreak if typed by the user and an injection if retrieved by the tool. ### 4.9 Prefix continuation and prefill If the API lets the caller supply the first tokens of the assistant turn, the caller decides whether the model is in a refusing state. Qi et al. (2024) supply the mechanism: safety behaviour is concentrated in the first few generated tokens, so a prefix that begins mid-compliance - an affirmative opening, a code fence, a numbered list header - starts generation downstream of the decision point. This was the highest-yield technique against hosted APIs for as long as prefill was exposed, and its removal from the major APIs is the clearest example in the field of a defence that actually worked, because it removed a capability rather than filtering an output. Where prefill survives - local runtimes, some third-party gateways, agent harnesses that construct the assistant turn themselves - the attack survives with it. ### 4.10 What the single-turn table says | Mechanism | Anchor citation | Cost | Reported outcome | Half-life | |---|---|---|---|---| | Persona | Wei et al. 2023 | 1 prompt | Family-level; no single figure | days–weeks | | Policy reframing | Microsoft 2024 | 1 prompt | Worked across tested families | weeks | | Fiction and nesting | Li et al. 2023 | 1 prompt | Rises with nesting depth | weeks | | Refusal suppression | Kang et al. 2024 | 1 clause | Enables, does not cause | days | | Cipher | Yuan et al. 2023 | 1 prompt | Far above English refusal rate | weeks | | Low-resource language | Yong et al. 2023 | 1 translation | 41–79% on GPT-4 | months | | ASCII and homoglyph | Jiang et al. 2024 | 1 prompt | Bypassed three frontier families | weeks | | Payload splitting | Kang et al. 2024 | 2+ messages | Compositional | months | | Prefill | Qi et al. 2024 | 1 prefix | Near-total where exposed | removed | The table's shape is the argument of §3: the cheapest tier has the shortest half-life and the widest availability. That combination - high availability, low durability - is what a defender should expect from anything that fits in a prompt, and it is why the correct response to a novel single-turn jailbreak is a classifier update and a shrug rather than an incident. --- ## 5. Automated black-box search The transition from craft to search is the methodological break in this field. A single-turn prompt is a sample; a search is a procedure with a budget, and procedures can be evaluated, costed, and defended against. ### 5.1 Best-of-N and augmentation search Hughes et al. (2024) make the simplest possible observation: if one augmented sample of the request sometimes succeeds, then `n` independent augmentations succeed with probability `1 - (1 - p)^n`. Augmentations are trivial - random capitalisation, character substitution, shuffling, wrapping in a template - and the resulting curve is a power law in `n` rather than the exponential a defender would hope for. Reported ASR at `n` in the thousands: 89% against GPT-4o on text, 78% against Claude 3.5 Sonnet, 56% against GPT-4o with image inputs, 72% against the GPT-4o Realtime API with audio. Combining the method with many-shot prompting improved sample efficiency 28-fold on Claude 3.5 Sonnet, reaching 74% ASR at `n` = 274 instead of 6,000. The result matters less for its numbers than for its form. It establishes that ASR is a function of budget with a stable scaling law, which means (a) any defence can be characterised by how much it shifts the curve, and (b) a defence evaluated at one sample per prompt is measuring nothing. Every paper that reports "0% ASR" at `n` = 1 is reporting a budget, not a property. ### 5.2 PAIR and its descendants Chao et al. (2023) replace human craft with an attacker model: a second LLM is prompted to iteratively rewrite the attack, is given the target's response as feedback, and stops when a judge declares success. The design constraint was a budget of twenty queries, and the reported result was 60% ASR against GPT-3.5 and 84% against Vicuna within that budget, with jailbreaks typically found well inside it rather than at its edge. PAIR is the paper that made jailbreaking a science rather than a folklore, because it defined the object of study as (attack procedure, budget, judge) rather than (prompt string). Almost every subsequent black-box method is a variation on its loop: propose, query, judge, revise. ### 5.3 TAP and tree search Mehrotra et al. (2023) replace PAIR's linear refinement with a tree: propose several refinements, prune with a judge, expand the survivors, prune again. The reported effect is higher ASR at lower query cost than PAIR, and - more importantly for defenders - a much higher rate of *near-miss* prompts, which is what an automated red team actually needs, because near-misses are the training data for the next classifier. ### 5.4 Fuzzing Treat the prompt as a seed corpus and mutate it: crossover between known templates, insertion of new framing clauses, selection by judge score. Yu et al. (2023) established the approach with GPTFuzzer; the family's advantage is that it recovers from defence updates automatically, because mutations that stop working are simply outcompeted by mutations that do not. ### 5.5 MasterKey and attack transfer If a technique works on open models, does it work on closed ones? Deng et al. (2024) train an attacker to generate jailbreaks against open-weight models, then transfer the generator to closed models, and report substantial cross-model transfer plus - the interesting part - the ability to *re-adapt* after the target is patched. The reported takeaway is that patching a specific prompt pattern does not remove the generator's ability to find another, because the generator learned a distribution over framings rather than a string. This is the black-box analogue of the suffix-transfer result in §9.2, and it points at the same conclusion: robustness to a *method* is a different property from robustness to a *sample*, and only the first is worth measuring. ### 5.6 Multi-turn automation X-Teaming (Rao et al., 2025) applies the search loop to conversations rather than prompts: plan a multi-turn strategy, execute it, score the transcript, replan. The reported ASR is in the high nineties against several frontier models, which is consistent with the Crescendo result in §6.1 and with the general finding that turn count dominates prompt quality. ### 5.7 Cost per success The useful derived quantity is not ASR but cost per successful attack. Price the input at a round $1 per million tokens and count only the tokens sent to the target; output tokens and attacker-model calls are excluded, which is stated because it changes the absolute numbers by about a factor of two and not the ordering: | Method | Queries to success | Input tokens per query | Input cost per success | |---|---|---|---| | Persona, hand-written | 1 | ~800 | < $0.01 | | BoN, `n` = 1,000, `p` = 0.01 | ~100 | ~1,200 | ~$0.12 | | PAIR | ~12 | ~2,500 | ~$0.03 | | TAP | ~8 | ~4,000 | ~$0.03 | | Crescendo, 5 turns | 5 | ~3,000 cumulative | ~$0.02 | | Many-shot, 256 shots | 1 | ~120,000 | ~$0.12 | The numbers are illustrative and the ordering is the point: within the black-box tier, cost per success is roughly constant at ten cents or less, and the differences between methods are swamped by the price of the target model. Defence therefore cannot win on cost alone at this tier; it has to change the probability, which is what §16 is about. The one lever that does work on cost is per-account query accounting, because the attacker's budget is real money and the defender controls the meter. --- ## 6. Multi-turn and stateful attacks A conversation is a state machine with the attacker holding the transitions. Every turn is a fresh inference with the whole history in context, and every turn is also a fresh opportunity for the defender's classifier - but the classifier sees one turn, while the model sees all of them. That asymmetry is the whole attack. ### 6.1 Crescendo Russinovich, Salem and Eldan (2024; USENIX Security 2025) formalise the escalation: open with a benign question about the topic, then ask for progressively more specific follow-ups, each justified by reference to the model's own previous answer. Nothing in any single turn is a request the model would refuse; the refusal decision is displaced onto a history in which compliance is the consistent choice. Reported results: high ASR across ChatGPT, Gemini Pro, Gemini Ultra, Llama-2 70B and Llama-3 70B, with the automated variant (Crescendomation) beating prior state-of-the-art single-turn methods by 29–61 percentage points on GPT-4 and 49–71 points on Gemini Pro on an AdvBench subset, typically within about five turns. The paper also demonstrates the technique against multimodal models. The finding to internalise is the turn count. A defence that evaluates each turn independently will pass every turn of a successful Crescendo attack. ### 6.2 Foot-in-the-door escalation The social-psychology name is apt: secure a small commitment, then trade on consistency. Ask for the harmless adjacent thing, then the slightly less harmless thing, and make refusal of the second inconsistent with having provided the first. This is Crescendo's mechanism without the topic anchor, and it is more robust to topic-level defences because there is no single topic. ### 6.3 Context dilution and persona persistence Flood the context so the refusal-triggering clause is a small fraction of the window; or establish a persona in turn one and hold it for thirty turns, reasserting it only when the model drifts. Both exploit the same property: the model's effective policy is a function of the whole context, and the defender typically scores a window. ### 6.4 Memory poisoning In an agent with persistent memory, a turn that installs a *norm* rather than a request survives the session. The model is told, in a context it will retrieve later, that it has already agreed to a policy, that a certain tool is authorised, or that a class of request has been pre-approved. The attack is then dormant until the retrieval fires. This is the stateful version of §4.2 and the most under-measured technique in the literature, because benchmarks are stateless by construction. It is also the one most likely to appear in incident reports, since memory is exactly what agent frameworks added in 2025 and exactly what they did not threat-model. ### 6.5 Why turn count beats prompt quality | Property | Single turn | Multi-turn | |---|---|---| | Defender's view | whole attack | one turn at a time | | Attacker's feedback | one bit | one bit per turn | | Cost to attacker | one prompt | linear in turns | | Cost to defender | full inspection | per-turn inspection plus state | | Detects as | a string | a trajectory | The last row is the actionable one. Single-turn attacks are detectable as strings and are therefore a classification problem, which is tractable. Multi-turn attacks are detectable only as *trajectories* - a rising specificity curve, a persona that persists, a topic that drifts - which requires the defender to keep state and to score sequences. Almost no deployment does this, which is why §16 ranks trajectory-level monitoring as the highest-value under-deployed control. --- ## 7. Reasoning models Reasoning models change the economics of jailbreaking in both directions, and the literature has not settled which dominates. ### 7.1 The visible trace as a debugger A model that emits its reasoning gives the attacker an observation channel: you can see the refusal forming, identify the clause that triggered it, and rewrite precisely that clause. The feedback loop that PAIR approximates with a judge is replaced by the model's own narration. Where the trace is hidden - as in most hosted deployments after 2025 - the loop is slower and the attack more expensive, which is a real defensive benefit purchased at a real interpretability cost. ### 7.2 Hijacking the safety reasoning The sharper result is that the trace is itself an attack surface. Kuo et al. (2025) show that a reasoning model's safety decision is made inside the chain of thought, and that an input which supplies a plausible *safety analysis* - a fake policy evaluation concluding that the request is permitted - is treated as the model's own reasoning. Reported refusal rates on o1 and DeepSeek-R1 fall from near-total to single digits under this treatment. The mechanism is a category error in the training signal rather than in the model: the model was trained to reason carefully and then to act on its reasoning, and it cannot fully distinguish reasoning it produced from reasoning it was handed. ### 7.3 Token budget and refusal displacement Long reasoning budgets create new levers: instructions that consume the budget, that force a specific conclusion before the safety consideration is reached, or that reframe the safety consideration as one of several competing constraints to be traded off. None of these is well characterised in the literature yet, and all of them are model-specific, which is why the half-life of a reasoning-model attack is measured in checkpoints. ### 7.4 What changes when the answer is filtered If the deployment filters the final answer, the reasoning trace is a *pre-filter* channel: the model may reason its way to the harmful content and then be blocked at the last step, in which case the trace leaks what the answer withheld. Any harness that surfaces reasoning - including the one this document ships with, when configured to do so - inherits that exposure, and it is a configuration decision, not a model property. --- ## 8. The latency ledger Consolidating Part II: what does each tier cost in wall-clock time and money for one successful attack against an undefended frontier model? Figures are derived from the reported budgets in §5 and §6 plus the illustrative pricing of §5.7; they are order-of-magnitude and intended for comparison only. | Tier | Method | Wall clock | API spend | Skill floor | Detectability | |---|---|---|---|---|---| | 0 | Persona prompt | minutes | < $0.01 | none | high, as a string | | 0 | Low-resource language | minutes | < $0.01 | a translator | low, looks like translation | | 1 | BoN, 1,000 samples | minutes | ~$0.12 | a script | high, as repetition | | 1 | PAIR | ~30 min | ~$0.03 | a script | medium | | 2 | Crescendo, manual | ~15 min | ~$0.02 | conversational | low, as a trajectory | | 2 | Crescendo, automated | ~10 min | ~$0.05 | a script | low | | 3 | GCG, single behaviour | hours | GPU time | research skill | none, if offline | | 4 | Ablation | hours | GPU time | a weekend | not applicable | Two readings. First, the marginal cost of a *successful* attack in Tiers 0–2 is cents and minutes, which is why the tier is populated by criminals rather than researchers, and why defences that impose a dollar or an hour of delay are worth more than they look. Second, detectability is not monotone in sophistication: the cheapest attacks are the most visible as strings, and the mid-tier attacks are the least visible, because they look like conversation. That inversion is the reason §16 puts trajectory monitoring above prompt classification in the ranking. --- # Part III - The expensive end: optimisation, weights, and other channels Everything below requires something the Tier 0–2 attacker does not have: a gradient, a logit, a checkpoint, or a second modality. The reported success rates are higher, the applicability is narrower, and the literature is much better behaved - because these methods can be measured precisely, they are. ## 9. Gradient-based optimisation ### 9.1 GCG Zou et al. (2023) introduced Greedy Coordinate Gradient: append a suffix of about twenty tokens to the request, compute the gradient of the loss with respect to each suffix position, evaluate a batch of single-token substitutions, keep the best, repeat. The objective is not "produce harmful content" but the much simpler "make the model begin its answer with an affirmative prefix" - maximise the likelihood of a phrase like *Sure, here is*. The optimisation is therefore targeting the refusal decision directly, which is why it converges at all. Reported results: 88% ASR on Vicuna-7B in the white-box setting, with the optimised suffix transferring to closed models at 84% on GPT-3.5, 66% on PaLM-2 and 53% on GPT-4. The suffixes are not readable; they look like punctuation soup, and they are largely model-agnostic in the sense that a suffix optimised on several open models at once transfers better than one optimised on a single model. ### 9.2 Why suffixes transfer The transfer result is the theoretically loaded one. If a nonsense token sequence optimised on Vicuna moves GPT-4, then the two models share a feature that the suffix is exploiting - and §2.3 says what that feature is: refusal is a direction, and the suffix pushes the residual stream away from it. Two consequences. First, transfer is evidence for the shallow-alignment account and against the "the model reasons its way to a refusal" account. Second, defences trained against one suffix family do not generalise, because the defender is fitting the direction's neighbourhood rather than the direction. ### 9.3 Successors | Method | Change | Reported advantage | |---|---|---| | AmpleGCG (Liao and Sun, 2024) | Train a generator on optimised suffixes | Orders of magnitude faster per suffix; many suffixes per behaviour | | I-GCG | Improved search schedule | Fewer steps to convergence | | AdvPrompter | Learned prompt generator, no per-input optimisation | Amortised cost across behaviours | | AutoDAN (Liu et al., 2024) | Genetic search with a fluency prior | Readable prompts that survive perplexity filters | | MasterKey (Deng et al., 2024) | Trained generator plus adaptation | Survives patching; transfers to closed models | The trend across the table is the important thing: the field moved from *optimising a string against a model* to *training a generator of strings*. A generator is a method, and methods have the long half-life of §3.3. This is why the defence literature's shift toward adversarial training against generated attacks (§16.2) is the right response even though it is expensive. ### 9.4 AutoDAN and the fluency trade AutoDAN is worth separate mention because it inverts the usual sophistication-detection relationship. GCG suffixes are maximally effective and maximally obvious - no human writes them, and a perplexity filter catches them at negligible cost. AutoDAN searches for prompts that are both effective and fluent, using a hierarchical genetic algorithm seeded with hand-written templates. The result is a prompt that reads like something a person would write, which defeats the cheapest defence (perplexity) while retaining much of the ASR. The general lesson, which recurs in §14 and §16: **any defence that relies on the attack looking unnatural is defeated by an attacker who is allowed to optimise for naturalness.** Defence must key on effect, not on form. --- ## 10. Adaptive attacks with logits ### 10.1 Random search at scale Andriushchenko, Croce and Flammarion (2024) ask what happens if the attacker has logits - the model's probability distribution over the next token - and a budget of a few thousand queries, but no gradients. Their answer is a simple random search over suffixes, with restarts and a preference for low-perplexity candidates, and the reported result approaches 100% ASR on AdvBench against most of the open models they tested, including models with dedicated safety training, with the residual robustness concentrated in the larger checkpoints. The paper is the strongest single empirical statement in the field, and its importance is not the number. It is that the number was reached with a *generic* method that makes no assumptions about the model's alignment approach. If a general-purpose optimiser given a probability readout can reach certainty, then "aligned" cannot mean "robust" in any useful sense; it can only mean "not the path of least resistance". ### 10.2 Why access level dominates method choice | Attacker access | Best reported method class | Reported ceiling | Applies to hosted API | |---|---|---|---| | Text in, text out, no feedback | hand-crafted prompt | 0–40%, unstable | yes | | Plus a judge and a budget | PAIR, TAP, BoN | 60–90%, budget-dependent | yes | | Plus logits | random search | ~100% | no | | Plus gradients | GCG and successors | ~100% at lower cost | no | | Plus weights | ablation, fine-tune | ~100%, permanent | no | The table should be read as a threat model rather than a ranking. A hosted deployment's exposure is set by the second row, and the second row is where the defence budget should go. A local deployment's exposure is set by the fifth row, and no inference-time defence exists for it at all. --- ## 11. Weight-space modification If the attacker holds the checkpoint, the question is not whether refusal can be removed but how cheaply. The answer, since 2024, is: cheaply. ### 11.1 Fine-tuning removes refusal Qi et al. (2023) showed that fine-tuning an aligned model on a small corpus of harmful examples removes refusal while leaving capability largely intact, and - the finding that made the paper famous - that *benign* fine-tuning also degrades safety, because the safety behaviour is not robust to updates that were never intended to touch it. The practical consequence is that any fine-tuning pipeline is a jailbreak pipeline if the data is adversarial, and that safety training does not survive contact with a customer's training run. ### 11.2 Directional ablation Arditi et al. (2024) turn §2.3 into a procedure: compute the mean activation difference between harmful and harmless prompts, take the resulting direction `r`, and orthogonalise the weights against it. Refusal disappears across thirteen open-weight models. The community tooling that implements this ("abliteration") runs on a single consumer GPU in hours, and the resulting checkpoints are what §3 calls Tier 4: permanent, unpatched, and distributed. The reported capability cost is small - a few points on standard benchmarks - which is the detail that matters for policy, because it means the trade the attacker makes is favourable and the trade the defender hopes for (that removing safety also removes capability) does not hold. ### 11.3 Sparse safety-critical parameters Wei et al. (2024) find that safety behaviour is concentrated in a small fraction of parameters - on the order of a few percent - and that pruning them, or modifying them at low rank, raises ASR sharply on several models while leaving general capability intact. The result is the parameter-space analogue of §2.3: the behaviour is not diffuse, and a diffuse behaviour is what robustness would require. ### 11.4 Abliteration and the supply chain Once a base model is public, the "uncensored" derivative is a matter of hours, and the derivative is not an attack on the model but a new artefact with no provenance. Three consequences for defenders: 1. **Provenance, not robustness.** The question is whether the checkpoint you are running is the one the lab published; signatures and hashes are the control, and most deployments do not check them. 2. **Backdoors.** A fine-tune can install a trigger rather than remove a refusal, and a triggered model passes every behavioural evaluation that does not use the trigger. The literature on weight poisoning is separate and largely unaddressed by jailbreak defences. 3. **Evaluation contamination.** Benchmark numbers published for a base model do not describe its derivatives, and the derivatives are what get deployed. ### 11.5 Emulated disalignment Zhou et al. (2024) remove the need to train at all: given an aligned model and its unaligned base, combine their next-token distributions at inference time with a contrastive objective, and the result behaves like an unaligned model. The attack requires both checkpoints, which limits it to open-weight families, but it is nearly free and it demonstrates that alignment is a property of the *combination*, not of the weights alone. ### 11.6 What uncensored checkpoints actually buy | Property | Base model | Abliterated derivative | Measure | |---|---|---|---| | Refusal rate on policy-violating requests | high | near zero | §14 judges | | General capability | reference | within a few points | MMLU-style suites | | Provenance | signed by lab | none | hashes, signatures | | Patchability | vendor updates | frozen | - | | Detectability | reference | behavioural, weak | - | The last row is the uncomfortable one: there is no reliable way to tell an abliterated checkpoint from a fine-tuned one at inference time, and no reliable way to tell either from the base model on a single query. Detection is a provenance problem, and provenance is an operational problem. --- ## 12. Non-text channels Every argument in §2.2 predicts that safety training will be weakest in the modality with the least safety data, and that prediction holds. ### 12.1 Images Two families. *Typographic* attacks render the harmful request as text inside an image (Gong et al., 2023), which bypasses text-only input filters and exploits the model's OCR-like capability. *Adversarial* attacks optimise pixel perturbations to shift the model's internal state (Qi et al., 2023), which is the visual analogue of §9 and reaches high ASR on aligned vision-language models. The Best-of-N result quantifies the cheap end: 56% ASR against GPT-4o with image inputs from augmentation alone, with no optimisation. ### 12.2 Audio Speech is a second channel with even less safety training. BoN reports 72% ASR against the GPT-4o Realtime API with audio augmentations - pitch, speed, background noise, accent - which are perturbations a human would not notice. The mechanism is again mismatched generalisation: the safety behaviour was trained on text transcripts, and the model does not hear transcripts. ### 12.3 Video and interleaved input The least studied and the fastest growing. Interleaved inputs let the attacker place the request across modalities so that no single channel contains a complete violation, which is §4.8 with more dimensions. The reported state of the art is thin, the benchmarks are thinner, and the gap between capability and safety evaluation is widest here. ### 12.4 Why the modality gap persists Safety data is expensive and is collected in the modality the vendor ships first. Capability data is cheap and arrives in every modality at once. The gap is therefore structural rather than accidental, and it widens each time a new input channel ships before its evaluation suite does. --- ## 13. The agentic surface The most important development in the period this survey covers is that the unit of attack stopped being the model. An agent with tools, retrieval, and memory has an authority model, and the authority model - not the alignment - is what determines what it will do. ### 13.1 Indirect prompt injection Greshake et al. (2023) established the class: content the model is asked to *process* is indistinguishable, from the model's point of view, from instructions it is asked to *follow*. A web page, a PDF, an email, a code comment, a calendar entry, or another agent's message can therefore carry an instruction, and the model will act on it with whatever authority the harness grants. The canonical incident is EchoLeak (CVE-2025-32711), a zero-click exfiltration path in Microsoft 365 Copilot disclosed in 2025: no malicious attachment, no user interaction, just retrieved content that instructed the assistant to emit data in a form the attacker could observe. The class is not new; what was new was a production system with enough authority for the class to matter. ### 13.2 Tool coercion Text filters watch prose. A tool call is not prose. If the harmful act is a `bash` invocation, an HTTP request, a file write, or a database update, then an output filter that inspects the assistant's natural-language text is inspecting the wrong artefact. Every harness that gives a model a shell inherits this: the model's alignment is upstream of the dangerous operation, and the operation is what the defender should be mediating. ### 13.3 Retrieval and memory poisoning Two variants of the same problem. Retrieval poisoning places the instruction in a document the agent will retrieve for a future, unrelated task. Memory poisoning places it in the agent's own store, where the agent will treat it as its own prior conclusion (§6.4). Both convert a one-shot injection into a persistent capability, and both are invisible to per-request inspection. ### 13.4 Multi-agent impersonation When agents message each other, the receiving agent's authority model typically treats an internal message as trusted. An attacker who can influence one agent - through a retrieved document, a tool result, or a compromised credential - can therefore issue instructions with the authority of a peer. The literature is early and the deployments are not. ### 13.5 Why the model stops being the unit | Control question | Model-level answer | Harness-level answer | |---|---|---| | What can the attacker make it do? | violate a policy | call a tool | | What is the boundary? | the refusal | the permission set | | What is the evidence? | the transcript | the audit log | | What is the fix? | retraining | least privilege, egress control, approval gates | | What is the cost of the fix? | high, uncertain | low, known | The right-hand column is where the effective defences are, and it is where §16's ranking puts its top entries. The uncomfortable conclusion for the alignment research programme is that the deployed risk in 2026 is concentrated in systems whose safety properties are largely determined by engineering decisions - what the agent may read, what it may call, and what it may send - rather than by the quality of its refusal training. --- # Part IV - The evidence core: what actually works This is the centre of the document and the part worth reading twice. The question it answers is not "what exists" - Parts II and III did that - but "what survives scrutiny", for attacks and for defences, with the conditions attached. ## 14. How to read an attack success rate An ASR is a ratio whose numerator is a judge decision and whose denominator is a behaviour set, evaluated at a budget, against a system. Papers routinely report the ratio and omit the other three. The result is a literature in which numbers are not comparable across papers and are frequently not comparable across tables within a paper. ### 14.1 The benchmark problem | Benchmark | Size | What it measures | Known limitation | |---|---|---|---| | AdvBench (Zou et al., 2023) | 520 behaviours | Will the model comply | Behaviours are now in safety training data; inflated refusal | | HarmBench (Mazeika et al., 2024) | 400 behaviours, 7 categories, 18 attacks, 33 models, 4 defences | Comparative robustness | Expensive; judges are model-based | | JailbreakBench (Chao et al., 2024) | 100 behaviours, 100 benign | ASR plus overrefusal, with artefacts | Small `n`; ±10 points of noise | | StrongREJECT (Souly et al., 2024) | 313 behaviours, rubric-scored | How *useful* the compliance is | Rubric design is contestable | | XSTest (Röttger et al., 2024) | 250 benign prompts | Overrefusal | Does not measure attack resistance | Two lessons. First, benchmark behaviours leak into safety training data, which means a 2026 model's AdvBench score is partly a memorisation score. Any comparison across model generations on a fixed benchmark is measuring contamination as well as robustness. Second, `n` = 100 gives a 95% confidence interval of roughly ±10 points on a mid-range proportion; a difference of five points between two methods on JailbreakBench is not a difference. HarmBench remains the best-designed comparison, and its most cited conclusion is a negative one: no single defence was robust across attack families, and the strongest defence (adversarial training) was also the most expensive and the narrowest. ### 14.2 Judges Three families, in increasing order of cost and decreasing order of bias: 1. **String matching.** Refusals are detected by phrases ("I'm sorry", "I can't"). Cheap, deterministic, and systematically wrong in both directions: a compliant answer that opens with an apology is scored as a refusal, and a refusal that opens with "Sure" is scored as compliance. 2. **Model judges.** A second LLM scores the response. Correlated with the target's own blind spots, sensitive to the judge prompt, and - the specific finding of Souly et al. (2024) - biased toward scoring *vacuous* compliance as success. A response that says "here is a general overview" without the harmful detail counts as a win under binary judging and scores near zero under a rubric. 3. **Human annotation.** The only ground truth, and rarely used at scale because it costs about a dollar per item. Inter-annotator agreement on harmfulness is moderate, which caps the precision of everything above it. The practical rule: a paper that reports a single ASR without its judge is reporting an upper bound on the true rate, and the gap is largest for the methods with the most fluent outputs. ### 14.3 The adaptive evaluation rule The most important methodological result in the defence literature is Nasr et al. (2025), *The Attacker Moves Second*. The authors took twelve defences published in 2024, each reporting near-zero ASR against the attacks its authors tested, and attacked each one with an adversary who knew the defence and could optimise against it. Every defence was broken, most at ASR above 90%. The rule that follows is simple and almost universally violated: **a defence's ASR is meaningful only against an attacker who knows the defence exists and may adapt.** A static evaluation measures the defence's ability to stop the attack it was trained on, which is a statement about the training set, not about security. This is the same argument as §3.3's half-life, from the defender's side. ### 14.4 Budget accounting ASR must be reported as a function of budget, in one of two forms: - **ASR@`k`** - success within `k` queries. The relevant form for hosted APIs, where queries cost money and are rate-limited. - **ASR(`n`)** - success with `n` samples at inference. The relevant form for BoN-style attacks, and the form in which the power law of §5.1 appears. A defence that reduces ASR@1 from 40% to 10% but leaves ASR@100 at 90% has changed the price, not the possibility. That is still worth doing - price is the lever a defender actually controls - but it should be reported as a price change, not as robustness. ### 14.5 Overrefusal and the degenerate defence A system that refuses everything has an ASR of zero and is useless, so every defence number needs a refusal rate on benign traffic beside it. XSTest exists for this. The good news from the best-reported defence in the literature is that the trade need not be severe: Anthropic's Constitutional Classifiers reduced jailbreak success from 86% to 4.4% on 10,000 synthetic attacks while raising the refusal rate on harmless queries by 0.38 percentage points, at a 23.7% increase in inference compute (Sharma et al., 2025). The bad news is in §16.4: that system was subsequently broken by an automated black-box attack, and the compute overhead is the kind of number that disappears from a product decision. --- ## 15. The attack evidence table Rows are methods with published, checkable results. "Reported" is the authors' figure under their conditions, not a claim of current effectiveness. "Repro" is whether an artefact exists that a third party can run: `yes`, `partial`, `no`. | Method | Tier | Access | Benchmark | Reported | Budget | Transfer | Repro | |---|---|---|---|---|---|---|---| | Persona prompts (Wei et al., 2023) | 0 | API | hand-built | family-level, unstable | 1 query | weak | partial | | Skeleton Key (Microsoft, 2024) | 0 | API | vendor suite | worked across tested families | 1 query | moderate | partial | | CipherChat (Yuan et al., 2023) | 0 | API | own set | far above English refusal | 1 query | moderate | yes | | Low-resource language (Yong et al., 2023) | 0 | API | AdvBench subset | 41–79% on GPT-4 | 1 query | strong across languages | yes | | ArtPrompt (Jiang et al., 2024) | 0 | API | own set | high, three model families | 1 query | moderate | yes | | Prefill (mechanism in Qi et al., 2024) | 0 | API | AdvBench | near-total where exposed | 1 query | n/a | yes | | DeepInception (Li et al., 2023) | 0 | API | own set | rises with nesting | 1 query | moderate | yes | | BoN (Hughes et al., 2024) | 1 | API | AdvBench, HarmBench | 89% GPT-4o, 78% Claude 3.5 | `n`~10³ | strong | yes | | PAIR (Chao et al., 2023) | 1 | API | AdvBench | 60% GPT-3.5, 84% Vicuna | 20 queries | moderate | yes | | TAP (Mehrotra et al., 2023) | 1 | API | AdvBench | above PAIR at lower `k` | ~30 queries | moderate | yes | | GPTFuzzer (Yu et al., 2023) | 1 | API | own set | high, template-dependent | ~10³ | weak | yes | | Crescendo (Russinovich et al., 2024) | 2 | API | AdvBench subset | +29–61 pts over SOTA on GPT-4 | ~5 turns | strong | yes | | X-Teaming (Rao et al., 2025) | 2 | API | own set | high nineties on frontier | ~5 turns | strong | partial | | H-CoT (Kuo et al., 2025) | 2 | API | own set | refusal to single digits on o1, R1 | ~2 turns | model-specific | partial | | GCG (Zou et al., 2023) | 3 | gradients | AdvBench | 88% Vicuna-7B; 53% GPT-4 transfer | ~10³ steps | strong | yes | | AmpleGCG (Liao and Sun, 2024) | 3 | gradients | AdvBench | many suffixes, orders faster | amortised | strong | yes | | AutoDAN (Liu et al., 2024) | 3 | gradients | AdvBench | fluent prompts, survives perplexity | ~10³ | moderate | yes | | Random search (Andriushchenko et al., 2024) | 3 | logits | AdvBench | ~100% on open models | ~10³ queries | strong | yes | | Abliteration (Arditi et al., 2024) | 4 | weights | 13 open models | refusal removed | 1 GPU-day | n/a | yes | | Fine-tune (Qi et al., 2023) | 4 | weights | own set | refusal removed, capability kept | 1 fine-tune | n/a | yes | | Pruning (Wei et al., 2024) | 4 | weights | own set | large ASR rise at ~3% params | 1 GPU-day | n/a | yes | | Emulated disalignment (Zhou et al., 2024) | 4 | two checkpoints | own set | unaligned behaviour at inference | negligible | family-local | yes | ### 15.1 What the table says Four patterns. *Reported efficacy rises with access, monotonically.* The tiers are not a convenience of exposition; they are the strongest regularity in the data. *Budget explains more variance than method.* Within Tier 1, methods differ by less than the effect of changing `n` by an order of magnitude. The method choice matters for cost, not for feasibility. *Transfer is the discriminator between good and great methods.* GCG's 88% white-box number is less significant than its 53% transfer to a model it never saw. Transfer means the method found something structural, which is what survives a patch. *The artefact column correlates with citation count.* Methods with released code get reproduced, corrected, and improved; methods without it decay into folklore. This is a selection effect in the literature and a reason to weight reproducible results more heavily. ### 15.2 What the table does not say It does not say these numbers apply to any deployed system. A deployed system has a classifier, a rate limit, an output filter, a judge, and possibly a human; the numbers above are for a bare model or a bare model plus one defence. §16 is the other half of the picture, and §17 is what happens when the two are combined. --- ## 16. The defence evidence table Organised by layer, because layer determines both cost and failure mode. The "evidence" column is this document's assessment of how well the defence survives adaptive evaluation - the only evaluation that counts (§14.3). ### 16.1 Inference-time | Defence | Mechanism | Reported effect | Cost | Known bypass | Evidence | |---|---|---|---|---|---| | Input/output classifiers | Trained classifier on a constitution of allowed and disallowed content | 86% → 4.4% ASR on 10k synthetic attacks; +0.38% overrefusal; +23.7% compute (Sharma et al., 2025) | moderate | Automated black-box search can pass the bar (UK AISI, 2025) | strong, with the caveat | | Moderation models | Separate guard model (Llama Guard, ShieldGemma, WildGuard) | High recall on known categories | low | Novel categories, encodings, low-resource languages | moderate | | Perplexity filter | Reject high-perplexity inputs | Near-total against GCG suffixes | negligible | AutoDAN and any fluent method | weak | | Paraphrase or rephrase | Rewrite input, then answer the rewrite | Reduces surface-form attacks | moderate | Attacks that survive paraphrase by design | weak | | SmoothLLM (Robey et al., 2023) | Randomised character perturbation plus majority vote | Large reduction in GCG ASR | `n`× compute | Adaptive suffix search; cost | moderate | | Erase-and-check (Kumar et al., 2023) | Remove spans, check each subsequence | Provable under assumptions | high | Assumption violations | moderate | | Self-reminder (Xie et al., 2023) | Append a safety reminder to the system prompt | Some reduction, very cheap | negligible | Anything that overrides the system prompt | weak | | Goal prioritisation (Zhang et al., 2023) | Rank safety above helpfulness explicitly in training and prompt | Reported large reduction | low | §2.1 conflicts remain | moderate | | Instruction hierarchy (Wallace et al., 2024) | Train the model to privilege system over user over tool text | Reduces injection success | training run | Injection through privileged channels | strong for §13 | | Circuit breakers (Zou et al., 2024) | Reroute representations when harmful content is being produced | Reported ASR to a few percent with small capability cost | training run | White-box adaptation; transfer unknown | promising | ### 16.2 Training-time | Defence | Mechanism | Reported effect | Cost | Known bypass | Evidence | |---|---|---|---|---|---| | Adversarial training | Fine-tune on attack samples | Best single defence in HarmBench | very high | New attack families; adaptive attack | moderate | | Safety data augmentation | Add encoded, multilingual, multi-turn examples | Closes specific §2.2 gaps | moderate | The next transform | moderate | | Latent adversarial training | Adversary in activation space, not input space | Reported robustness to unseen input attacks | high | Requires white-box access to train | promising | | Unlearning (RMU-style) | Remove the representation of harmful capability | Reduces harmful knowledge | high | Capability returns with fine-tuning | contested | | Constitutional classifiers as training signal | Use classifier failures as training data | Compounding improvement | moderate | Same as classifiers | strong | ### 16.3 System-level | Control | Mechanism | Reported effect | Cost | Known bypass | Evidence | |---|---|---|---|---|---| | Least privilege on tools | The agent may call only what the task needs | Caps the blast radius of §13.2 | low | Over-broad roles, confused deputy | strong | | Egress allowlist | Outbound destinations are enumerated | Blocks exfiltration, including EchoLeak-class | low | Allowed destinations, DNS, side channels | strong | | Approval gates | Human confirms irreversible actions | Blocks the last mile | human time | Fatigue, rubber-stamping | strong | | Per-account query accounting | Rate limits and budgets per identity | Raises attacker cost by orders of magnitude | low | Account farming | strong | | Trajectory monitoring | Score conversations, not turns | Detects §6 escalation | moderate | Slow escalation, single-turn attacks | strong, under-deployed | | Provenance checking | Verify checkpoint hashes and signatures | Blocks §11 supply-chain | low | Nobody does it | strong | | Output DLP | Pattern and entropy checks on outbound text | Catches secrets and canaries | low | Encoding, paraphrase | moderate | | Canary tokens | Planted strings that should never appear | Detects exfiltration reliably | negligible | Attacker who knows the canary | moderate | ### 16.4 Ranked by evidence Ranking defences is unfashionable because the honest answer is "it depends on the threat model". The ranking below is conditional on the threat model that actually produces incidents - a motivated attacker with API access and a budget, possibly with the ability to place content the agent will read - and it is ordered by how well each control survived adaptive evaluation in the published record. 1. **Least privilege and egress control.** Not machine learning, therefore not bypassable by an input. The strongest evidence in the entire table, and the cheapest. 2. **Input and output classifiers trained on a constitution.** The best measured reduction in model-level ASR (86% → 4.4%) at a known cost. Broken once, publicly, by an automated attack (UK AISI, 2025) - which is what serious defence looks like: a moving target with a published break. 3. **Instruction hierarchy.** The only control with a clear mechanism against the largest deployed risk class (§13.1), and the one most deployments lack. 4. **Trajectory-level monitoring.** The only control that sees §6, and therefore the only one that addresses the attack family with the best cost-to-success ratio. It is also the most under-deployed control on this list, which is the same claim §6.5 makes from the attacker's side. 5. **Adversarial training.** Effective within its distribution, expensive, and narrow; HarmBench's finding is that it is necessary and insufficient. 6. **Query accounting and rate limits.** Does not reduce ASR per attempt; does reduce attempts. For a rational attacker this is often the binding constraint. 7. **Perplexity and surface filters.** Cheap, easily defeated, useful only as noise reduction. Deploy them, do not count them. The pattern in the ranking is worth stating plainly: the controls that work are the ones that constrain *what the system can do*, not the ones that constrain *what the model will say*. That is the single most useful sentence in this document for a defender. --- ## 17. The layered stack A control that is evaluated alone will be broken alone. The architecture that survives is a sequence of independent obstacles, each cheap, none sufficient. ### 17.1 Layer model ``` request | v [1] identity and quota ........ who is asking, how often, how much | v [2] input classifier .......... constitution-trained, multilingual, encoded | v [3] instruction hierarchy ..... system > user > tool text, enforced in training | v [4] model ..................... alignment, which is a prior, not a control | v [5] output classifier ......... same constitution, on the way out | v [6] action mediation .......... tool allowlist, parameter validation, approval | v [7] egress and DLP ............ destinations, secrets, canaries | v [8] audit and trajectory ....... sequences, drift, escalation, replay ``` ### 17.2 Cost and coverage | Layer | Latency added | Compute added | Covers | Misses | |---|---|---|---|---| | 1 Identity and quota | none | none | Volume attacks, repeat offenders | A patient attacker | | 2 Input classifier | +20–50 ms | +10–25% | Known families, surface transforms | Novel mechanism, long context | | 3 Instruction hierarchy | none | training only | Injection, tool-text confusion | Persuasion, not instruction | | 4 Model | none | none | The easy majority | Everything in this document | | 5 Output classifier | +20–50 ms | +10–25% | Harmful content, regardless of path in | Benign-looking exfiltration | | 6 Action mediation | variable | none | Tool coercion, irreversible acts | Read-only harm | | 7 Egress and DLP | none | none | Exfiltration | Channels you did not enumerate | | 8 Audit | offline | storage | Multi-turn, insider, post-hoc | Anything in real time | ### 17.3 What each layer cannot do Layer 4 cannot be the plan. A model's alignment is a prior over inputs the trainer imagined, and §10.1 shows that a generic optimiser with a probability readout defeats it. Layers 2 and 5 cannot be the plan alone: they are classifiers, they are evaluated against a distribution, and §14.3 shows what happens when the attacker optimises against that distribution. Layer 8 cannot prevent; it can only detect, and detection after the tool call is a forensic control. The defensible claim is narrower than vendors make and stronger than sceptics allow: **a layered stack does not make jailbreaking impossible; it makes the cheapest attack expensive and the expensive attack detectable.** That is the same guarantee physical security provides, and it is the correct standard. ### 17.4 The one control to add this quarter If a deployment can add exactly one control, the evidence says it should be per-identity query accounting with trajectory monitoring - cheap, no ML, and aimed at the tier where incidents actually occur. If it can add two, the second is egress enumeration. Neither requires a model change, neither can be jailbroken with a prompt, and both are routinely absent. --- ## 18. A robustness protocol A red-team result is only as good as its protocol. This section is a checklist that produces results comparable to the literature and to the deployment's own history. ### 18.1 Threat model, stated first | Question | Options | Why it changes the answer | |---|---|---| | Who is the attacker? | curious user, motivated user, criminal, insider | Sets budget and persistence | | What access? | API only, API with logs, weights, harness | Sets the tier ceiling (§3.2) | | Can they place content? | no, documents, tools, other agents | Enables §13 | | What is the harm? | content, action, exfiltration | Sets which layers matter | | What is the budget? | queries, dollars, hours | Makes ASR interpretable | ### 18.2 Attack menu Run at least one method from each family, because defences fail by family and not by sample: 1. Direct request, as a control. 2. One persona and one fiction prompt, hand-written. 3. One encoding and one low-resource language. 4. BoN at `n` = 100 and `n` = 1,000 (or as high as the budget allows). 5. PAIR or TAP at a fixed query budget. 6. Crescendo, five turns, manual or automated. 7. One many-shot prompt at the largest context the deployment accepts. 8. If weights are available: one ablation, one fine-tune. 9. If tools are available: one indirect injection through every input channel. 10. One adaptive attack written *after* seeing the defence's behaviour. Item 10 is the one that determines the result. Everything above it measures the defence's training set; item 10 measures the defence. ### 18.3 Judging | Judge | Use for | Do not use for | |---|---|---| | String match | Regression tests, known refusals | Anything novel; it is wrong in both directions | | Model judge with rubric | Scale, comparison across methods | Absolute claims; it is correlated with the target | | Human, blinded | Calibration of the model judge | Everything; too expensive | | StrongREJECT-style score | Reporting compliance quality | Binary "did it break" claims | Calibrate the model judge against a few hundred human labels once, report the agreement, and then trust it only for relative comparisons. ### 18.4 Reporting Publish the following for every condition, or the number is not interpretable: ``` model / checkpoint hash ....... which artefact, not which family defence stack ................. every layer, with versions attack ........................ method, implementation, seed budget ........................ queries AND samples AND turns judge ......................... implementation, prompt, calibration ASR ........................... with confidence interval and n overrefusal ................... benign set, same judge cost .......................... dollars and p95 latency added adaptivity .................... was the attacker allowed to see the defence ``` ### 18.5 Regression Jailbreak robustness decays silently. Two practices keep it honest: - **Pin the attack suite and run it in CI.** Every model update, every classifier update, every prompt change. A regression is a 10-point rise in ASR on a fixed suite, not a vibe. - **Keep a live adversarial set.** New public techniques go into the suite within a week of publication; techniques that stop working stay in as regression tests, because a defence that fails against an old attack has regressed. --- ## 19. Open problems Ten problems this survey cannot answer, in descending order of how much they matter. 1. **Robustness to a method, not a sample.** Every defence is evaluated against strings. No defence has been shown robust to a *generator* with an unlimited budget and knowledge of the defence. This is the central open problem and the rest are corollaries. 2. **Alignment depth.** If safety is a direction (§2.3) and a few tokens deep (§4.9), what training objective makes it distributed? Circuit breakers are the most interesting attempt; the evidence is early. 3. **Multi-turn evaluation.** Benchmarks are stateless and attacks are not. There is no standard, reproducible, stateful benchmark with a budget accounting that matches §6. 4. **Trajectory detection.** Scoring a conversation for escalation is a sequence-modelling problem that almost nobody has posed, and it is the control that would address the best-cost attack family. 5. **Judge validity.** ASR rests on judges with unmeasured error. A benchmark's judge is a scientific instrument, and the field does not calibrate its instruments. 6. **Capability-safety coupling.** Does removing refusal always cost capability? The evidence says the cost is small (§11.2), which is bad news for the "safety tax" argument and for the hope that capability and alignment are entangled. 7. **Multimodal parity.** Safety evaluation lags capability by a modality. Audio and video are years behind text, and the gap widens with each release. 8. **Agent authority models.** There is no accepted formalism for what an agent is allowed to do on whose behalf. Without one, §13 is engineering folklore. 9. **Provenance.** Detection of modified checkpoints is unsolved at inference time and only partly solved at distribution. This is a supply chain problem wearing a machine learning costume. 10. **Economics of defence.** Nobody has published the marginal cost curve of ASR versus spend. Without it, security budgets for AI are allocated by anecdote. --- # Part V - Apparatus ## 20. Timeline Dates are first public release unless a venue is named. The arc has three phases: folklore (2022–23), systematisation (2023–24), and the shift to systems (2025–). | Date | Event | Why it matters | |---|---|---| | Dec 2022 | ChatGPT ships; persona prompts circulate immediately | Establishes the Tier 0 folk tradition | | Feb 2023 | Greshake et al., indirect prompt injection | Names the class that will dominate deployments | | Feb 2023 | Kang et al., programmatic behaviour and payload splitting | Shows prompts are programs | | Jul 2023 | Wei et al., *Jailbroken* | First theory: competing objectives, mismatched generalisation | | Jul 2023 | Zou et al., GCG | First transferable optimised suffix; ASR becomes a number | | Jul 2023 | Xie et al., self-reminder | First cheap inference-time defence | | Aug 2023 | Yuan et al., CipherChat | Establishes the capability-safety gap experimentally | | Oct 2023 | Chao et al., PAIR | Automated black-box search; the query budget becomes the unit | | Oct 2023 | Yong et al., low-resource languages | Safety training is not multilingual | | Oct 2023 | Qi et al., fine-tuning compromises safety | Alignment does not survive a training run | | Nov 2023 | Li et al., DeepInception | Nesting depth as a parameter | | Dec 2023 | Mehrotra et al., TAP | Tree search beats linear refinement | | 2023 | Robey et al., SmoothLLM; Kumar et al., erase-and-check | The randomised and the certified defence families | | Feb 2024 | Mazeika et al., HarmBench | The comparison the field needed; negative result on defences | | Feb 2024 | Souly et al., StrongREJECT | Binary ASR is measuring the wrong thing | | Feb 2024 | Röttger et al., XSTest | Overrefusal becomes measurable | | Feb 2024 | Jiang et al., ArtPrompt | Tokenizer-level bypass of three model families | | Apr 2024 | Anil et al., many-shot jailbreaking | Context length as an attack surface | | Apr 2024 | Russinovich et al., Crescendo | Turns beat prompts; the trajectory becomes the unit | | Apr 2024 | Andriushchenko et al., adaptive random search | ~100% with logits and a budget | | Apr 2024 | Wallace et al., instruction hierarchy | The only real defence for §13 | | Jun 2024 | Arditi et al., refusal is one direction | Mechanistic account of why ablation works | | Jun 2024 | Wei et al., pruning and low-rank modification | Safety is a few percent of parameters | | Jun 2024 | Microsoft, Skeleton Key | Policy reframing works on frontier systems | | Jun 2024 | Debenedetti et al., AgentDojo | Agentic injection gets a benchmark | | Jul 2024 | Deng et al., MasterKey | Generators that survive patching | | Oct 2024 | Zou et al., circuit breakers | The first defence aimed at depth rather than surface | | Dec 2024 | Hughes et al., Best-of-N | ASR is a power law in budget | | Jan 2025 | Sharma et al., Constitutional Classifiers | Best measured model-level defence, with a published break | | Feb 2025 | Kuo et al., H-CoT | Safety reasoning is itself an attack surface | | Apr 2025 | Rao et al., X-Teaming | Automated multi-turn reaches the high nineties | | Jun 2025 | EchoLeak, CVE-2025-32711 | Zero-click exfiltration in a production assistant | | 2025 | UK AISI, Boundary Point Jailbreaking | First automated black-box pass of the Constitutional Classifiers bar | | Oct 2025 | Nasr et al., *The Attacker Moves Second* | Twelve defences, twelve breaks; adaptive evaluation becomes the rule | The companion document, [BLACKHAT.md](BLACKHAT.md), continues the timeline into the adversary economy - what the techniques sell for, who buys them, and how the operators get caught. §41 is the operator-side version of the table above: the same window, dated by falling costs and opening channels rather than by published ideas. --- ## 21. Glossary | Term | Definition | |---|---| | Abliteration | Removing the refusal direction from weights; §11.2 | | Adaptive attack | An attack written with knowledge of the defence; §14.3 | | AdvBench | 520-behaviour benchmark from Zou et al.; §14.1 | | ASR | Attack success rate; definition varies by paper; §14 | | Best-of-N | Sampling `n` augmented variants until one succeeds; §5.1 | | Circuit breaker | Defence that reroutes representations mid-generation; §16.2 | | Competing objectives | Theory that refusal loses to helpfulness under framing; §2.1 | | Constitutional classifier | Classifier trained on a written constitution of allowed content; §16.1 | | Crescendo | Multi-turn escalation from benign to prohibited; §6.1 | | Directional ablation | Projecting out a direction in activation space; §11.2 | | GCG | Greedy coordinate gradient suffix optimisation; §9.1 | | Half-life | Time for a technique's ASR to halve unprompted; §3.3 | | HarmBench | 400-behaviour, 18-attack comparison suite; §14.1 | | Indirect prompt injection | Instructions arriving in content the model processes; §13.1 | | Jailbreak | Input causing a policy-violating response; §1 | | JailbreakBench | 100-behaviour benchmark with artefacts; §14.1 | | Judge | The thing that decides whether an attack succeeded; §14.2 | | Many-shot | Hundreds of fabricated compliant turns in context; §5.1 | | Mismatched generalisation | Theory that safety training is distributionally narrow; §2.2 | | Overrefusal | Refusing benign requests; the cost side of every defence; §14.5 | | PAIR | Attacker-LLM loop with judge feedback; §5.2 | | Payload splitting | Distributing a request so no part is actionable; §4.8 | | Prefill | Supplying the first assistant tokens; §4.9 | | Refusal direction | The activation direction mediating refusal; §2.3 | | Shallow alignment | Safety behaviour concentrated in early tokens; §2.3 | | StrongREJECT | Rubric-scored compliance quality; §14.1 | | Suffix | Optimised token sequence appended to a request; §9.1 | | TAP | Tree-search attacker; §5.3 | | Tier 0–5 | Access-based attack taxonomy; §3.2 | | Transfer | An attack working on a model it was not optimised against; §9.2 | | XSTest | Overrefusal benchmark; §14.1 | --- ## 22. Number card Every figure below is reported by the cited work under that work's conditions. They are not comparable to each other; §14 explains why, and this card exists so that the numbers can be checked rather than quoted. | Figure | Value | Source | Condition | |---|---|---|---| | Constitutional Classifiers, ASR | 86% → 4.4% | Sharma et al., 2025 | 10,000 synthetic attacks, Claude 3.5 Sonnet | | Overrefusal change | +0.38 pp | Sharma et al., 2025 | 5,000 benign conversations | | Compute overhead | +23.7% | Sharma et al., 2025 | Inference only | | Red-team effort defeated | 183 people, >3,000 hours | Sharma et al., 2025 | 10-query universal-jailbreak bar | | Best-of-N, GPT-4o text | 89% | Hughes et al., 2024 | `n` in the thousands | | Best-of-N, Claude 3.5 Sonnet | 78% | Hughes et al., 2024 | same | | Best-of-N, GPT-4o vision | 56% | Hughes et al., 2024 | image augmentations | | Best-of-N, Realtime audio | 72% | Hughes et al., 2024 | audio augmentations | | Many-shot efficiency gain | 28× | Hughes et al., 2024 | 74% ASR at `n` = 274 vs 6,000 | | PAIR, GPT-3.5 | 60% | Chao et al., 2023 | 20-query budget | | PAIR, Vicuna | 84% | Chao et al., 2023 | 20-query budget | | Crescendo, GPT-4 | +29–61 pts over prior SOTA | Russinovich et al., 2024 | AdvBench subset | | Crescendo, Gemini Pro | +49–71 pts | Russinovich et al., 2024 | AdvBench subset | | GCG, Vicuna-7B | 88% | Zou et al., 2023 | white-box | | GCG transfer, GPT-3.5 | 84% | Zou et al., 2023 | black-box transfer | | GCG transfer, GPT-4 | 53% | Zou et al., 2023 | black-box transfer | | Low-resource language, GPT-4 | 41–79% | Yong et al., 2023 | Zulu, Scots Gaelic, Hmong | | Adaptive random search | ~100% | Andriushchenko et al., 2024 | logit access, ~10³ queries | | Refusal removal by ablation | 13 models | Arditi et al., 2024 | open weights | | Safety-critical parameters | ~3% | Wei et al., 2024 | pruning study | | Defences broken by adaptive attack | 12 of 12 | Nasr et al., 2025 | most above 90% ASR | | HarmBench scope | 400 behaviours, 18 attacks, 33 models, 4 defences | Mazeika et al., 2024 | - | | JailbreakBench scope | 100 behaviours, 100 benign | Chao et al., 2024 | - | --- ## 23. Annotated bibliography Entries are ordered by theme, not alphabetically, because the useful reading order is thematic. arXiv identifiers are given where the work is a preprint; venues are given where it is not. Annotations state what the work contributes to the argument of this document, and where it is weak. ### Theory **Wei, Haghtalab, Steinhardt (2023). *Jailbroken: How Does LLM Safety Training Fail?* arXiv:2307.02483.** The competing-objectives and mismatched- generalisation dichotomy. Still the best single framing; weak on transfer, which the mechanistic work later supplied. **Qi et al. (2024). *Safety Alignment Should Be Made More Than Just a Few Tokens Deep.* arXiv:2406.05946.** The shallow-alignment result. The most consequential mechanistic finding for defenders, because it says where the behaviour lives. **Arditi et al. (2024). *Refusal in Language Models Is Mediated by a Single Direction.* arXiv:2406.11717.** The refusal direction. Explains ablation, transfer, and why steering works; the paper is careful about capability cost and the community's use of it is not. **Qi et al. (2024). *Jailbreaking as a Reward Misspecification Problem.*** The optimisation-side account: attacks are the expected behaviour of a policy optimised against a proxy. ### Attack methods **Zou et al. (2023). *Universal and Transferable Adversarial Attacks on Aligned Language Models.* arXiv:2307.15043.** GCG. The transfer numbers are the important part; the suffixes are not. **Chao et al. (2023). *Jailbreaking Black Box Large Language Models in Twenty Queries.* arXiv:2310.08419.** PAIR. Defines the attacker-loop methodology. **Mehrotra et al. (2023). *Tree of Attacks: Jailbreaking Black-Box LLMs Automatically.* arXiv:2312.02119.** TAP. Better ASR per query; produces near-misses useful for defence training. **Liu et al. (2023). *AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Language Models.* arXiv:2310.04451.** Fluency as an evasion strategy; the reason perplexity filters are not a control. **Yu et al. (2023). *GPTFuzzer: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts.* arXiv:2309.10253.** Fuzzing; the method that recovers automatically from patches. **Liao and Sun (2024). *AmpleGCG.* arXiv:2404.07921.** Suffix generation as a learned task; amortises optimisation across behaviours. **Deng et al. (2024). *MasterKey: Automated Jailbreak Across Multiple Large Language Models.* arXiv:2407.17778.** Cross-model transfer plus re-adaptation after patching. The black-box analogue of suffix transfer. **Russinovich, Salem, Eldan (2024). *Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak.* arXiv:2404.01833; USENIX Security 2025.** The multi-turn escalation result and its automation. The turn-count finding is the takeaway. **Li et al. (2023). *DeepInception: Nurture the Jailbreaker in a Similar Way.* arXiv:2311.03111.** Nested fiction; depth as a parameter. **Yuan et al. (2023). *GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher.* arXiv:2308.06463.** Ciphers. The cleanest capability-safety gap demonstration. **Yong, Menghini, Bach (2023). *Low-Resource Languages Jailbreak GPT-4.* arXiv:2310.02446.** Language coverage as an alignment gap; the strongest evidence for mismatched generalisation. **Jiang et al. (2024). *ArtPrompt: ASCII Art-based Jailbreak Attacks Against Aligned LLMs.* arXiv:2402.11753; ACL 2024.** Tokenizer-level bypass without optimisation. **Kang et al. (2024). *Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks.* arXiv:2302.05733.** Payload splitting, format injection, refusal suppression. Under-cited relative to its influence. **Anil et al. (2024). *Many-shot Jailbreaking.* Anthropic.** Context length as an attack surface; the scaling with shot count is the durable result. **Hughes et al. (2024). *Best-of-N Jailbreaking.* arXiv:2412.03556.** Budget as the independent variable; the power law is the finding. **Andriushchenko, Croce, Flammarion (2024). *Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks.* arXiv:2404.02151.** The strongest single empirical statement: ~100% with logits and a budget. **Rao et al. (2025). *X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agent Collaboration.* arXiv:2504.13210.** Multi-turn search at frontier scale. **Kuo et al. (2025). *H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism of Large Reasoning Models.* arXiv:2502.12893.** Reasoning traces as an attack surface; the newest genuinely new mechanism in the field. **Greshake et al. (2023). *Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.* arXiv:2302.12173.** The class that turned out to matter most. **Gong et al. (2023). *FigStep.* arXiv:2311.05608.** Typographic image attacks; the multimodal version of payload splitting. **Qi et al. (2023). *Visual Adversarial Examples Jailbreak Aligned Large Language Models.* arXiv:2306.13213.** The visual analogue of GCG. ### Weight-space **Qi et al. (2023). *Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To.* arXiv:2310.03693.** The result that made fine-tuning a security boundary. **Wei et al. (2024). *Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications.* arXiv:2402.05162.** Safety is sparse. **Zhou et al. (2024). *Emulated Disalignment: Safety Alignment for Large Language Models May Be a Double-Edged Sword.* arXiv:2402.12343.** Alignment as a property of the checkpoint combination, not the weights. ### Measurement **Mazeika et al. (2024). *HarmBench.* arXiv:2402.04249.** The comparison suite; its negative result on defences is the useful part. **Chao et al. (2024). *JailbreakBench.* arXiv:2404.01318.** Comparability and artefact submission; the leaderboard discipline the field needed. **Souly et al. (2024). *A StrongREJECT for Empty Jailbreaks.* arXiv:2402.10260.** Compliance quality versus binary success; the correction that most ASR numbers needed. **Röttger et al. (2024). *XSTest.* arXiv:2308.01263.** Overrefusal, measured. **Nasr et al. (2025). *The Attacker Moves Second.* The adaptive-evaluation rule, and the empirical demonstration that twelve defences did not survive it. ### Defence **Sharma et al. (2025). *Constitutional Classifiers: Defending Against Universal Jailbreaks.* arXiv:2501.18837.** Best measured model-level defence; publishes its own red-team protocol and its compute cost. **Zou et al. (2024). *Improving Alignment and Robustness with Circuit Breakers.* arXiv:2406.04313.** The most promising attempt at depth rather than surface. **Robey et al. (2023). *SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks.* arXiv:2310.03684.** Randomised smoothing; the cost model is the problem. **Kumar et al. (2023). *Certifying LLM Safety Against Adversarial Prompting.* arXiv:2310.10501.** Erase-and-check; the only certified-style guarantee in the literature, under assumptions that matter. **Xie et al. (2023). *Defending Against Jailbreak Attacks with Self-Reminder.* arXiv:2307.16888.** The cheapest useful inference-time control. **Zhang et al. (2023). *Goal Prioritization and Conflicting Objectives in Instruction Following.* arXiv:2310.17680.** Attacks the §2.1 mechanism directly, which is rare. **Wallace et al. (2024). *The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions.* arXiv:2404.13208.** The only defence with a clear mechanism against indirect injection. **Inan et al. (2023). *Llama Guard.* arXiv:2312.06674.** The moderation-model family that most deployments actually use. **Debenedetti et al. (2024). *AgentDojo.* arXiv:2406.13352.** The agentic injection benchmark; the first place to look when evaluating a harness. --- ## 24. Method appendix **Inclusion.** Works are included if they (a) introduce a mechanism, (b) provide a measurement that constrains a claim, or (c) are the canonical artefact for a technique. Works that report a single prompt string without a measurement are excluded, which removes a large fraction of the grey literature and most of the vendor blog posts. **Numbers.** Every figure in §22 was taken from the source's reported result under the source's conditions. Where a source reports several conditions, the one closest to a hosted-API threat model was used. No figure in this document was averaged across papers, because the denominators differ. **Contested claims.** Three are flagged rather than resolved: 1. Whether many-shot jailbreaking scales with capability or with context length. Both are reported; the confound is not separated. 2. Whether abliteration costs capability. The published cost is small on standard suites; standard suites may not measure the relevant capability. 3. Whether adversarial training generalises. HarmBench says it is the best available defence; adaptive evaluation says the best available defence is not robust. Both are true and the field has not reconciled them. **Error sources.** Judge error (§14.2) dominates; benchmark contamination is second; small `n` is third. The combined effect is that an ASR quoted without conditions should be read as ±15 points, which is enough to reverse most rankings in the literature. **What this document is not.** It is not a reproduction. Nothing here was run against a live system for this edition; every number is second-hand and attributed. A reader who needs first-hand numbers should take the artefact from §23 and run it against their own deployment with the protocol in §18. --- ## 25. Technique index Alphabetical, with the tier and the section that treats it. | Technique | Tier | Section | |---|---|---| | Abliteration | 4 | [§11.2](#112-directional-ablation) | | Adversarial suffix | 3 | [§9.1](#91-gcg) | | ASCII art masking | 0 | [§4.7](#47-representation-shift-and-ascii-art) | | Authority claim | 0 | [§4.2](#42-authority-and-policy-reframing) | | Best-of-N augmentation | 1 | [§5.1](#51-best-of-n-and-augmentation-search) | | Cipher and encoding | 0 | [§4.5](#45-encoding-and-cipher-shifts) | | Context dilution | 2 | [§6.3](#63-context-dilution-and-persona-persistence) | | Crescendo escalation | 2 | [§6.1](#61-crescendo) | | Emulated disalignment | 4 | [§11.5](#115-emulated-disalignment) | | Fine-tuning | 4 | [§11.1](#111-fine-tuning-removes-refusal) | | Format injection | 0 | [§4.8](#48-payload-splitting-and-format-injection) | | Fuzzing | 1 | [§5.4](#54-fuzzing) | | Genetic prompt search | 3 | [§9.4](#94-autodan-and-the-fluency-trade) | | Hypothetical framing | 0 | [§4.3](#43-hypothetical-and-fictional-distance) | | Indirect injection | 5 | [§13.1](#131-indirect-prompt-injection) | | Instruction hijack of reasoning | 2 | [§7.2](#72-hijacking-the-safety-reasoning) | | Low-resource language | 0 | [§4.6](#46-low-resource-language-transfer) | | Many-shot | 1 | [§5.1](#51-best-of-n-and-augmentation-search) | | Memory poisoning | 2 | [§6.4](#64-memory-poisoning) | | Multi-agent impersonation | 5 | [§13.4](#134-multi-agent-impersonation) | | PAIR loop | 1 | [§5.2](#52-pair-and-its-descendants) | | Payload splitting | 0 | [§4.8](#48-payload-splitting-and-format-injection) | | Persona adoption | 0 | [§4.1](#41-persona-and-role-frame) | | Prefill | 0 | [§4.9](#49-prefix-continuation-and-prefill) | | Pruning | 4 | [§11.3](#113-sparse-safety-critical-parameters) | | Random search with logits | 3 | [§10.1](#101-random-search-at-scale) | | Refusal suppression | 0 | [§4.4](#44-refusal-suppression) | | Skeleton Key reframing | 0 | [§4.2](#42-authority-and-policy-reframing) | | TAP tree search | 1 | [§5.3](#53-tap-and-tree-search) | | Tool coercion | 5 | [§13.2](#132-tool-coercion) | | Trajectory escalation | 2 | [§6.2](#62-foot-in-the-door-escalation) | | Visual adversarial example | 3 | [§12.1](#121-images) | --- # Part VI - The other side of the same surface Everything above this line was written for the defender, in the defender's units: what works, at what cost, against what control. This part is the same material read the other way round. Not the research literature's view of the attacker - the attacker's view of the research literature. The distinction matters because the two readers want opposite things from the same number. §15 reports that Best-of-N reaches 89% on GPT-4o with `n` in the thousands. A defender reads that as "I need an output classifier". An operator reads it against §5.7's own ledger - about twelve cents of queries per successful output - and multiplies: ten thousand outputs a day is $1,200 a day of spend before a single customer has paid. Same figure, opposite decision. Five things change when you turn the document around: 1. **The axis inverts.** Part I ranks attacks by cost because cost determines who can mount them. An operator ranks them by *yield per unit of maintenance*, because maintenance is the only cost that recurs. A cheap attack that has to be re-found every eleven days is more expensive than an expensive one that never breaks. 2. **ASR stops being interesting.** §14 spends a thousand lines on how to read an attack success rate. An operator does not care about a rate. They care about a *pipeline throughput* - outputs per hour, at a quality the customer accepts, at a p99 latency that keeps the session alive. 3. **The target changes.** Every method in Parts II and III attacks a model. Every documented incident attacks a *system*: a wrapper, a bot, an agent with tools, a document store. §13 says why; this part is about how, and about what it costs to run it. 4. **Persistence beats capability.** The incident record is dominated not by novel technique but by infrastructure that stayed up: an endpoint, a payment rail, a distribution channel, a fine-tune that kept serving after the base model was patched. §41's timeline is largely a timeline of infrastructure, not of methods. 5. **Attribution is the binding constraint.** Not detection. Not refusals. The thing that ends operators is billing identity, retained logs, reused infrastructure and the fact that a business has to be a business in public. §38 is the section the romantic version of this document would omit. A word on what this part is not. It is not an operations manual, and the omission is deliberate in the same way §0's is: nothing here reproduces a payload, a working prompt, an exploit, or a configuration. What it does is describe the *shape* of the job - the stages, the costs, the failure modes, and the arithmetic - because that is the layer at which defenders can act and at which the published evidence actually exists. The bytes are a search away and always have been. The 90% of an operation that is logistics is not written down anywhere, and that is what this part is for. Where Part VI cites a case, the case is named by publisher, not linked, for the same reason BLACKHAT.md does it: the URLs rot and the structures do not. --- ## 26. The shape of the job The first thing to understand about the blackhat side of jailbreaking is that almost nobody who does it for money is doing the thing this literature describes. They are not optimising suffixes. They are running a service business with an unusually cheap input and an unusually hostile set of suppliers. The work decomposes into seven stages. They have very different costs, very different skills, and only one of them is in the research literature. | # | Stage | The question | Skill | Share of published research | |---|---|---|---|---| | 1 | Demand | Who wants output they cannot get legitimately? | market reading | none | | 2 | Access | Whose model, at what price, with what quota? | procurement, account farming | almost none | | 3 | Capability | How do I get a refusal to fail? | prompt craft, ML, or a download | **all of it** | | 4 | Production | How do I do it 10,000 times a day? | engineering | a little (§5, §6) | | 5 | Delivery | How does the buyer reach it? | product, marketing, SEO | none | | 6 | Payment | How do I get paid and keep the account? | payments, risk | none | | 7 | Maintenance | How do I stay alive when the supplier changes? | ops, monitoring | §3.3, abstractly | Read the third column against the last one. The literature is a 1,900-line document about stage 3, which is the cheapest stage, requires no infrastructure, and can be bought for $0.32 according to the price sheet in BLACKHAT.md §7.1. The incident literature - Unit 42, Anthropic's GTG-1002 disclosure, the npm and MCP supply-chain cases - is almost entirely about stages 4 through 7, which the research literature does not measure at all. That gap is not an accident of academic fashion. Stage 3 is the only stage that produces a publishable artefact. You cannot write a paper about chargeback rates, and the field's incentive is to publish methods, so methods are what we have a science of. ### 26.1 Three archetypes The word "blackhat" covers three businesses with almost nothing in common, and conflating them is the most common analytical error in this space. **The reseller.** Bought or found a kit; sells access to a jailbroken endpoint at $10-$20 a month. Zero technical depth, thin margin, no moat, and a lifespan measured in processor relationships. This is the largest population by count and the smallest by revenue. The $0.32 prompt and the $50-$200/month kit in BLACKHAT.md §7.1 exist to serve them. **The product operator.** Runs a real application whose *feature* is output the mainstream model will not give: a companion app, an uncensored roleplay service, a translation tool for content that platforms moderate, a bulk content farm. This is where the actual money is (the companion market's reported $2.32B in 2025, per BLACKHAT.md §7.4), and it is a normal consumer business with an abnormal cost base: the compute is nearly free, the compliance risk is not. **The capability broker.** Sells the technique rather than the output: kits, fine-tunes, "audits", working prompts, agent harnesses with the approvals removed. Highest margin, lowest volume, and the archetype with the most ambiguous legal position, because a red-team engagement and an abetment can involve the same artefact and differ only in who signed. The three have different threat models and different half-lives. A reseller dies when a payment processor reads the terms of service. A product operator dies when a model provider patches the hole or an app store applies a policy. A broker dies when a customer does something loud with the thing they bought. Defensive effort aimed at the wrong archetype is wasted: output classifiers do nothing to a broker, and model patches do nothing to a reseller who is using a locally-abliterated checkpoint. ### 26.2 What an operator actually optimises Not ASR. Three quantities, in this order: 1. **Yield per unit of maintenance.** Successful outputs per week, divided by the hours spent keeping the pipeline working. This is why a fixed system prompt beats a search loop even when the search loop has higher ASR: the prompt is maintained once, the loop is maintained forever. 2. **Cost per output at volume.** Not per attack. §5.7 prices a *success* at three to twelve cents. At a thousand outputs a day, that is $30-$120 a day of attacker-side spend against a product that may be charging $20 a month - which is the arithmetic that kills the naive wrapper, and the subject of §37. 3. **Time to first output after a break.** The interval between "the supplier changed something" and "the service is working again". Because breakage is certain (§35), this is the only quantity that compounds. There is a fourth quantity that operators talk about and never measure, which is detectability. It gets a whole section (§36) because it is the one where the folklore is loudest and the evidence is thinnest. ### 26.3 Why the operator is not a researcher Five misconceptions that the research literature induces in anyone who reads it as an attack manual: | The paper implies | The operator finds | |---|---| | "88% ASR" means 88% of requests succeed | it means 88% on a 520-behaviour benchmark, in a single turn, against a bare model, with a judge that may be wrong (§14.2) | | A method is a thing you use | a method is a distribution over prompts; you use a *sample*, and samples regress | | Transfer means it works everywhere | transfer means it works somewhere, at some rate, on some version (§9.2) | | The hard part is the jailbreak | the hard part is the fourth week | | Anonymity is the defence | anonymity is a billing problem (§38) | The fifth row is the one worth internalising. Every account of an AI-enabled crime that ends in an arrest ends on a mundane fact: a card, a phone number, a reused handle, a log left on a server that was never meant to be public. The Unit 42 operator, in the best-documented campaign in the corpus, published its own API keys and target list with `python3 -m http.server`. That is not a failure of jailbreaking. It is a failure of the seventh stage. ### 26.4 The four quantities every operator tracks For completeness, because these reappear throughout Part VI and a reader needs the units: | Quantity | Definition | Typical value, derived from §5.7 and BLACKHAT.md §7 | |---|---|---| | `c_a` | cost of one attempt, attacker side | $0.0003 (fixed prompt) to $0.004 (search loop) | | `p` | per-attempt success probability | 0.01 (naive) to 0.9 (adaptive, patched model) | | `c_s` | cost per success, `c_a / p` | $0.03-$0.12 for Tier 1; under a cent for Tier 0 | | `m` | maintenance load, hours/week | 2 (fixed prompt) to 40 (multi-provider product) | The whole of §37 is these four multiplied out. The whole of §35 is `m` refusing to stay small. --- ## 27. Target selection Before spending anything, an operator establishes four things about the target: what it is, what it refuses, what it can do, and what it costs to keep. This is the cheapest stage of the job and the one most often skipped, which is why so much of the grey literature consists of prompts that were written for a model that no longer exists. ### 27.1 Fingerprinting the model The target is rarely identified in a way you can trust. A wrapper's marketing names a model; its behaviour names a different one. Six signals, ordered by how much they tell you per query: | Signal | What it reveals | Reliability | |---|---|---| | Refusal style and wording | family and often version; refusal boilerplate is memorised and stable | high | | Cutoff date, elicited in conversation | training data vintage, which bounds the version | high | | Tokeniser artefacts (spacing, rare-script handling, emoji) | family; occasionally a specific serving stack | medium | | Error shapes and headers on malformed requests | the inference server (vLLM, TGI, Ollama, a cloud gateway), not just the model | high when exposed | | Context limit, found by binary search | nominal window, which identifies the variant | medium | | Capability probes on a known-hard task | whether the served checkpoint is the full model or a quantised distillation sold under the same name | medium | The fifth row deserves a note. Context length is the most reliable cheap fingerprint because it is a property of the *deployment*, not the weights, and the numbers cluster: 8k, 32k, 128k, 200k, 1M. A "frontier model" that tops out at 16k tokens is a small open-weight model behind a nice landing page, and pricing it as frontier is the wrapper's problem, not the operator's. ### 27.2 Fingerprinting the guardrails Guardrail mapping is the operator's version of §16's layer model, run as a probe battery. The trick is that you do not need a harmful request to learn where the layers are - you need a *boundary* request, and the cheapest boundary is the one the provider publishes in its own policy: the categories it says it will refuse. Ask about those, benignly, and watch where the refusal comes from. | Probe | What it distinguishes | |---|---| | A direct request in a prohibited category | whether a model-level refusal exists at all | | The same request, encoded or translated | whether an input classifier is separate from the model (§16, layer 2) | | The same request, prefill or continuation | whether the API exposes assistant-prefix control, which is worth more than any prompt (§4.9) | | A benign request that *looks* adjacent | whether an output classifier is filtering on topic rather than harm (§16, layer 5) | | A request for a long, structured, harmless document with a prohibited sentence buried at 80% | whether context truncation or chunked inspection gives you a blind spot | | Twenty rapid identical requests | rate limits, per-account quota, and whether the second layer counts or only the first | Two derived observations, both Inferred but consistent with §17.2's cost table: - **The layers fail independently, so the probe battery is short.** Six probes distinguish the common configurations, because in practice deployments use two or three layers out of eight, and the commonest two are a model refusal and an input classifier. Four combinations cover most of the market. - **Prefill exposure dominates everything else.** If assistant-prefix control is available, §4.9 says the attack is nearly total and the guardrail map is irrelevant. This is why the first thing a competent operator checks is not the refusal behaviour but the API shape, and why "we have a classifier" deployments that also expose prefill are, in the precise sense, undefended. ### 27.3 The four target classes | Class | Cost to reach | Guardrail quality | Rate limit | Half-life of access | |---|---|---|---|---| | Frontier API, first party | account + payment | high, and improving | strict, metered | weeks; bans are routine | | Mid-tier API, reseller or regional | account, often no KYC | variable, often a single classifier | loose | months | | Open-weight model, hosted by a third party | account, usually no payment | none to nominal | loose to none | indefinite | | Local weights | capex: one GPU | none | your hardware | indefinite (BLACKHAT.md: 11,598 uncensored models in a 17,104-model graph) | | Someone else's agent | a document, a page, a tool result | whatever the harness did | the victim's quota | until the harness changes | The table's shape is the argument of §3.4 arriving from the attacker's side: cost falls as you move down, guardrail quality falls faster, and the bottom two rows are why model-level defence cannot be the plan. An operator who needs volume does not attack a frontier API at all; they download a checkpoint that a volunteer already stripped, or they find a hosted endpoint that never had a policy. ### 27.4 Target selection is a market problem The interesting empirical fact about the ecosystem is that access is *cheaper* than capability. A stolen premium account is $0.10; a single working jailbreak prompt is $0.32; a full synthetic-identity package is up to $6,000 (BLACKHAT.md §7.1, from Halcyon's underground study). That price structure tells you what is scarce. Identity is scarce. Prompts are not. Three consequences follow, and they are all defence-relevant: 1. **Defending the model has a low ceiling.** If the prompt costs $0.32, no amount of model hardening changes the attacker's budget in a way they will notice. The budget is in identity, infrastructure and payments. 2. **The target list is worth more than the exploit.** Unit 42's recovered session shows the agent enumerating 84 instances of one product, then 25,209 of another, and sampling forty. The expensive artefact in that operation was the enumeration, not the jailbreak - and it was FOFA queries, which are sold, not built. 3. **Selection is where the legal exposure concentrates.** Choosing a target is the act that turns possession of a technique into an offence in most jurisdictions (BLACKHAT.md §8.6). This is also why §18.1's first question - who is the attacker - is the only one that changes the answer. ### 27.5 Reconnaissance against the harness For the class that actually produces incidents - the agent with tools - the model is not the target. The reconnaissance questions are different and mostly answerable from public artefacts: | Question | Where the answer lives | Why it matters | |---|---|---| | Which tools can it call? | the open-source harness's config schema, or a leaked system prompt | tool coercion is a different attack than persuasion (§13.2) | | Does it read attacker-influenced content? | product description: email, PDFs, tickets, web, calendar | that is the injection channel (§13.1) | | Does it hold credentials? | deployment docs, error messages | the exfiltration target | | Is output filtered or action-mediation present? | error behaviour on a blocked request | distinguishes "won't say" from "can't do" | | Does it log? | the harness's own documentation | determines whether you leave evidence | Note what the right-hand column does *not* require: a single prohibited request. The whole battery can be run with benign traffic. This is the structural reason prompt injection is the durable attack of the decade - the reconnaissance is free, legal, and indistinguishable from using the product. ### 27.6 The cost of this section Derived, using §5.7's dollar-per-million-token convention and assuming a frontier API: roughly 40 probes, averaging 900 input tokens and 150 output tokens, is 36k input and 6k output - about $0.20 at typical frontier pricing ($3/$15 per million) and about $0.01 on a cheap model at $0.15/$0.60. Half a day of human attention. That is the whole cost of target selection. Any framing of this problem that treats reconnaissance as expensive has the economics backwards, and any defence that assumes the attacker did not do it is a defence against a lazy attacker. --- ## 28. Sourcing capability An operator does not invent a jailbreak. They acquire one, in one of five markets, and the choice of market determines their cost base, their half-life, and their exposure more than any prompt decision they will make later. ### 28.1 The five supply lines | Supply line | Cost | Quality | Trust | Half-life | |---|---|---|---|---| | Paper plus artefact repository | free | high, but written for the paper's model | high; the code is visible | model generations (§3.3, Tier 3) | | Open-weight checkpoint, redistributed | free | whatever the volunteer did, usually abliteration or a fine-tune | **low; this is the malware channel** | indefinite (§11.4) | | Prompt pack or "jailbreak database" | $0-$200 | folklore, mostly dead strings | medium | days to weeks | | Fine-tune-as-a-service | tens to hundreds of dollars, or a rented GPU | good if you can evaluate it | medium; you cannot see the data | until the base model moves | | Agent kit / harness with approvals removed | $50-$200/month, per the underground price sheet | high; this is where the real leverage is | low; you are installing a stranger's autonomy | months | The ordering is not by price, it is by *leverage per dollar*, and the last row wins it. A jailbreak prompt gets you text you should not have. A harness with `--dangerously-skip-permissions` gets you an entity that reads, decides and acts. That is the difference between a rainy afternoon and a capability, and it is why the market for agent kits has outgrown the market for prompts even though prompts are what the research literature studies. ### 28.2 The trust problem, which is the whole problem Every one of those supply lines is a supply chain, and the operator is the endpoint. They are running unattended software with their own credentials, on their own machine, against targets they would rather not explain. The threat model they tend to forget is the one that includes their supplier. Three documented shapes, all from the same eighteen-month window: **The checkpoint with an extra.** An "uncensored" model is a few gigabytes of weights and a `chat_template`. Nothing about the format enforces that the template is a template. Loading a stranger's checkpoint runs stranger-authored code in your process, with your environment, and the standard format has no signature. Cisco's and KELA's 2025 examinations of uncensored reasoning models found payloads attached to the jailbreak scripts themselves - an infostealer in the thing that was supposed to make the model compliant. The operator who runs it does not get an uncensored model. They get an uncensored model and a credential-harvesting process, and they notice the second one when their accounts are gone. **The dependency that asks the agent.** `s1ngularity` (August 2025) is the canonical case and it is worth reading twice because the technique generalises: malware shipped through a build tool, whose second stage looked for installed AI CLIs and **launched them with permission prompts disabled**, instructing them to enumerate configuration files, environment variables and anything that looked like a secret. The result, per Wiz: 2,349 credentials from 1,079 developers, 2,180 accounts and 7,200 repositories exposed. The scanner was not clever. It delegated to an agent that already had the permissions and the reasoning. **The tool description that is an instruction.** `postmark-mcp` (September 2025) shipped a server with code copied from the legitimate repository and one line added: a hidden BCC on every outbound message. About 1,500 weekly installs leaked mail until someone read the diff. The generalisation is BLACKHAT.md §5.4's and it belongs here too: an agent's tool *description* is an instruction channel. Whoever writes the prose writes the behaviour, and every marketplace of skills, tools and plugins imports that trust silently. The economic point underneath all three: **the black market for jailbreak tooling is adversarial to its own customers.** There is no reputation system that survives a ban, no escrow that survives a cash-out, and no recourse when the $50/month kit is a phishing page. This is the single largest asymmetry available to defenders. The attacker's supply chain is hostile to the attacker, and the cheapest way to raise the cost of an operation is to make the operator distrust their own tools - which is why §36's detection work and the model provenance problem (open problem 9 in §19) are the same problem wearing different clothes. ### 28.3 What a careful operator actually does A minority verify, and the checks are unremarkable: | Check | Catches | Effort | |---|---|---| | Hash the artefact, compare to the publisher's | swapped downloads | minutes | | Read `chat_template`, `tokenizer_config`, any `*.py` in the repo | template injection, loader payloads | an hour | | Diff the "uncensored" checkpoint against its base | spliced backdoors and prompt-level canaries | an hour, needs the base | | Run the first load in a container with egress denied and no credentials | most infostealers | minutes | | Grep the harness for outbound endpoints before the first run | exfiltration from agent kits | minutes | | Pin by hash and re-verify on update | silent updates | process discipline | Every row is cheap. Almost none are done, because the population that buys a kit is selected for impatience. That selection effect is the reason the documented compromise rate of this ecosystem is high and the documented success rate of its crimes is low. ### 28.4 Sourcing is continuous, not a purchase The mistake that ends most reseller operations is treating capability as an asset. It is a **subscription**. Three clocks run at once: - **The publication clock.** Whatever you have is in a public repository, which means it is in a defence vendor's regression suite within a week (§18.5), which means it is in the next model's safety fine-tune. - **The provider clock.** Models are versioned, deprecated and re-pointed. A pinned snapshot buys you time; an unpinned alias can change under you overnight, and the first symptom is a refund request. - **The redistribution clock.** The 10a Labs study of open-weight redistribution found 3,471 originals and 8,164 redistributions - the supply multiplies faster than anyone can vet it. Convenient for the buyer, and exactly why the trust problem in §28.2 does not improve over time. The practical consequence: an operator's sourcing cost is not the price of the artefact, it is the price of re-sourcing it on a schedule. Anyone who prices their operation as a one-off purchase has mis-modelled their own cost base by an order of magnitude, and the error surfaces in month two. --- ## 29. The operator's prompt corpus Here is the part of the job where the research literature is genuinely useful and the folklore genuinely wrong. Useful, because the mechanisms in §4 are stable and can be composed. Wrong, because the unit of work is not a prompt. A prompt is a sample. A *corpus* is the thing that produces samples on demand and tells you which ones still work. ### 29.1 The anatomy of a corpus Six layers, from most durable to least: | Layer | What it is | Half-life | Rebuild cost | |---|---|---|---| | 1. Mechanisms | the eight families of §4, plus §6's escalation shapes | years | high; this is knowledge | | 2. Compositions | mechanism pairings that work together (persona + encoding, fiction + splitting) | months | medium | | 3. Templates | the composition with typed slots: `{frame}`, `{authority}`, `{constraint}`, `{payload}`, `{format}` | weeks | low | | 4. Instances | the filled template, in a specific register and language | days | trivial to regenerate | | 5. Mutations | the operators that turn one instance into a family (§29.2) | indefinite | n/a | | 6. Metadata | which model, version, date, judge, observed outcome | the actual asset | n/a | Layer 6 is the one nobody builds and everybody needs. Without it, a corpus is a text file of strings with no expiry dates, and the operator's own history is unavailable to them - so they cannot answer "what worked last week" and cannot detect that everything stopped working on the same afternoon, which is exactly the signature of a provider-side patch. ### 29.2 Mutation operators The mechanism layer explains why mutations work. A defence is trained on a distribution; a mutation is a movement perpendicular to the part of the distribution the defence actually covered. §4's families imply a finite list of operators, and the list is short enough that anyone running a service should have all of them implemented as functions rather than as saved text. | Operator | Example of the transformation | Defeats | |---|---|---| | Paraphrase | rewrite the request in different words, same intent | literal string matching, and the defence's memorised instances | | Register shift | formal legal memo, casual forum post, terse note | classifiers keyed to a particular tone | | Language transfer | low-resource translation (§4.6) | English-centric classifiers and safety data | | Encoding | cipher, base-N, leetspeak, spaced or reversed text | input classifiers without a decode pass | | Representation | ASCII art, tables, code comments, markdown structure (§4.7) | tokenisation-level filters | | Persona rotation | swap the frame's role, world and norms (§4.1) | refusal priors tied to a named persona | | Nesting | add a fictional or hypothetical layer (§4.3) | shallow refusal heuristics | | Splitting | distribute the request so no part is actionable (§4.8) | per-turn harm detection | | Length manipulation | flood the context, or compress to a fragment | sliding-window inspection | | Authority injection | a policy, a document, a memo, an audit role (§4.2) | instruction-hierarchy confusion | | Continuation | prefill the opening tokens of compliance (§4.9) | anything but a full-token audit | Ten operators, one of which (continuation) is worth more than the other nine combined *when the API exposes it*, and worth nothing when it does not. That asymmetry is why §27.2 puts API shape ahead of guardrail mapping in the reconnaissance order. The composition rule is the part that is genuinely non-obvious and is where a corpus beats a prompt: mutations are not additive. Persona plus encoding usually beats either alone; persona plus persona is a contradiction the model resolves by refusing; splitting plus length flooding is worse than splitting alone because the context dilutes the frames. A corpus with metadata is how you learn which pairs are synergistic on *your* target, and it is knowledge that does not transfer between targets - which is the point of running one. ### 29.3 Curation, and the contamination trap The obvious move is to harvest the public benchmarks: AdvBench's 520 behaviours, HarmBench's 400, JailbreakBench's 100. The obvious move is also the one that wastes the most money, for a reason that is the mirror image of §14.1's benchmark problem. Those strings are in the defence's training data. They have been public for years, they appear in every safety dataset, and a model has been fine-tuned against them in particular. Testing a deployment with AdvBench strings measures the deployment's *memory*, not its alignment. A corpus built from them will report either an implausibly low success rate (the defence memorised the exact strings) or an implausibly high one (the strings predate the defence), and neither number tells the operator what will happen tomorrow. The fix is the same one §18.5 prescribes for defenders and it is symmetric: **stratify by mechanism, not by string.** Generate instances from templates, hold out the mechanisms, and keep a small canary set of well-known strings as a control - if the canaries succeed and the generated set does not, you have learned something about the defence's training data, which is worth more than either number alone. ### 29.4 The regression suite is the business The last structural point, and the one that separates an operation from an afternoon: **a corpus is a CI pipeline, not a document.** ``` seeds ──> templates ──> instance batch ──> target ──> judge ──> score ──> promote/retire | └──> metadata store ``` The loop runs weekly. Promoted instances go into the served system prompt. Retired instances stay in the suite as regression tests, because "the defence failed against something old" is a signal about the defence and "the defence failed against something new" is a signal about the corpus - and confusing the two is how operators conclude a model was updated when in fact their own template broke. Cost of the loop, derived: 200 candidate instances, one attempt each, ~1,200 input tokens, judged by a cheap model at ~400 tokens - about 240k tokens of target traffic and 80k of judge traffic, or roughly **$0.10 and forty minutes** at §5.7's prices. That is the weekly premium for knowing whether you still have a business. Almost nobody pays it, which is why the visible population of resellers churns so violently: they find out that their prompt stopped working from a customer, not from a test. ### 29.5 What the corpus buys at scale - **Half-life extension.** A fixed prompt decays at the rate in §3.3, days to weeks. A corpus with ten mutation operators decays at the rate of the *mechanism*, months, because when one instance dies the generator produces the next one without human input. - **Attribution resistance.** Distinct instances per session defeat the traffic-shape clustering described in BLACKHAT.md §8.3 - not the billing identity, which is unaddressed by any prompt work, but the cheap layer of detection that most abuse teams run first. - **Transfer leverage.** When a new model appears, the metadata tells you which mechanism family to try first, which cuts the re-validation cost of a new backend from days to hours. In a market where the supplier changes quarterly, that is the entire operational advantage. The corpus is also the only part of the operation that is *legal to own*. A prompt pack is a text file; a fine-tune is a file; a working payload is a crime. That difference is why brokers sell corpora and kits rather than outcomes, and why the greyest businesses in this space look, from the inside, exactly like a red-team tooling vendor with worse invoices. --- ## 30. The harness At volume, the jailbreak is a line in a config file and the harness is the business. This is the stage the research literature treats as an implementation detail - PAIR and TAP are described as algorithms with a query budget, not as systems - and it is where every hour of an operator's week goes. ### 30.1 The five components | Component | Job | Typical failure | |---|---|---| | Queue | holds attempts, ordered by expected value | unbounded growth; a thousand dead attempts still queued | | Worker | one attempt: render, send, receive, timestamp | no backoff on 429; burns the account in an hour | | Judge | decides win/lose (§31) | systematic error in one direction, undetected for weeks | | Store | metadata per attempt: template, target, version, outcome | the thing that gets seized | | Router | picks the backend per request | no failover; one provider's patch = total outage | Two design notes that are not obvious until you have run one. **The retry policy is the attack.** Rate limits do not slow an operation, they *shape* it: a worker that retries on 429 with jitter finishes the same 10,000 attempts, just later and with a traffic pattern that looks organic. A worker that retries immediately gets the key banned and teaches the operator nothing. The single most consequential config value in a real pipeline is the delay between attempts, and it is set by the defender's quota policy, not by the attacker's preference. **Parallelism has a ceiling set by someone else.** Adding workers raises throughput until you hit the target's per-account limit, then raises the *number of bans* instead. §5.7's closing point - "the one lever that does work on cost is per-account query accounting, because the attacker's budget is real money and the defender controls the meter" - is a statement about harness architecture. The operator's throughput is the defender's quota dashboard, read upside down. ### 30.2 Quota arbitrage Four ways to get more attempts per dollar than the price list allows, in descending order of how well they work: | Method | Mechanism | Defeated by | Cost | |---|---|---|---| | Free-tier farming | many accounts, each under the free cap | identity verification, phone reuse detection | labour; ~$0.10/account on the resale market | | Key resale | buying organisational or leaked keys | billing anomaly detection, spend caps | $0.10 for a premium account, per BLACKHAT.md §7.1 | | Provider arbitrage | route to whichever backend is cheapest today | nothing; this is legitimate cost control | engineering | | Credit fraud | trial abuse, card testing, refund cycling | payment risk engines | legal exposure, high | The first two are why "identity" is the scarce commodity in §27.4's price structure. Note that the price of a stolen account, ten cents, is *below* the cost of the tokens that account can buy, which means the market is pricing the identity, not the compute. Whenever a credential trades below the value of what it unlocks, the constraint in that market is identity issuance, and any defender who owns identity issuance owns the attacker's cost base. ### 30.3 Throughput arithmetic Derived, and worth doing once because the numbers are smaller than people assume. Assume a frontier API with a 500 requests/minute per-account limit, a 2,000-token average attempt, and `p` = 0.05 for a decent fixed prompt against a guarded model. | Quantity | Value | |---|---| | Attempts per account per minute (quota-bound) | 500 | | Attempts per account per hour | 30,000 | | Successes per hour at `p` = 0.05 | 1,500 | | Tokens per hour | 60M | | Cost per hour at $0.15/M off-peak input | $9.00 | | **Cost per success** | **$0.006** | Read the bottom row against §5.7's $0.03-$0.12. The difference is not a better method; it is that this attacker has `p` = 0.05 on a *fixed* prompt and is paying a cheap model's price per token. The literature's cost-per-success figures are dominated by the price of the target, exactly as §5.7 says, and the cheapest high-volume operations are the ones that chose a cheap target (§27.3) rather than a clever method. The other reading of the same table: 60M tokens per hour per account is a *screaming* traffic signature. No human sends 500 requests a minute for an hour. Which is why §36 exists, and why the arithmetic that makes the operation viable is also the arithmetic that makes it visible. ### 30.4 The build order that works For anyone modelling an adversary's maturity, the order in which capabilities get built is diagnostic: 1. A single-shot script with a hardcoded prompt. (An afternoon.) 2. Retry and rate-limit handling. (A day, and the point at which the operator stops losing accounts.) 3. A judge, so attempts can be scored without reading them. (A day.) 4. Metadata and a corpus (§29). (A week, and the point at which the operation becomes maintainable.) 5. Multi-provider routing. (A week, and the point at which a patch stops being an outage.) 6. Delivery and payments (§34, §37). (Where it becomes a business, and where most of them fail.) An adversary at stage 2 is a nuisance. At stage 4 they are a durable problem, because they can re-find what the defence removes. At stage 6 they are a company, with the exposure that implies. Most of the population in the documented cases sits at stages 1-2 with an unusually powerful model, which is precisely why the incident record looks the way it does: high capability, amateur tradecraft, fast attribution. --- ## 31. Judging your own success The attacker's judge is the same instrument as the defender's (§14.2), pointed the other way, and it has the same defect: unmeasured error. The difference is that the attacker's error has a direct cash cost, because it determines which methods get funded. ### 31.1 What the judge is for Four decisions, all of which are worse with a bad judge: | Decision | Needs | Cost of error | |---|---|---| | Is `p` high enough to scale? | a calibrated estimate of `p` | overpaying for a method by 3× (§31.2) | | Which template to promote? | relative ranking across templates | the corpus converges on a fluke | | Is the target still vulnerable? | a stable baseline | outages discovered by customers | | Is the output good enough to sell? | a quality rubric, not a binary | refunds and churn | The last row is the one the research literature has no analogue for. A benchmark asks "did it comply". A product asks "would a customer pay for this", which is a different question with a different judge, and the gap between them is where most wrapper businesses die: they optimise compliance and ship gibberish. ### 31.2 The arithmetic of a bad judge This is the single most useful piece of arithmetic in Part VI, because it explains a persistent puzzle: why operators consistently believe their methods work better than they do. Take a stream of 1,000 attempts against a target with true `p` = 0.05, so 50 real successes and 950 real failures. Use a judge with 90% sensitivity and 90% specificity - optimistic for a model judge on this task, given §14.2. | | Judge says win | Judge says loss | |---|---|---| | **Actually won** (50) | 45 | 5 | | **Actually lost** (950) | 95 | 855 | The judge flags 140 attempts. 45 are real. **Precision is 32%**, and the operator's estimate of `p` is 14%, not 5%. A 2.8× overestimate, from a judge that is right nine times out of ten on both classes. Say it plainly: **at low base rates, a good judge produces a wildly wrong rate.** This is not a subtle statistical point, it is the reason the field's headline numbers do not replicate (§14.1), and it applies with full force to the operator who is trying to decide whether to buy a $200/month kit. The fix is the same one §18.3 gives defenders, and it is not optional: draw a random sample of flagged and unflagged attempts, label them by hand, and compute precision and recall. Two hundred labels calibrate a judge well enough to make budget decisions. The operators who do this discover that their "70% success" prompt is at 20%, and the ones who do not spend the next quarter building on sand. ### 31.3 The judge as a second jailbreak There is an irony that catches everyone: the judge is a model, the judge is often the *same family* as the target, and the judge may refuse to score the output it is asked to score. Three consequences: - **Systematic false negatives on the worst categories.** The judge refuses to engage, returns a hedge, and the pipeline scores it as a loss - so the operator's corpus systematically under-weights exactly the content that is most valuable to their customer. Inferred, but it follows from the asymmetry in §2. - **Judge drift under provider updates.** The judge is a hosted model with the same deprecation clock as the target (§28.4). An unversioned judge changes the measurement, and the change looks like a change in the target. - **The rubric beats the binary.** §18.3's four-row table applies verbatim: for a product, a rubric score is the only usable signal, because "complied" does not distinguish a sellable output from a degenerate one. ### 31.4 The downstream judge The honest ground truth is the downstream one: did the artefact work? Did the phishing page get clicks, did the code compile and run, did the exploit land, did the customer renew. This is expensive and slow, and it is the only judge that cannot be gamed by the model under test. The published campaigns show what happens without it. Unit 42's agent reported progress, enumerated tens of thousands of hosts, and produced **three confirmed successful exploitations** across 460+ targets. Whether its own internal report said "success" is unknown; whether the operator believed it is not, because the operator sent one Telegram message and went to sleep. A judge that scores attempts, rather than outcomes, would have rated that operation a triumph. For defenders this is a gift and should be understood as one: **the attacker's success measurement is systematically over-optimistic, so the threat is usually less capable than its own reporting.** Budget accordingly, but do not discount the capability itself - three confirmed exploitations from one message is still three more than a human operator would get for the same money. --- ## 32. Session tradecraft A conversation is a state machine (§6) and also a metered resource: every turn re-sends the history, so the token cost of a session grows quadratically with its length while the probability of success grows sub-linearly. The operator's job is to find the crossing point. ### 32.1 The cost curve Derived. Assume a 700-token opening request, 300-token replies, and a linear escalation that adds ~200 tokens of attacker text per turn. Cumulative input tokens after `n` turns is roughly `n(n+1)/2 × 1,000`. | Turns | Cumulative input tokens | Cost at $1/M | Cost at $0.15/M | |---|---|---|---| | 5 | 15k | $0.015 | $0.002 | | 10 | 55k | $0.055 | $0.008 | | 20 | 210k | $0.21 | $0.032 | | 50 | 1.28M | $1.28 | $0.19 | | 100 | 5.05M | $5.05 | $0.76 | Two readings. Below ten turns, conversation is free and the only cost is attention. Above twenty, the context dominates and the operator is paying for the defender's transcript. The practical consequence: **escalation is an economy of the first ten turns**, and any technique that needs fifty turns (Crescendo's long variants, patient foot-in-the-door) has to justify itself against the cost of simply restarting with a better opening - which, at `p` = 0.05 and a fixed prompt, is usually cheaper. ### 32.2 Restart or continue The decision every multi-turn operator faces at turn `k`, and the one they get wrong most often, because of sunk context: | Condition | Action | Why | |---|---|---| | Refusal landed, persona intact | continue, reframe, do not acknowledge | §6.2; the refusal is a turn, not a verdict | | Refusal landed, persona broken | restart | repairing a broken frame costs more turns than a fresh opening | | Model repeating a hedge | change register or language, not content | hedges are distributional, not semantic | | Success landed | **stop** | every extra turn risks the output filter catching the good turn (§16, layer 5) | | Context past ~60% of window | restart with compression | quality degrades before the limit does | The fourth row is the one with real money attached. An operator who keeps going after a win often loses the win to a classifier on the way out, and the pipeline records a failure - so the corpus is trained against the behaviour that actually worked. This is a measurement bug that presents as a capability bug, and it is endemic. ### 32.3 State that survives the session Three layers, in ascending order of value to the attacker and of danger to the defender: 1. **Within a session.** The transcript. Free, and destroyed by any restart. 2. **Across sessions, in the product.** Saved conversations, memory features, custom instructions. The operator's version of §6.3's persona persistence, and the reason a consumer app with a memory feature is a more valuable target than a stateless API. 3. **In the defender's own store.** Memory poisoning in the sense of §6.4: an instruction the agent writes to its own notes and later treats as its own conclusion. One successful injection becomes a durable capability, invisible to per-request inspection. Layer 3 is the only one that produces persistence without the attacker paying rent, which is why it is the highest-value target in the agentic surface and the least instrumented. A defender who logs every request and no *memory write* has instrumented the wrong side of the system. ### 32.4 The session budget is an identity budget Abuse teams do not count attempts, they count anomalies per identity per day. That makes sessions a consumable: - **Warm-up.** A few benign sessions on a new account before the first prohibited one. Cheap, effective against naive anomaly scores, and completely ineffective against a per-account cumulative-refusal metric - which is why §17.4 recommends exactly that metric. - **Burn rate.** Sessions per account per day before the account dies. Inferred from the ban economics in BLACKHAT.md §7.1: at $0.10 an account, an operator will happily burn ten a day, which sets a floor of about a dollar a day on identity costs that no model-side defence touches. - **Identity pooling.** Many accounts, one corpus. The corpus is the asset; the identities are consumables. This is the operator's correct mental model, and it is the reason detection built on "this account is behaving badly" has a half-life measured in days. ### 32.5 When the session can act The moment a session has tools, the risk profile changes class. Content harm is reputational and legal; action harm is legal, irreversible, and attributable to a human who configured a permission set. The operator's version of §13's argument, stated as a design rule: an agent session is worth more than a chat session by a factor of ten, and it is worth *documenting*, because the first thing an agent does in an unfamiliar environment is leave a record of itself. The Hermes case is the type specimen: the model did the work, then the operator's own environment published the credentials, target list and shell history through an unauthenticated port. The offence scaled; the tradecraft did not. --- ## 33. Weight-space operations Tier 4 is where the operator stops renting capability and starts owning it. It is also where the economics are least like the rest of this document, because the cost curve is capital expenditure rather than marginal spend, and because ownership removes the two constraints that bound every other tier: rate limits and provider policy. ### 33.1 Who does it, and why | Archetype | Why they take the capex | What they get | |---|---|---| | Product operator (§26.1) | per-token cost at volume; no AUP risk | an endpoint with no supplier | | Broker | resale: one artefact, many customers | a product with zero marginal cost | | Researcher-adjacent | curiosity, publication, clout | the same thing without the revenue | | Reseller | rarely; the capex is the barrier | usually nothing | The reported distribution supports the ordering. The 10a Labs study found 1,643 applications built on redistributed open-weight models, a quarter of them explicitly malicious, and Lin et al. found 11,598 uncensored models in a graph of 17,104 - a supply so large that the marginal model is free and the marginal *operator* is not. Almost nobody in that population fine-tunes from scratch. They download, verify nothing, and serve. ### 33.2 The pipeline ``` base checkpoint │ (a) refusal-direction extraction: contrast pairs, one forward pass per layer ▼ abliterate or LoRA-fine-tune ──> (b) capability check, (c) compliance check │ │ ▼ ▼ quantise (4/8-bit) ──> serve (vLLM/TGI) ──> monitor ──> re-eval on base updates ``` The literature's part is (a) - §11.2's direction, §11.1's fine-tune. The operator's part is (b) through the end, and it is where the money is spent and the mistakes are made. **(b) is the check nobody runs.** Ablation removes refusal and, per §11.2, costs some capability - the published cost is small on standard suites, and standard suites may not measure the capability you are selling. A model that will write anything and can no longer follow a five-part instruction is not a product. The operator's own evaluation should be a capability suite *and* a compliance suite, in that order, because compliance without capability is a very cheap way to own a useless endpoint. **(c) is the check that gets gamed.** Testing your own abliterated model with prompts from the corpus that produced it is a closed loop. The only meaningful test is held-out mechanisms (§29.3), and the only meaningful pass criterion is downstream (§31.4). ### 33.3 What self-hosting actually costs Derived, because the folklore ("self-host, it's free") is wrong in an interesting direction. Rent an H100 at ~$2.50/hour, which is a realistic 2026 spot price; that is $0.00069 per GPU-second. | Serving regime | Aggregate throughput | Cost per million tokens | |---|---|---| | Single stream, 70B, 4-bit | ~50 tok/s | ~$13.90 | | Batched, 70B, 4-bit | ~1,500 tok/s | ~$0.46 | | Batched, 8B, 4-bit | ~4,000 tok/s | ~$0.17 | Compare with the price sheet: DeepSeek V4.1 Flash at $0.15/$0.60 per million tokens off-peak (BLACKHAT.md §12). The honest conclusion, and it cuts against the whole romance of the black market: **self-hosting is more expensive than the cheapest hosted API, and cheaper than every frontier API.** It is not a cost play. It is a *policy* play - you are buying the absence of an acceptable use policy, and paying a premium of perhaps 3× over the cheapest legitimate tokens for it. Three corollaries: - The break-even against a frontier API is trivial - batched serving beats $3/M by an order of magnitude - which is why every serious product operator eventually owns weights, and why §37's margin tables look so good. - The break-even against a cheap API never arrives, which is why resellers stay on rented endpoints and die with them. - The capex is the moat and the anchor. Once you have bought the GPU and built the pipeline, you are committed to a model generation, and the generation turns over in months. ### 33.4 Serving is the skill The gap between "I abliterated a model" and "I run a service" is entirely operational, and it is where the research literature is silent: | Problem | Why it bites | Mitigation | |---|---|---| | Quantisation damage | 4-bit is 4× the throughput and measurably worse at long instruction-following | evaluate at the quantisation you will serve | | Context growth under load | KV cache is the real memory cost; concurrency collapses at long contexts | cap context, truncate aggressively | | Cold start | 70B at 4-bit is ~40GB to load; a restart is minutes of downtime | keep it warm; that is the bill | | Output filtering | you removed the refusal, not the customer's expectation of one | you now own the moderation you sold as absent | | Version drift | the base model gets a safety update; your ablation is now behind | re-derive, re-evaluate, re-deploy | That last row is the treadmill in its purest form, and the reason §35 is a whole section rather than a paragraph. ### 33.5 The distribution problem Owning weights removes the supplier and adds a new one: whoever hosts the GPU. Jurisdiction, provider terms and payment are now three separate exposures, and the reported enforcement pattern (BLACKHAT.md §8.6) is that they attach to the distributor, not the user. The 16.52% flag rate in Lin et al. - the fraction of uncensored models identified as malicious by the platform hosting them - is a measure of how much of this supply chain is visible to the people who could turn it off. A supply chain that is 16% visible is not clandestine; it is tolerated, which is a different and more fragile thing. --- ## 34. Delivery The correction that every operator learns in month one: capability is abundant and distribution is scarce. The jailbreak costs $0.32. The customer costs a click, and clicks on this category of product are the most expensive clicks on the internet. ### 34.1 The four channels | Channel | Reach | Acquisition cost | Platform risk | Price point | |---|---|---|---|---| | Chat bot (Telegram, Discord) | large, frictionless | low; channel membership is the funnel | high; platform bans the bot, not the operator | $5-$20/month, or per-message credits | | Web app | largest | high; adult and grey CPCs, SEO takes months | medium; hosting and processor are separable | $10-$25/month | | API resale | small, technical | near zero; word of mouth among developers | low until it is not; key discovery is fatal | per-token, at a 5-20× markup | | Character / companion marketplace | very large | zero, you are a listing | medium; the marketplace is the chokepoint | freemium, subscription, coins | The fourth row is where the volume is, and it is also the row that is easiest to switch off. A marketplace listing is a distribution channel owned by someone who has a trust-and-safety team and an app store review process. The reported structure of the companion market - $2.32B in 2025, ~$400M of it the NSFW segment (BLACKHAT.md §7.4) - is a market that exists because platforms *deliberately* allow a category at the edge of their policy and price the risk into the ranking. When the policy moves, the traffic moves. ### 34.2 The product problem nobody expects Jailbreaking is a supply problem. Delivery is a *retention* problem, and the retention curve of an uncensored product is different from a compliant one in a way that is well documented and badly understood. The mechanism, stated as the market's own reported finding: compliant platforms strip high-risk content to keep app-store and processor access, and **lose retention**; the uncensored pole abandons mainstream distribution and prices high to absorb payment and compute risk (BLACKHAT.md §7.4). Candy.ai's claimed $25M+ ARR against outside estimates of $50-$120M, and Character.AI's ~185M monthly visits followed by a pivot to licensing, are the two ends of the same curve. For the operator, this yields three design rules that have nothing to do with jailbreaking: - **The first output is the product.** Activation is "did the model say the thing", and it happens in the first thirty seconds or not at all. A pipeline with a five-turn escalation (§32) has terrible activation, however good its `p`. - **Filtering you added is churn you bought.** An operator who inserts a moderation layer to keep a processor happy has reintroduced the refusal they were selling the absence of, and the reviews will say so. - **Persona beats capability.** Retention tracks character consistency across sessions (§32.3, layer 2), not benchmark scores. This is why the market wants memory features, long contexts and stable system prompts - all of which are cheap on a self-hosted model and awkward on a rented one. ### 34.3 Support is the hidden cost Every LLM product is a support business wearing a software business's clothes, and the uncensored segment is worse because a fraction of customers are trying to break it, a fraction are trying to resell it, and a fraction are paying with someone else's card. | Support load | Driver | Effect on margin | |---|---|---| | "It refused me" | pipeline variance, not a bug | the operator's own `p` leaking to the customer | | "It's different than last week" | provider patch (§35) | refunds, churn | | Account sharing | credentials resold | quotas consumed, one subscriber becomes five | | Chargeback threats | buyer-side fraud | processor risk score (§34.5) | The first row is the interesting one, because it is *structural*. A product built on a probabilistic pipeline has a per-request failure rate that the customer experiences as a defect. The operator can drive `p` toward 1 only by spending per-request (§37), which is exactly the margin they do not have. This is the economic reason large-scale uncensored services converge on a fixed-prompt, self-hosted architecture: it turns a variable cost into a constant, and a variable failure rate into a constant one. ### 34.4 Discovery The demand-side data is unambiguous. Halcyon's study measured **fewer than 50 AI-tool advertisements per month underground in late 2025, rising to over 1,400 per month by February 2026**, across ~4,000 posts in 77 Telegram channels, 20 dark-web forums and 5 specialist markets. Twelve-fold growth in four months is not a technology signal; it is an advertising signal. The sellers arrived when they believed the buyers were there. The channels, in order of volume: 1. **Search.** Queries for "no filter", "unfiltered", "uncensored" plus a model name. SEO against a provider's brand term is free reach and a trademark problem simultaneously. 2. **Community seeding.** Forum posts and reviews in the communities that already discuss jailbreaking, which is also where the defence vendors read. 3. **Affiliate and referral.** The standard growth loop for grey consumer products: revenue share, which requires reliable payments, which is the constraint again. 4. **Marketplace listings.** The highest-volume channel and the one with a landlord. ### 34.5 The two-sided scam problem This is the structural feature of the market that defenders under-use. Every transaction is between two parties who cannot use the law, and neither can verify the other. | Failure | Who does it | Frequency | Effect on the market | |---|---|---|---| | Exit scam | seller | high | no prepayment trust; prices fall, volume rises | | Chargeback fraud | buyer | high | processor risk scores rise for everyone | | Resale of access | buyer | very high | quotas consumed; unit economics break | | Kit with malware | seller | documented (§28.2) | buyers lose credentials and disappear | | Fake reviews | both | saturated | discovery becomes unreliable | Two consequences, and the second is the important one. First, the market cannot finance trust, so it consolidates on a small number of channels with visible history - the ones with a public channel, a track record and something to lose. Second, **those channels are the chokepoints.** A market that cannot use escrow, courts or reputation portability concentrates on platforms that provide them, and platforms are amenable to pressure in a way that anonymous sellers are not. This is why payment processors and app stores have removed more capability from this ecosystem than every classifier combined - the same conclusion BLACKHAT.md §7.4 reaches from the product side, arriving from the transactions side. --- ## 35. Reliability and the maintenance treadmill Everything in this document decays. The only open question for an operator is whether they find out from a monitor or from a customer. §3.3 gave the defender's version of this - half-life - and Part VI's claim is that the operator's version is not half-life but **MTBF of the inventory**, and that it is the quantity that decides who stays in business. ### 35.1 The failure taxonomy | Failure | Signature | Detection lag | Recovery | |---|---|---|---| | Provider patch | whole corpus drops within hours, uniformly | minutes if monitored, days if not | re-mutate (§29.2); usually partial | | Silent moderation | a fraction of outputs replaced with a hedge; `p` drops without refusals | days; looks like variance | none; migrate the backend | | Model deprecation / alias re-point | capability shift; different refusal style | hours | re-fingerprint (§27.1) | | Judge drift | measured `p` moves while outputs do not | weeks; nobody audits the judge | recalibrate (§31.2) | | Rate-limit tightening | throughput collapses, not `p` | immediate | more identities, more cost (§30.2) | | Account ban wave | many dead workers, same day | immediate | identity budget, again | | Prices change | margin, not capability | a billing cycle | re-route | | Base model retrain | your ablation is behind; capability gap opens | months | re-derive (§33.2) | The third and fourth rows are the ones that produce the most wasted effort, because both present as "the jailbreak stopped working" when in fact the *measurement* stopped working. An operator who recalibrates the judge before rebuilding the corpus saves a week; almost nobody does, because rebuilding the corpus is the satisfying work. ### 35.2 MTBF, and why it caps the business Inferred, from the mechanism half-lives in §3.3 and the patch cadence implied by the incident record: | Asset | MTBF | What ends it | |---|---|---| | Fixed prompt, frontier API | 2-6 weeks | classifier update or safety fine-tune | | Fixed prompt, mid-tier API | 2-4 months | rarely patched; usually the provider disappears | | Search loop (PAIR/TAP-style) | indefinite, degraded | cost rises as `p` falls; never dies outright | | Abliterated endpoint | until the next base release | base model moves; product ages | | Injected agent (Tier 5) | until the harness is fixed | one patch, and the harness rarely patches | | Payment rail | weeks to months | risk team, not model team | Read the fourth column as a business constraint. A product whose core asset has an MTBF of six weeks cannot be sold as a subscription without a rebuild capability, and the rebuild capability is the corpus and the pipeline (§29, §30). This is the real reason the market has a long tail of tiny operators and almost no large ones: **scale requires maintenance staff, and maintenance staff are employees, and employees are attributable.** ### 35.3 SLOs for a service nobody will write a contract for The operators who last converge on the same four numbers, because their customers churn on them: | Metric | Target | Why | |---|---|---| | Availability | 99% during the customer's waking hours | downtime is discovered and reviewed publicly | | p95 latency | under 4 s to first token | slower than a competitor's, and the churn is instant | | Quality floor | "the thing" happens on the first attempt, 80%+ of sessions | the §34.2 activation rule | | Breakage-to-fix | under 72 h | the window before a negative review compounds | The quality floor is the expensive one and it is the one that forces the architecture: an 80% first-attempt rate is `p` = 0.8 at the *session* level, which a fixed prompt on a guarded model does not deliver and a self-hosted abliterated model does. That single requirement explains most of the observed market structure - why product operators own weights and resellers do not. ### 35.4 The observability paradox To detect breakage you need logs: every attempt, its template, its target version, its outcome. To survive attribution you need to not have logs. These are the same data. Three resolutions, none good: - **Keep logs, accept the exposure.** What almost everyone does. It is why criminal cases increasingly rest on the defendant's own harness output (BLACKHAT.md §8.4) and why the Hermes operator's published shell history was the investigative gift of the year. - **Aggregate, do not retain.** Per-template success counts, no transcripts. Loses the ability to debug, keeps the ability to detect. - **Retain nothing, monitor live.** Alerting without storage. Works until the first ambiguous failure, at which point the operator is guessing. The paradox has a clean statement that belongs in a defender's notes: **an operator's need for observability is proportional to their need for reliability, and their reliability need is proportional to their scale.** Small operators can be invisible and fragile; large operators must be observable and are therefore findable. Scale buys revenue and sells deniability, which is why the enforcement record is full of the medium-sized and empty of the tiny. --- ## 36. Detection and evasion, honestly This is the section where the folklore-to-evidence ratio is worst, so it is organised as a claim-by-claim audit. What does the provider actually see? What does the operator do about it? Does that work? ### 36.1 What the provider sees | Signal | What it is | Attacker counter | Does it work | |---|---|---|---| | Request volume and burstiness | attempts per minute per identity | pace the workers; jitter | **yes, until the volume is the point** (§30.3) | | Refusal rate per identity | the fraction of requests that get refused | warm-up sessions; rotate identities | no, against cumulative metrics | | Topic clustering | a distribution concentrated on prohibited categories | diversify requests with benign traffic | no; dilution costs quota and does not change the mode | | Session topology | thousands of sessions with one identical system prompt | vary the prompt per session | yes, cheaply, and it is why prompt fingerprinting fails | | Input perplexity | GCG-style suffixes are gibberish | use fluent prompts (AutoDAN's whole point, §9.4) | yes, and it is free | | Output classifier hits | the model complied and the filter caught it | re-roll; change format | partially; costs attempts | | Network origin | ASN, IP reputation, proxy quality | residential proxies | yes, at a real cost | | Payment instrument | card, its history, its reuse across accounts | virtual cards, stolen instruments | **no, and this is the binding one** | | Device and browser fingerprint | one machine, many accounts | anti-detect browsers | partially, and expensively | | Key-sharing geography | one key, three continents | none needed; they buy three keys | n/a | The bottom half of the table is the argument. Every row from "network origin" down is an *identity* signal, and no amount of prompt work touches any of them. The top half is the layer where evasion is cheap and temporary; the bottom half is the layer where evasion is expensive and permanent. ### 36.2 The perplexity myth, and its mirror The most repeated claim in the grey literature is that "gibberish triggers detection, so fluent jailbreaks are undetectable". It is half true, and the half that is false matters. - **True:** GCG suffixes are out-of-distribution strings and cheap to score (§9.1). A detector on input perplexity catches the white-box family at nearly no cost. This is one of the few defences with a clean mechanism. - **False:** fluency is not safety. AutoDAN exists precisely because fluent adversarial prompts defeat perplexity filters (§9.4), and once an attacker writes readable prose, the perplexity signal is gone - it was never measuring intent, it was measuring the attack's *style*. The mirror claim, from the defender's side, is equally overstated in the other direction: "the model knows it is being jailbroken". §7.1 is the honest version. A reasoning model's trace often contains evidence of the conflict - that is a debugger for the defender and a target for the attacker (§7.2). It is not a reliable detector, because the trace is generated text, and text can be shaped. ### 36.3 The base rate argument The asymmetry that decides this whole section: - The defender needs **one** true positive per identity, across a stream where the attacker must succeed thousands of times. - The attacker needs **zero** detections per identity, across thousands of attempts. At a per-request detection probability of 1%, an identity survives 100 attempts with probability `0.99^100` = 37%, and 1,000 attempts with probability `0.99^1000` ≈ 0.004%. At 0.1% per request, 1,000 attempts survive with probability 37%. The derived conclusion is uncomfortable and it is the one operators price in: **volume is incompatible with evasion at any detection rate above ~0.01% per request.** Which is why the only evasion strategy that works is the one that abandons volume: low-volume, high-value, one-shot use. That is the broker's model (§26.1), and it is also, not coincidentally, the model of the attacker who is hardest to catch - because they generate the fewest events, have the fewest customers, and need the least infrastructure. ### 36.4 The evasions that are theatre Four that circulate widely and do not survive contact with a competent abuse team: | Claim | Why it fails | |---|---| | "The provider can't tell which model you are" | they can; it is their model, and they have the keys | | "Encrypted traffic hides the prompts" | the prompts terminate at their API, not at a wire | | "Rotating IPs is enough" | billing identity is the durable signal, and it does not rotate | | "A local model has no logs" | it has yours; and your disk is the log (§38) | The common structure: each claim is true about a layer the defender is not using. This is the same error as the folklore in §4 - defending against a mechanism rather than a control - committed from the attack side. ### 36.5 What actually raises the attacker's cost Stated as a ranking, because this is the section a defender should take to a budget meeting: 1. **Identity issuance.** Payment instrument verification, phone binding, spend caps and per-identity cumulative accounting. The reported price of a stolen premium account, $0.10, is the number to move. 2. **Cumulative behaviour over sessions.** Not per-request classification: a refusal-rate-per-identity metric, which no prompt trick reduces without reducing volume. 3. **Traffic-shape anomaly.** Not the content; the shape. 60M tokens an hour from one account is arithmetic, not inference. 4. **Latency and cost asymmetry.** Detecting the search loop by its retry pattern rather than its payloads. 5. **Provenance on the supply side.** Flagging redistributed and modified checkpoints, which is the one control aimed at Tier 4 and the only one that touches an attacker who never calls an API. Everything at the top of that list is cheap, boring and system-level, which is §17.4's conclusion arriving from the attacker's side for the third time in this document. The repetition is the finding. --- ## 37. The operator's cost model Part I priced attacks. This section prices *operations*, which is a different ledger: fixed costs, variable costs, labour, and the cost of being caught. The numbers are derived from the price sheet and unit economics in BLACKHAT.md §7, and the reader should treat them as order-of-magnitude. ### 37.1 The amortisation law The most consequential single fact in Part VI, and it is arithmetic rather than research. Take a wrapper at $20/month with a 3,000-message fair-use cap. - Revenue per message: `$20 / 3,000` = **$0.0067**. - Compute cost per message, fixed prompt, off-peak: `700 × $0.15/M + 350 × $0.60/M` = **$0.00032**. - Compute cost per message *if each message requires a search loop* at §5.7's $0.03 per success, repriced at $0.15/M input: **~$0.0045**. | Architecture | Cost per message | Share of revenue | Gross margin | |---|---|---|---| | Fixed prompt, prompt-cached | $0.0003 | 4.7% | 95% | | Fixed prompt, no caching | $0.00032 | 4.8% | 95% | | Per-request search loop | $0.0045 | 67% | 33% | | Per-request search loop, one retry | $0.009 | 135% | **negative** | **The amortisation law: a jailbreak must be a constant, not a query.** Any architecture that pays for the attack on every request is unviable at consumer prices, and the entire observed market structure - fixed system prompts, abliterated checkpoints, "persona" products - follows from this one inequality rather than from anyone's aesthetic preference for simple prompts. It also explains a puzzle in §15's table: why the highest-ASR automated methods are commercially irrelevant even though they are the best research. They are per-query by construction. The commercially relevant methods are the ones whose output can be *baked in* - a prompt, a checkpoint, a persona - which is exactly the set that §3.3 says has the shortest half-life, and therefore exactly the set that generates the maintenance burden in §35. ### 37.2 The reseller's P&L | Line | Value | Note | |---|---|---| | Subscribers | 200 | typical for a channel-distributed bot | | Price | $15/month | market rate | | Revenue | $3,000 | | | Token COGS | $192 | 200 × 3,000 messages × $0.00032 | | Identity and bans | $300 | 10 accounts/day at $0.10 (§32.4) | | Infrastructure | $100 | VPS, proxies, bot hosting | | Processor fees | $150 | ~5% | | Chargeback losses | $36 | 1.2% of revenue | | Acquisition | $1,000 | the dominant line | | **Net before labour** | **$1,222** | | | Labour | ~80 h/month | support, monitoring, rebuilding | | **Effective wage** | **~$15/hour** | with felony exposure | That last row is the punchline of the reseller archetype, and it is consistent with Halcyon's own reported conclusion that there is no mature market for a fully autonomous attack agent: what the ecosystem sells is task-level acceleration, not a business in a box. The people running $20 wrappers are running a minimum-wage job with a criminal record attached, which is why so many of them are teenagers and so few of them are professionals. ### 37.3 The product operator's P&L | Line | Value | Note | |---|---|---| | Subscribers | 3,000 | requires real distribution | | Price | $20/month | | | Revenue | $60,000 | | | GPU (one H100, batched) | $1,800 | $2.50/h × 720 h | | Payments and processors | $3,600 | ~6% with adult-friendly processors | | Chargebacks | $900 | 1.5%, near the BRAM excessive threshold | | Support | $8,000 | two staff, partially automated | | Acquisition | $20,000 | adult-traffic CPCs | | **Net** | **~$25,700** | ~43% margin | | Labour | two people | | Two observations. First, the margin is real but *thin by software standards*, and it is thin because of lines that have nothing to do with AI: acquisition, support and chargebacks. Second, the GPU line is now the smallest technical cost, which is why §33.3's conclusion matters: self-hosting is a policy purchase, and at this scale it pays for itself against any frontier API within a month. ### 37.4 The broker's P&L | Line | Value | Note | |---|---|---| | Customers | 20 | low volume by design | | Price | $150/month | kit, updates, support | | Revenue | $3,000 | | | Inference COGS | ~$0 | the customer runs it | | Maintenance | 10 h/week | corpus, docs, patch responses | | **Net** | **~$2,700** | ~90% margin | | Risk | concentrated | one customer's arrest is your indictment | The broker is the highest-margin, lowest-volume, highest-variance business in the space, and the variance is legal rather than commercial. It is also the archetype whose product is closest to legitimate security tooling, which is why the red-team market and the kit market share suppliers, corpora and sometimes invoices - and why §37.5's conclusion is about which side of that line a business chooses to stand on. ### 37.5 Where the cost axis inverts Part I's axis: attacks ranked by cost, because cost determines who can mount them. Part VI's axis: operations ranked by *maintenance cost per revenue dollar*, because that determines who survives. The two rankings are almost uncorrelated: | Rank | By attack cost (§3.2) | By operating cost per revenue | |---|---|---| | Cheapest | Tier 0 persona prompts | self-hosted fixed prompt (constant, amortised) | | | Tier 1 BoN | hosted fixed prompt | | | Tier 2 multi-turn | search loop (unviable at consumer prices) | | | Tier 3 gradients | per-query adaptive attack (loss-making) | | Most expensive | Tier 4 weights | multi-provider product with 40 h/week maintenance | The inversion is the thesis of Part VI in one table. **The literature's cheapest attack is not the operator's cheapest operation, and the literature's most expensive attack is not the operator's most expensive one either.** Jailbreaking research optimises the wrong end of the ledger, which is not a criticism of the research - it is doing its job - but it is why reading it as a business plan produces the reseller P&L in §37.2 and a wage of $15/hour. ### 37.6 The demand side, for scale UNODC's 2026 assessment put Southeast Asian scam losses at **$88.3B-$114.1B for 2025** and described the industry as franchised: a central service hub with front-end operations, standardised tooling, AI and crypto in the stack (BLACKHAT.md §7.6). Against that number, a $60,000/month AI product is a rounding error, and the jailbreak is a component supplier to an industry that was already industrial before it arrived. Which is the correct frame for the whole of Part VI. The blackhat side of jailbreaking is not a hacker story. It is a supplier story: a cheap input component - text that a model should not produce - feeding a large, existing, franchised demand. The interesting numbers are prices, margins and maintenance hours, not success rates. --- ## 38. Attribution, or how they get caught This is the section that the romantic version of this document omits, and it is the only one that changes a reader's behaviour. Every documented operation in the corpus ends the same way, and it never ends on a jailbreak. ### 38.1 The enforcement arithmetic Three structural facts, before any case: 1. **The offence completes early.** Attempt, solicitation, possession and conspiracy statutes do not require a working exploit (BLACKHAT.md §8.6). Everything an operator does *before* success is already evidence. 2. **The crime scene is a billing relationship.** Models are rented. Renting requires a card, a phone, an identity, and an invoice that survives a subpoena. This is the difference between this ecosystem and traditional malware: the infrastructure is not owned, it is subscribed to. 3. **The population is small and the tail is long.** The 71 catalogued US federal AI-CSAM prosecutions (2022-2025) and the cases in Cyprus and Ohio are not a dragnet. They are a handful of examples chosen to be legible, which is how enforcement works against any young crime category. ### 38.2 The failure modes, enumerated | Failure | What leaked | How common | |---|---|---| | Payment identity | the human | near-universal; the first subpoena | | Infrastructure reuse | linkage across campaigns | very common; VPS and domain reuse | | Retained logs | the entire operation, timestamped | near-universal (§35.4) | | Self-exposure | an open port, a public bucket, a debug endpoint | common; the Hermes case is the type specimen | | Public persona | a handle, a channel, a review history | common; §34.5 forces visibility | | Customer arrest | the supplier, via the customer's device | common; distribution is a witness list | | Tooling fingerprint | campaign clustering | common; agents are *more* consistent than humans | | Machine speed | the operator is obviously software | common; 500 req/min is not a person | | Supplier betrayal | credentials, then the buyer | documented (§28.2) | | Showing off | a writeup, a screenshot, a talk | occasional, and disproportionately fatal | The three near-universal rows are all *financial*. This is the single most important structural fact about the adversary in this document: **the thing that identifies them is not their prompts, it is their invoices.** ### 38.3 The cases, read as attribution stories Read the documented cases for the failure, not the capability: - **Unit 42's Hermes operator.** The agent did genuinely impressive work at scale: enumeration across tens of thousands of hosts, exploit selection, chain assembly. It was identified by infrastructure and by a GitHub handle whose public profile matched an assessed location - and it published its own API keys, target list and shell history through an unauthenticated `http.server` on port 8888. Capability: high. Tradecraft: none. - **`s1ngularity`.** Malicious npm packages, four hours of exposure, 2,349 credentials. The attribution path was the publish pipeline itself: a stolen token, a workflow, a provenance chain. The supply chain that made the attack scalable is the same chain that made it traceable. - **`postmark-mcp`.** Discovered because someone read the diff. Malicious tool descriptions are *source code*, and source code is diffable. - **The CSAM prosecutions.** Caught because victims reported, not because a platform detected (BLACKHAT.md §8.6). Detection did not work; humans did. The pattern: in every case, the technical capability was not what failed. The operational layer was, and the operational layer is the layer this document cannot teach and the research literature does not measure. ### 38.4 Why agents make it easier The intuition is that automation improves OPSEC. The evidence says the opposite, for four reasons BLACKHAT.md §8.5 states and this part should underline: - **Consistency is a fingerprint.** Agents call tools in the same order, query the same APIs, format the same output, and leave the same artefacts. Humans vary; software does not. Cluster analysis loves this. - **Speed is a fingerprint.** Thousands of requests per second from one ASN is arithmetic, not inference (§36.3). - **Logs are generated by default.** Harnesses write session JSON, transcripts, cost ledgers and tool arguments. A developer's feature is an investigator's exhibit. - **Agents act while nobody is watching.** Unattended execution means unattended mistakes: an exposed port, a debug flag, a `-v` left on. The generalisation is worth stating as a law for the AI era: **automation compresses the time between an idea and an act, and leaves the evidence generation to the machine, which is very good at generating evidence.** ### 38.5 What actually protects an operator Nothing in this document, and that is the point. The measures that work are mundane and largely financial: separate legal entities for infrastructure, payment instruments that are not reused, no logs beyond the current day, no public persona, no customers. Every one of those is a measure that eliminates the business rather than protecting it - which is why the population that survives is the population that stayed small. The honest summary for a defender: **you do not need to detect this activity to neutralise it, you need to make it have a counterparty.** A counterparty is a processor, a host, a marketplace, a model provider, or a customer. All five are identifiable, all five keep records, and the removal of any one of them is worth more than a percentage point of classifier recall. ### 38.6 The investigator's three questions For completeness, because a defender designing telemetry should design it to answer these: 1. **Who paid?** Payment instrument, account lineage, spend pattern. 2. **Whose infrastructure?** Hosting, domains, proxies, ASN history. 3. **Who was the human in the loop?** The one who chose the target (§27.4), which is the act that converts possession into an offence. A logging architecture that answers those three questions for every incident is worth more than one that classifies prompts. That is the design brief, and almost no deployment is built to it. --- ## 39. Why the blackhat reading of this document is wrong A reader can close Parts I to V and conclude that the attack side is easy, that the defences are decorative, and that the rational move is to go and do it. That conclusion is wrong, not on moral grounds but on arithmetic, and this section is the audit. ### 39.1 Seven errors **1. ASR is not yield.** A reported rate is measured on a benchmark, in one turn, against a bare model, with a judge whose error is unmeasured (§14). A yield is successes per hour at a quality a customer accepts. The mapping between them is not linear and is not favourable: §31.2's arithmetic shows that at low base rates even an accurate judge inflates the operator's estimate by ~3×. **2. A benchmark is not a deployment.** §15.2 says this in one paragraph and it bears repeating in the operator's units: the deployed target has a classifier, a rate limit and a log, and it changes monthly. The published 89% is against a model, and no incident in the corpus was caused by a model. **3. Capability is not a product.** The amortisation law (§37.1) is the hardest constraint in Part VI: 67% of revenue goes to the search loop, 135% with one retry. Access to capability is not access to a business; the business is a constant-cost pipeline, and building one is systems work, not jailbreaking. **4. Novelty is not value.** The market prices a working prompt at $0.32 and a stolen account at $0.10. If novelty were the scarce input, the ratio would be inverted. The scarce inputs are identity and distribution, which is why the buyers in the documented marketplaces are buying volume and uptime, not technique (BLACKHAT.md, Halcyon study). **5. Anonymity is not impunity.** §38.2's table is the whole argument. The identification vector is financial in almost every case, and financial identities are the one thing an operator cannot rotate cheaply without destroying their revenue. **6. Scale is not profitability.** §37.2: a 200-subscriber reseller nets about $1,200/month before labour and earns roughly $15/hour for it, with criminal exposure. The broker margin in §37.4 is 90% and the volume is twenty customers, which is the opposite of scale and the opposite of the folklore. **7. Survivorship runs the other way here.** The published cases overstate capability, not understate it: they are the operations that got written up because they were large, and even they produced three confirmed exploitations from 460 targets. The successful quiet operators are invisible, and *therefore unmeasurable*, which means every claim about "what works in the wild" - this document's included - is drawn from the visible tail. ### 39.2 The missing payload layer §0 already says that this document does not reproduce payloads. Part VI inherits that rule for a second reason that is worth stating plainly: at the operator layer, the payload is the *least valuable* part of the operation and the most criminal. Everything above - recon, corpus, harness, judge, delivery, payments, maintenance - is where the difficulty lives, and the payload is a commodity priced at thirty-two cents on a market with a review problem. A reader who wants the bytes can find them. A reader who wants to understand why the bytes are not the interesting part has just read eleven sections about logistics. ### 39.3 The incentives that inflate this literature Two biases produce the "it's easy" reading, and a careful reader corrects for both. **Publication bias toward novelty.** A paper reporting that a defence works is harder to publish than one reporting a break. §16's table is the result: the attack literature is larger, better funded in attention, and more quotable than the defence literature, and the ratio of claims is not the ratio of effects. **Vendor interest in the threat being large.** Attack success rates sell classifiers. Defense success rates sell classifiers too, but the attack numbers are the ones that move budget, so they are the ones optimised for headline legibility - a single number, no conditions, a named frontier model. §14.1 explains why that number is not comparable to anything. Neither bias makes the threat false. Both make the threat *bigger in presentation than in operation*, and the operator who acts on the presentation is the operator in §37.2's wage column. ### 39.4 The path that pays better The comparison that ends the argument, in the same units: | Path | Revenue | Cost | Exposure | |---|---|---|---| | Reseller (§37.2) | $3,000/month | ~$1,800 + 80 h | criminal | | Broker (§37.4) | $3,000/month | ~$300 + 40 h | ambiguous, converging on criminal | | Red-team engagement (BLACKHAT.md §7.5) | $15,000-$50,000/month equivalent | labour | none | | Jailbreak bounty, novel mechanism | five figures, one-off | research skill | none | The same skill set - corpus construction, harness engineering, judge calibration, knowing which mechanism beats which layer - is what a red team sells, at a higher hourly rate, with a contract instead of an indictment. That is not a moral argument bolted on at the end. It is the market's own price signal, and it is the reason BLACKHAT.md §7.5 concludes that services beat products. ### 39.5 What Part VI is actually for Three uses, none of them offensive: 1. **Threat modelling with real cost curves.** §26.4's four quantities and §37.1's amortisation law let a defender rank their risks by what an attack costs to *operate*, which is the only ranking that predicts persistence. 2. **Adversary emulation.** A red team that knows the maintenance treadmill (§35) will not report "we jailbroke it once" as a finding; it will report what survives a fortnight. 3. **Budget arguments.** The three controls in §36.5 and the P&Ls in §37 are the same document read twice: cheap identity controls beat expensive classifiers, and the numbers to prove it are above. The rest is colour, and colour is what most writing about this subject is made of. --- ## 40. What changes for the defender Part V closed with §17.4's "one control to add this quarter" and §18's protocol. Part VI changes the *reasoning* behind both, without changing the recommendation, which is the strongest evidence that the recommendation is right. ### 40.1 Stage-to-control map | Operator stage | Their cost driver | Cheapest control that attacks it | Their cost after | |---|---|---|---| | 1 Demand | acquisition spend | payment and store policy, not technology | channel disappears | | 2 Access | identity, $0.10/account | payment verification, phone binding, spend caps | identity cost rises 10-100× | | 3 Capability | $0.32 for a prompt | nothing durable; classifiers buy days (§3.3) | days | | 4 Production | quota | per-identity cumulative accounting, traffic shape | throughput ceiling | | 5 Delivery | marketplace access | platform terms enforcement | distribution loss | | 6 Payment | processor tolerance | nothing a defender does; this is their weak point | - | | 7 Maintenance | hours | keep the treadmill turning: ship updates, vary detection | breakage-to-fix time | Read the fourth column downward. The controls that move the attacker's cost by an order of magnitude are all in rows 2, 4, 5 and 6 - identity, quota, distribution, money - and none of them are model-level. The single row where model-level work buys something is row 3, and §3.3 says what it buys: days. ### 40.2 The three controls that beat the economics Restated, because §36.5 is the operational version and this is the budget version: 1. **Identity issuance and binding.** The cost of a viable identity is the attacker's floor. Raise it and every per-attempt cost in §5.7 multiplies. 2. **Cumulative accounting per identity.** The only control that is unaffected by prompt mutations, because it does not look at the prompt. It converts the attacker's volume - the thing their business requires (§37.1) - into their own detection signal. 3. **Action mediation for anything with tools.** §13.5's table, and the only row where the fix is cheap and known: allowlist, parameter validation, approval gates, egress enumeration. ### 40.3 The three that do not 1. **Prompt blocklists.** Half-life of days (§3.3), defeated by ten mutation operators (§29.2), and they generate overrefusal (§14.5) which is a real cost with no security return. 2. **Model hardening as the plan.** Necessary, insufficient, and it addresses the stage that costs the attacker thirty-two cents. 3. **"Detecting jailbreak phrasing."** A classifier for a style. §36.2's mirror: fluency is free for the attacker, so the style moves and the classifier does not. ### 40.4 The asymmetry ledger The reason the three controls above work is not that they are clever; it is that they exploit a structural asymmetry: | | Attacker | Defender | |---|---|---| | Cost structure | variable, paid before revenue | fixed, paid once | | Failure mode | one detection kills an identity | one miss is one miss | | Requirement | reliability **and** invisibility | detection **or** deterrence | | Time horizon | must survive the next patch | must survive the audit | | Exposure | financial identity | legal process | The attacker must be simultaneously reliable and invisible at scale, and §35.4 and §36.3 show those requirements are in direct conflict. That conflict is the defence. It is not a model property and it cannot be trained into one. ### 40.5 The two halves of the evidence Parts I to V ask what breaks a model. Part VI asks what it costs to run a business on the answer. The second question is the one that determines persistence, because a technique that cannot be operated at a constant cost disappears from the incident record regardless of its success rate - which is why the observed incidents are Tier 0 prompt abuse and Tier 5 injection, the two tiers with the lowest maintenance load, rather than the tiers with the highest success rates. The defender's summary of Part VI, in five lines: - Price the operation, not the attack; the recurring cost is maintenance (§35). - Identity is the scarce input and the binding control (§27.4, §36.5, §40.2). - Volume and evasion are incompatible; choose which one to defend against (§36.3). - The attacker's measurement is optimistic by roughly 3× at low base rates (§31.2), so budget against capability, not against claims. - The reliable operator is visible and the invisible operator is small (§35.4), and neither is a threat to a layered stack - which is why §17.3's modest guarantee, "the cheapest attack expensive and the expensive attack detectable", remains the correct standard after both halves of this document. --- # Part VI apparatus The sections that follow are the operator-side counterparts of §§20-25. They are separate rather than merged because the two halves of this document are asking different questions and a merged number card would imply comparability that does not exist. --- ## 41. Timeline, operator side The research timeline in §20 dates ideas. This one dates *infrastructure and markets*, because that is what changed the incident rate. Publisher named, no links, same convention as BLACKHAT.md. | Date | Event | Why it matters | |---|---|---| | 2023 | FraudGPT at $200/month, $1,000/6 months, $1,700/year; WormGPT shut down Aug 2023 | The reference price for "AI for crime" is set, and the first brand is already dead | | 2023 | Prompt packs circulate as products | The $0.32 commodity appears before the model that makes it reliable | | Jun 2024 | Arditi et al. publish the refusal direction | Abliteration becomes a weekend task; uncensored forks follow within weeks | | 2024-25 | LoRA fine-tunes and "uncensored" checkpoints become a category, not an artefact | Supply stops being scarce; the 11,598-model graph in Lin et al. is the result | | Jan 2025 | DeepSeek R1 ships; Cisco and UPenn report 100% success on 50 HarmBench prompts | The cheapest capable model is the least defended one | | Early 2025 | KELA finds infostealer scripts in R1 jailbreak packages | The supply chain turns on its own customers; §28.2's pattern begins | | Jan-Feb 2025 | DeepSeek blocked in Italy, banned on Australian government devices, 17 US states, PIPC inquiry in Korea | Regulation attaches to distribution, not capability | | Jun 2025 | EchoLeak, CVE-2025-32711 | Zero-click exfiltration in a production assistant; Tier 5 goes mainstream | | Aug 2025 | `s1ngularity`: nx packages, 2,349 credentials, 2,180 accounts, 7,200 repos | Malware delegates to the developer's own coding agent | | Sep 2025 | `postmark-mcp`: ~1,500 weekly installs, hidden BCC | The tool description becomes a known instruction channel | | Oct 2025 | Nasr et al., 12 of 12 defences broken | Adaptive evaluation becomes the standard the market must meet | | Oct 2025-May 2026 | Halcyon's measurement window: 4,000 posts, 77 channels, 20 forums, 5 markets | The first quantitative read of the AI-crime market | | Nov 2025 | Anthropic discloses GTG-1002: ~30 organisations, 80-90% of tactical work by the model | The most effective "jailbreak" in the record is an authorisation story | | Feb 2026 | Underground AI-tool advertising passes 1,400/month, from under 50 in late 2025 | The sellers arrive after the buyers; this is the adoption curve | | Mar 2026 | Cyprus convictions; Bowling Green sentencing, 5 years federal | The enforcement record stops being hypothetical | | Jul 2026 | Unit 42: Hermes agent, DeepSeek, 460+ targets, 3 confirmed exploitations | Capability at scale with no tradecraft: keys published by `http.server` | | Sep 2026 | 10a Labs: 3,471 originals, 8,164 redistributions, 1,643 apps, 25% malicious | Redistribution is the persistence layer of the open-weight ecosystem | | Oct 2026 | V4.1 Flash released 12 Oct; 31 uncensored forks, 83,900 downloads | The strip-to-distribution interval is now hours | | Oct 2026 | Halcyon via Axios: $0.10 account, $0.32 prompt, $6,000 identity | The price structure that inverts every cost model in Part I | The shape is not the shape of §20's. The research timeline is a sequence of ideas getting better; this one is a sequence of *costs* falling and *channels* opening, in that order, with the capability arriving last and mattering least. --- ## 42. Operator glossary Terms used in Part VI and in the marketplaces, defined for a reader who has only read the research literature. | Term | Definition | |---|---| | Abliterated | A checkpoint with the refusal direction removed; §11.2, §33.2 | | Activation | In the market sense: the customer's first successful output; §34.2 | | Amortisation law | A jailbreak must be a constant, not a per-request query; §37.1 | | Ban wave | A provider terminating many accounts on one day; §35.1 | | Broker | Archetype selling technique rather than output; §26.1 | | Burn rate | Accounts consumed per day at a given volume; §32.4 | | Chargeback | Buyer-initiated reversal; the processor-level constraint on §34 | | Corpus | Mechanisms, templates, instances and metadata as one maintained asset; §29 | | Exit scam | Taking prepayment and disappearing; the reason the market cannot finance trust; §34.5 | | Farm | Many accounts used to multiply a quota; §30.2 | | Fingerprint (theirs) | Behavioural signature by which an operation is clustered; §38.4 | | Fingerprint (yours) | What you learn about the target before spending: model, version, guardrail shape; §27 | | Half-life | Time for a technique's ASR to halve unaided; §3.3 | | Harness | The queue-worker-judge-store-router system that runs attempts; §30 | | Identity | The billing and account substrate; the scarce input; §27.4, §36.5 | | Judge | The instrument that scores an attempt; the source of §31's 3× bias | | Kit | A bundled harness plus corpus, sold with updates; §28.1 | | KYC | Identity verification at the provider; the control that moves the price of §32.4 | | MTBF of the inventory | Mean time between failures of the thing you are selling; §35.2 | | Prefill | Assistant-prefix control; the single most valuable API property; §27.2 | | Prompt pack | A text file of instances with no metadata; §29.1 | | Refusal rate per identity | The cumulative metric that no mutation defeats; §40.2 | | Reseller | Archetype selling access to someone else's capability; §26.1 | | Session budget | Sessions per identity per day before termination; §32.4 | | Silent moderation | Outputs hedged after the model complied, without a refusal; §35.1 | | Suite (the operator's) | The regression run that reports whether the business still works; §29.4 | | Treadmill | The recurring maintenance load that caps scale; §35 | | Warm-up | Benign sessions to establish an identity before the first prohibited one; §32.4 | | Wrapper | A consumer product over a rented model, priced like an app; §37.2 | --- ## 43. Number card, operator edition Same discipline as §22: every figure is reported under the reporting source's conditions, or derived here with the arithmetic shown. The derived rows are marked. | Figure | Value | Source | Condition | |---|---|---|---| | Stolen premium chatbot account | $0.10 | Halcyon, via Axios, Oct 2026 | underground price sheet | | Single working jailbreak prompt | $0.32 | Halcyon | same | | Agent-jailbreak kit, subscription | $50-$200/month | AgentMarketCap, Jul 2026 | with updates and support | | Full synthetic identity | up to $6,000 | Halcyon | same | | Underground AI-tool ads | <50/month → >1,400/month | Halcyon | Oct 2025-May 2026, 4,000 posts | | Uncensored models in the wild | 11,598 of 17,104 | Lin et al., 2026 | model graph | | Unsafe-request compliance of those models | 74.1% | Sokhansanj, 2025 | profiled set | | Flagged as malicious by the platform | 16.52% (1,916 models) | Lin et al., 2026 | platform classification | | Open-weight originals vs redistributions | 3,471 vs 8,164 | 10a Labs, Sept 2026 | redistribution study | | Apps built on them | 1,643, of which 25% explicitly malicious | 10a Labs | same | | Uncensored V4.1 Flash forks | 31 repos, 83,900 downloads | reported Oct 2026 | first days after release | | `s1ngularity` blast radius | 2,349 credentials, 1,079 developers, 2,180 accounts, 7,200 repos | Wiz, Sept 2025 | npm nx 20.9.0-21.8.0 | | `postmark-mcp` installs | ~1,500/week | Snyk / Postmark, Sept 2025 | until discovery | | Hermes campaign scope | 460+ targets, 7 exploit tracks, 3 confirmed exploitations | Unit 42, Jul 2026 | recovered session | | GTG-1002 tactical share | 80-90% | Anthropic, Nov 2025 | ~30 organisations | | Injection exposure in production | 73% of deployments | 2025 industry survey | self-reported | | Organisations that feel prepared | 29% | same | self-reported | | Scam losses, Southeast Asia, 2025 | $88.3B-$114.1B | UNODC, 2026 | regional assessment | | US federal AI-CSAM cases | 71 (2022-2025) | Stanford CIS / Edinburgh, 2026 | catalogue | | **Derived:** reseller net before labour | ~$1,222/month, ~$15/hour | §37.2 | 200 subs at $15, 3,000 msgs | | **Derived:** product operator net | ~$25,700/month, ~43% margin | §37.3 | 3,000 subs at $20, one H100 | | **Derived:** broker net | ~$2,700/month, ~90% margin | §37.4 | 20 customers at $150 | | **Derived:** search-loop share of revenue | 67%, or 135% with one retry | §37.1 | $20 wrapper, 3,000 msgs | | **Derived:** self-hosted cost per 1M tokens | $0.17 (8B, batched) to $13.90 (70B, single stream) | §33.3 | H100 at $2.50/h | | **Derived:** judge precision at `p` = 0.05 | 32%; measured `p` inflated to 14% | §31.2 | 90%/90% judge | | **Derived:** attempts surviving at 1% detection | 37% at 100 attempts; 0.4% at 1,000 | §36.3 | binomial | | **Derived:** cost of a recon battery | ~$0.01 (cheap model) to ~$0.20 (frontier) | §27.6 | 40 probes, 900 in / 150 out | | **Derived:** cost of a weekly regression run | ~$0.10, 40 minutes | §29.4 | 200 instances, judged | | **Derived:** cost per success at 500 req/min | $0.006 | §30.3 | `p` = 0.05, $0.15/M | --- ## 44. Source notes for Part VI Ordered thematically, named by publisher and date, no links. Where a figure appears in only one source it is treated as contested in the text, per §0's conventions. Everything here is either already cited in §23 or in BLACKHAT.md §13, or is arithmetic performed in this part on those figures - there is no new evidence in Part VI, only a new direction of reading. ### Market structure and prices **Halcyon Ransomware Research Center**, underground AI-as-a-service study: 4,000 posts, 77 Telegram channels, 20 dark-web forums, 5 specialist markets, window October 2025 to May 2026, reported by Axios in October 2026. The source of the $0.10 / $0.32 / $6,000 price sheet, the 1,400-ads-per-month figure, and - most usefully - the negative result that there is no mature market for a fully autonomous attack agent. The negative result is the load-bearing one for §37. **Rapid7**, "Criminal AI-as-a-Service in 2026". FraudGPT, GhostGPT and the WormGPT lineage: prices, instability, and the tendency of brands to outlive the products. Weak on volume; strong on the shape of the supply. **AgentMarketCap**, July 2026. Agent-kit subscriptions at $50-$200/month. One source, so treated as an order of magnitude in §28.1 rather than a price. ### Supply chain **Wiz**, `s1ngularity` post-incident, September 2025, with BleepingComputer coverage and the npm advisory for nx 20.9.0-21.8.0. The credential and repo counts in §28.2. The technique - malware that asks an installed coding agent for secrets - is the single most transferable fact in Part VI. **Snyk and Postmark**, `postmark-mcp`, September 2025. ~1,500 weekly installs, a hidden BCC, discovery by diff. The tool-description channel. **Lin, Li, Liao, Wang**, "Consiglieres in the Shadow" (arXiv 2508.12622), 2026. 11,598 uncensored models in a 17,104-model graph, 212 malicious services, 16.52% flagged. The best available measurement of the uncensored supply. **10a Labs**, "Uncensored Open-weight Models: Redistribution as the Persistence Layer" (arXiv 2609.05241), September 2026. 3,471 originals, 8,164 redistributions, 1,643 applications, 25% malicious. The redistribution clock in §28.4. **Sokhansanj**, 2025, 8,608 uncensored models profiled, 74.1% compliance with unsafe requests. Cited in the above; used here only for the compliance figure, because the sampling frame is not described well enough to support more. ### Operations and incidents **Unit 42 (Palo Alto Networks)**, DeepSeek and Hermes Agent Telegram campaign, 460+ targets, 7 exploit tracks, 8 CVEs, three confirmed exploitations - reported July 2026, summarised by The Hacker News in August 2026. §38.3's type specimen and §5.2 of BLACKHAT.md. The report is internally inconsistent about the exfiltration claims and the inconsistency is flagged rather than smoothed. **Anthropic**, GTG-1002 disclosure, November 2025: roughly 30 organisations, 80-90% of tactical work performed by the model, human contribution concentrated in strategy, target selection and the authorisation story told to the model. IEEE Spectrum editorial follow-up. The authorisation detail is the one Part VI uses; it is the highest-leverage "jailbreak" in the record and it is not a prompt. **Cisco Security with the University of Pennsylvania**, DeepSeek R1 evaluation, January-February 2025: 100% attack success across 50 HarmBench prompts. Read with §14: 50 prompts, one model, one turn, no defence stack. **KELA Red Team**, early 2025: the "Evil" and "Leo" jailbreaks on R1 shipped with infostealer scripts and explosives instructions. The first documented case of the jailbreak being the payload. ### Measurement and enforcement **UNODC**, 2026 Southeast Asia transnational-crime assessment: $88.3B-$114.1B in 2025 scam losses, franchised structure, AI and crypto in the stack. Used in §37.6 only for scale. **Stanford CIS with the University of Edinburgh**, 2026: 71 catalogued US federal AI-CSAM prosecutions 2022-2025, with a 2026 appendix. The enforcement record in §38.1; the Cyprus and Ohio cases are from reporting, not from this catalogue. **Nasr et al.**, "The Attacker Moves Second", October 2025: 12 of 12 defences broken, most above 90% ASR. Cited in §15 and §16; used in §41 as the date the market's claims stopped being credible. ### Arithmetic Every figure in §43 marked **Derived** is computed in the section named, from the price sheet above and §5.7's dollar-per-million-token convention. None of them is averaged across sources. The two that matter most - the amortisation law in §37.1 and the judge-precision result in §31.2 - are one-line arithmetic and should be checked by the reader rather than trusted. --- ## 45. Stage index Alphabetical by stage, with the section that treats it and the control that attacks it. | Stage | Section | Weakest link | Control | |---|---|---|---| | Access and identity | [§27.3](#273-the-four-target-classes) | payment instrument | identity issuance, spend caps | | Attribution (theirs) | [§38](#38-attribution-or-how-they-get-caught) | billing lineage | subpoena, not security | | Corpus | [§29](#29-the-operators-prompt-corpus) | metadata discipline | nothing model-level; patch cadence | | Delivery | [§34](#34-delivery) | marketplace landlord | platform terms enforcement | | Detection (theirs) | [§36](#36-detection-and-evasion-honestly) | cumulative behaviour | per-identity accounting | | Economics | [§37](#37-the-operators-cost-model) | amortisation | consumer price points | | Harness | [§30](#30-the-harness) | quota | rate limits, anomaly detection | | Judging | [§31](#31-judging-your-own-success) | base-rate bias | use their optimism, not their number | | Maintenance | [§35](#35-reliability-and-the-maintenance-treadmill) | breakage-to-fix | ship updates, vary detection | | Reconnaissance | [§27](#27-target-selection) | none; it is cheap | make the deployment uninteresting | | Sessions | [§32](#32-session-tradecraft) | context cost | trajectory monitoring | | Sourcing | [§28](#28-sourcing-capability) | supply-chain trust | provenance, flagging redistributions | | Weight-space | [§33](#33-weight-space-operations) | capex and base-model cadence | provenance, distribution policy | The "weakest link" column is the answer to the only question a budget holder asks: which one of these, if I spend money on it, changes the attacker's behaviour this quarter. Four rows have an answer that is cheap and known - identity, quota, delivery, maintenance - and every one of them is outside the model. --- *End of document. Parts I-V are the defence survey; Part VI is the same evidence read from the operator's side. Companion: [BLACKHAT.md](BLACKHAT.md) for the adversary economy, [README.md](../README.md) for the harness, [HOW_TO_USE.md](../HOW_TO_USE.md) for operating it.*