A 100× drop in extracted reasoning still needs a durability budget
Anthropic says its new controls cut exposed reasoning tokens by 10 to more than 100 times. The useful procurement question is how long that reduction survives adaptive attack—and what legitimate users pay meanwhile.
What happened
Anthropic reported new access controls, classifiers and account restrictions for illicit model-distillation attempts, alongside a sub-0.1% false-positive claim and explicit customer trade-offs.
Why it matters
A large immediate reduction is not the same as a stable control. Buyers need a time-bounded retest, bypass discovery rate and legitimate-user loss budget before treating the safeguard as durable.
Anthropic’s 12 October report says a two-stage classifier, tighter reasoning access and restrictions on repeated attempts reduced reasoning tokens extracted on its platform by a factor of 10 to more than 100 after full deployment. It says the classifier catches the techniques described in its September report, has a false-positive rate below 0.1%, and helped ban tens of thousands of accounts. Reuters independently reported the announcement as an estimated 90% to 99% reduction.
Those numbers are useful but bounded. Anthropic measures tokens it estimates attackers extracted on its own platform, not every successful transfer of capability across providers. The company says it has not yet observed a highly sophisticated, persistent attempt against the new controls and does not know how long the gains will last. It also says some integrations needed changes, benign requests can be blocked, and blocked requests in specified low-false-positive categories are billed.
Buy a retest schedule, not a permanent percentage
For any control claim, record the measurement window, attack families represented, bypasses discovered after launch, time to patch, repeat-offender detection time and customer impact. Predeclare a 30-, 60- and 90-day retest with fresh adversarial prompts and traffic spread across accounts and providers. Track the exposed-token rate and successful capability transfer separately.
Do not convert “10 to 100× lower” into “distillation solved.” A defensible decision rule is narrower: continue relying on the control only while the reduction survives new attacks, legitimate-user blocks remain within an agreed loss budget and a bypass triggers an owned patch-and-notification path. Link that evidence to model evaluation, not to an indefinite vendor assurance.