Model extraction via query interfaces¶

Extraction represents an attack executed through the front door of a machine learning system. Every query sent to an application interface returns a response, and every response carries data reflecting the internal function that produced it. Gathering sufficient query outputs builds a functional copy of the target model, allowing adversaries to plan subsequent actions offline. The interface performs precisely as designed, operating one query at a time without breaking security perimeters.

Learned models are uniquely vulnerable because the trained model embodies both the operational value and the output channel. Traditional rules can be published without compromising underlying proprietary data, whereas a trained model embeds data, labelling effort, and parameter tuning, leaking a sample of all three with every prediction. High development costs combined with low query costs create the economic gap where extraction thrives.

Exposed query interfaces¶

Attack surfaces span standard application programming interfaces:

• Moderation and abuse classifiers accessible via preview endpoints.

• Fraud and risk scoring engines returning outputs directly to evaluated entities.

• Pricing, ranking, and recommendation models with observable outputs.

• Any model interface where callers observe labels, confidence scores, or final outcomes.

Richer responses, such as full probability vectors instead of binary labels, accelerate the extraction process by leaking higher-fidelity data per query.

Constructing a functional replica¶

Adversaries transform observable outputs into working models through several structured techniques:

• Functional extraction involves submitting broad query streams, recording responses, and training a substitute model on input and output pairs. The substitute requires output alignment rather than architectural identity, granting attackers an unlimited testing environment without detection risks.

• Offline boundary mapping combines extraction with evasion. Reconstructing a local decision boundary allows attackers to test adversarial inputs against the clone without query limits or detection traces, deploying polished attacks in a single clean pass against the live system.

• Threshold recovery utilises confidence scores. Walking an input across a decision threshold while monitoring score shifts maps boundary locations and steepness precisely.

• Logic harvesting targets commercial models. Varying single factors in pricing or risk queries exposes how individual variables drive outcomes, revealing proprietary business logic without code disclosure.

• Sanitisation bypasses format controls. Input validation filters check for malformed data or injection strings, but extraction queries remain entirely valid and well-formed. The attack exists in aggregate patterns across thousands of legitimate calls rather than in individual requests.

Operational applications in commerce and defence¶

State actors, commercial competitors, and criminal groups apply extraction to acquire proprietary capabilities.

Competitor model extraction via APIs¶

Enterprises training large language models face a structural trade-off where commercial accuracy enables extraction. Competitors expend a fraction of original training costs through API query fees, harvesting input and output datasets to train substitute models. The resulting clone reproduces learned behaviour, transferring proprietary utility across public output channels. Legal disputes regarding this boundary remain unsettled, often allowing cloned models to achieve wide circulation before proceedings conclude.

Acoustic classification via crafted queries¶

Classifiers analysing submarine acoustic signatures, trained on classified historical recordings, represent prime extraction targets. Surveillance platforms submitting varied acoustic probes accumulate datasets to train substitute classifiers. Adversaries design propulsion signatures optimised to trigger misclassifications in both the clone and the original system, bypassing direct network access entirely.

Targeting system extraction via captured terminals¶

Proprietary targeting systems operated through field APIs remain vulnerable if adversaries capture a physical terminal. Feeding battlefield imagery through the terminal records classifications, enabling substitute training at a remove. Adversaries test camouflage designs, decoy placements, and formation geometries against the clone prior to field deployment.

Graph neural network extraction efficiencies¶

Work published in November 2025 demonstrated that graph neural networks can be stolen under a severely restricted query budget, and with an extraction defence already in place. Recovering the model backbone without direct queries first, then spending a small query allowance on the most informative inputs, is what makes it work. Query efficiency of that order widens the range of models exposed to extraction and blunts rate limiting as a control.

Pattern extraction for malware development¶

The GREYVIBE group leverages commercial artificial intelligence APIs to generate malware components and post-compromise tooling. Extracting behavioural patterns from commercial APIs reduces reliance on reused code signatures, which are primary indicators for attribution analysis.

Detection limits and strategic constraints¶

Detecting extraction remains challenging because individual queries resemble ordinary user activity. Adversaries distribute probes across mundane tasks like translation or code assistance, rendering single-request filtering ineffective. Rate limiting slows collection without preventing it, and adjusting query pacing bypasses simple throttles.

Accuracy and extractability move in tandem. Systems returning precise probabilities provide higher utility to legitimate users while accelerating extraction. Conversely, reducing output precision or adding noise degrades product quality.

Limiting information yield¶

Defenders manage extraction risks through specific operational controls:

• Monitoring account-level query patterns to identify systematic input space sweeps, recognising that genuine users rarely query uniformly across decision boundaries.

• Restricting response granularity by returning coarse risk bands or bare labels instead of calibrated probability vectors, withholding the data needed for precise boundary reconstruction.

• Enforcing strict rate limits and per-caller query budgets aligned with legitimate operational use, increasing the economic cost of data harvesting.

• Anticipating the operational utility of a cloned model, allowing organisations to plan countermeasures based on what an adversary gains rather than relying solely on prevention.