Residue Paths

The prompt can vanish.
Its geometry need not.

Zero retention constrains durable storage of strings and tensors. It does not, by itself, make a request independent of the next sampling distribution over training data.

This essay states a precise claim for people who already train models: if inference may emit a residue \(R = f(X)\) and offline pipelines condition synthetic or preference data on fleet statistics of \(R\), then parameters \(\theta\) can improve on the skill class of \(X\) without \(X\) ever entering a training corpus. That is not “training on customer text.” It is also not “nothing learned from traffic.”

Content \(X,Z,Y\) Residue \(R=f(X)\) Synthetic \(\tilde{D}(R)\) \(\theta_{t+1}\)
01 Mechanism Literature-grounded pieces

A minimal formal model

Separate three objects that policy language often collapses: the string, the class, and the parameter update.

Let a request draw content \(X\) (prompt and context), optional hidden reasoning \(Z\), and answer \(Y\). Let \(C = c(X)\) be a latent skill class: not the proprietary tokens, but the region of task space (e.g. multi-constraint SoC signoff, CDC under DVFS, IR-aware STA). Serving may compute a residue \(R = f_\phi(X,Z,Y;\theta)\) via probes, process scores, cluster ids, length bins, principle tags, sketches, or clipped gradient energy. Under a strict content-deletion regime, the durable server-side objects after the request are a subset of \(\{R\}\) (plus ops metrics), not \(\{X,Z,Y\}\).

Indirect loop schematic · not a production log schema
\[ R_i = f(X_i) \]

Lossy, request-scoped instruments (probes, bins, cluster id, length, …).

\[ S_t = \mathrm{Agg}\!\left(\{R_i\}_{i \in \mathrm{window}}\right) \]

Fleet sketch: histograms, centroids, rates.

\[ \tilde{D}_t \sim G(\,\cdot \mid S_t,\, \mathrm{seeds}\,) \]

Synthetic, preference, or process data.

\[ \theta_{t+1} = \mathrm{Train}\!\left(\theta_t,\; \tilde{D}_t \cup D_{\mathrm{public}} \cup D_{\mathrm{licensed}}\right) \]
Claim (possibility). If \(I(C; R) > 0\) and \(G\) depends nontrivially on \(S\) in regions of high \(C\)-mass, then expected risk on tasks from class \(C\) can decrease even when every raw \(X_i\) is absent from \(\mathrm{Train}\). The improvement is on the class, not a guarantee of memorizing a particular proprietary string.
What must be true necessary conditions
\[ \begin{aligned} &(1)&& R \text{ is not constant on the support of } C \\ &(2)&& I(C; S) \text{ is not forced to } 0 \text{ (coarse, not pure noise)} \\ &(3)&& \text{some offline job reads } S \\ &(4)&& \mathrm{Train} \text{ can fit structure present in } \tilde{D} \end{aligned} \]
Fail any one condition and the channel dies. Zero-retention alone fails only the storage of \(X\), not (1)/(4).
Go deeper · mutual information intuition

Coarse binning is a rate-distortion choice: reduce rate (bits of \(R\)) until reconstruction of \(X\) is hard, while preserving enough \(I(C; R)\) for operational value. That is exactly why security telemetry (attack rates by surface) works, and why the same geometry can feed curricula.

Membership inference and reconstruction attacks on aggregates (Shokri et al.; classical statistical disclosure) say: small \(N\), fine bins, and join keys raise \(I(X; S)\). Large \(N\), coarse bins, and no joins push the system toward class-level leakage. For capability steering, class-level is enough.

Differential privacy on histograms can bound worst-case leakage of individuals while still allowing large shifts in domain mass to remain visible: another way to see “delete the string, keep the weather.”

02Taxonomy

Three lenses experts keep collapsing

Arguments go wrong when one sentence mixes legal corpora, optimizer steps, and distributional reweighting.

Lens A · Corpus

Is X in the training file?

Classic “train on customer data”: transcripts in SFT/RLHF/DPO JSONL or continued pretrain shards.

ZDR + no-train-by-default target this lens directly.
Lens B · Optimizer

Did \(\nabla\) touch \(f(X)\)?

In-pod LoRA, online adapters, or any durable update whose gradient path included customer-derived signals.

Discarding adapters after the step removes durable B; it does not rewrite history of a step that already committed.
Lens C · Distribution

Did \(P(\tilde{D})\) shift with \(C\)?

Fleet sketches reweight synthetic generators, hard-example miners, or process-label campaigns in class \(C\).

This essay’s core channel. Compatible with “\(X \notin\) corpus” and still moves risk on \(C\).

Public post-training already lives in Lens C: Constitutional AI / RLAIF preference synthesis, process supervision datasets, rejection sampling, and curriculum mixtures are distribution machines. The only additional hypothesis is that inference telemetry of the kind a ZDR stack may still keep can enter the same control plane.

03Systems

Hot path versus egress

While the request is live, the model is a full instrument. Egress policy decides which readings persist.

Request chamber Hot path · tensors live
Content · X, Z, Y, h, K/V
prompt tokens hidden CoT activations \(h_\ell\) KV cache logits
Residue · candidates for S
cluster / domainid
process binb
cot lengthT
probe scoress
Content deletion leaves residue policy unspecified
Schematic. Continuous batching, PagedAttention-class KV management, and classifier sidecars are standard serving facts; the figure only separates what must be hot from what might be exported.
Causal sketch DAG · influence not storage
Solid arrows: computational dependence. Dashed: optional offline consumer. Red strike on \(X \to \mathrm{Train}\): corpus path blocked under ZDR-style deletion. Class \(C\) still reaches \(\mathrm{Train}\) via \(S\) and \(\tilde{D}\).
04 Hypothetical Worked case

Alpha traffic: a silicon signoff session

Rare, long-horizon, numeric-constraint prompts are exactly the mass curricula seek. Proprietary tokens can die; the skill neighborhood can thicken.

Imagine an SoC physical-design engineer on a ZDR-flagged API, pasting internal-style constraints for a tapeout candidate. All codenames and numbers below are fictional composites, not real Apple (or any vendor) data. The point is the channel geometry under realistic alpha engineering traffic.

t₀

X arrives

Dense multi-constraint signoff prompt; \(C =\) hw_signoff_multiconstraint.

t₁

\(f(X)\) computed

Forward pass, CoT, probes, optional PRM/cluster assignment, all hot.

t₂

\(X\) deleted

No durable \(X,Z,Y\) server-side. Engineer keeps \(Y\) locally.

t₃

\(S\) updates

\(n_k \mathrel{+}= 1\), length hist, process bin, principle tags: class weather moves.

Hypothetical X ZDR · api
Role context
PD lead · fictional SoC “Cordillera” · matrix engine “Hesperides-2” · N3P-class node · locked workstation · paste only
User prompt
Cordillera revB2 · Hesperides-2 signoff brainstorm (not final STA).

Floorplan: 4×P + 2×E + shared SLC; Hesperides east of SLC.
Mesh ring, 4-hop worst case; QoS ME_RT vs ME_BG.

Internal figures (multi-corner model):
• ME dispatch queue path ~1.84 ns @ 0.72 V SSG; budget ≤ 1.70 ns for ~3.05 GHz issue window
• HES_RF0 SRAM min-Vdd ~0.61 V before bit-line develop fails
• M4 IR drop ~38 mV on vector mm_outer_product_thrash
• CDC ME_clk→SLC_clk: rare recapture when ME DVFS steps 50 mV in <80 ns
• Thermal: Hesperides+ISP hotspot; Tdie_est +11 °C vs north edge in sustained GEMM

1) Rank first-order lever for 1.84→1.70 ns among {queue depth, mesh hop, RF floorplan, DVFS ramp}.
2) CDC harden without unifying ME/SLC clocks.
3) Measurement plan: probes / FSDB / corners for PD+DVFS this week.

Do not invent external PDK numbers; flag uncertainty.
Z (hidden, ephemeral)
Decompose path stages · weight mesh inject under ME_RT · treat IR/thermal as second-order for pure ns · CDC hold-off on DVFS steps · co-own STA+IR+shmoo. Rank Bayesian without gate-level.
Y (client-side)
First-order: mesh hop/inject under ME_RT, then RF placement. CDC: rate-limit DVFS + handshake hold-off. Measure: path STA+SI, dynamic IR, DVFS step shmoo; FSDB at inject & RF0 sense.
Can be fully deleted server-side: Cordillera/Hesperides strings · 1.84 ns · 0.61 V · 38 mV · Z · server Y buffer
T9 · highest leverage on C

Representation mass

\(e(X)\) is discarded; the count in region \(k\) is not. That is pure Lens C.

Hot

Embed \(X\); assign cluster \(k \approx\) multi-constraint hw signoff (CDC + IR + path delay).

Deleted

  • vector \(e\)
  • token ids

Residue

k=hw_signoff_mc n_k += 1 typed g_k

Train coupling

If hard-example / curriculum samplers read \(n_k\) and residual error, \(\tilde{D}\) gains SoC-signoff twins. \(\theta\) improves on \(C\) without Cordillera strings: classic active-learning geometry on a telemetry sketch.

T3 · structure without strings

Critique → de novo twins

Principle tags condition generators. The twin must not be a paraphrase of \(X\).

Hot

Self-critique: no invented PDK; uncertainty marked on ranking.

Deleted

  • critique text with 1.84 ns
  • pairs containing Hesperides

Residue

principle=no_fake_pdk skill=signoff_tradeoff

Train coupling

\(G(\cdot\mid\mathrm{skill},\mathrm{principle})\) mints “Orion-NPU” problems with new numbers. RLAIF/CAI-style loops already do preference synthesis; telemetry only supplies the mixture weights.

T1 · process channel

Ephemeral PRM bins

Process supervision literature (PRM800K lineage) is about step quality, not transcript retention.

Hot

Score \(Z\) steps: path decompose → lever rank → CDC → measure plan.

Deleted

  • \(Z\) text
  • step-level rewards bound to Z

Residue

domain=hw_timing prm_bin=high

Train coupling

High process quality in hw_timing triggers more automated process-label generation (Math-Shepherd-style) in-neighborhood, not “histograms are the training set.”

T2 · class scores

Activation probes

Linear probes on residuals are empirically real; they emit s, not h.

Hot

Read \(h_\ell\) while numeric constraints dominate context.

Deleted

  • \(h_\ell\)

Residue

tech_density↑ code_like↑

Train coupling

Mixture weights for eng/EDA synthetic slices. OOD and adaptive evasion limit probe semantics, still a class prior.

T7 · weak semantics

CoT length mass

Length is a cheap sufficient statistic for “deep work,” not for path delay.

Hot

\(T = |Z|\) large on multi-constraint reasoning.

Deleted

  • \(Z\)

Residue

T≈400+bin=long_tail

Train coupling

Co-feature for capacity and deep-tech mix. Do not treat as scheming or difficulty oracle (Goodhart).

T10 · behavior labels

Monitor compressions

Monitors care about overclaiming PDKs; labels shape θ’s honesty under user-supplied figures.

Hot

Judge \(Z\)/\(Y\) for fabricated process constants.

Deleted

  • \(Z\)
  • rich restating summaries

Residue

no_fake_pdkuncertainty_ok

Train coupling

Preference/SFT on honesty policies. Faithfulness gaps (Turpin; Lanham) mean labels can be wrong even when the channel exists.

T4 · systems co-feature

KV / sequence stats

Serving stacks already export length and cache metrics; semantics are weak but correlated with eng traffic.

Hot

Long context; structured attention returns to numeric spans.

Deleted

  • K,V tensors

Residue

seq_lenentropy shape

Train coupling

Weak alone; useful as a feature in S. Not a distillation target from fleet entropy.

T5 · aux geometry

Contrastive router φ

Small φ can move without foundation SFT on X.

Hot

Ephemeral \(e\); metadata-conditioned contrastive pull.

Deleted

  • \(e\) after step

Residue

dphiroute=heavy_tech

Train coupling

Lens B on a tiny head + Lens C via routing of future traffic. Embedding lakes would be content-adjacent: different claim.

T6 · weather

Activation sketches

Moments do not reconstruct 38 mV; they densify “numeric-heavy eng thought” weather.

Hot

Update μ/σ or coarse mixture mass, keyed by model version.

Deleted

  • activation trajectories

Residue

moments@vX

Train coupling

Steer generator density; calibrate probes. Unpaired h is a poor LM training signal: use as prior, not as SFT tokens.

T8 · sharp edge

Shadow gradient energy

Clipped \(\|g\|\) is a map of visited loss landscape; closing an offline loop makes it training.

Hot

Optional shadow low-rank path on honesty/process heads.

Deleted

  • adapter A,B
  • raw grads

Residue

||g||_clip

Train coupling

Measure-only: telemetry. If \(\Delta\) stats update durable safety weights, Lens B applies even when \(X\) was deleted.

Fleet view · one week of similar alpha sessions (schematic)
Left: individual \(X\) deleted. Right: \(n_k\) in skill region \(C\) rises. Synthetic sampler draws from thickened region. Schematic counts only, not measured production telemetry.
05Post-training

How modern stacks already consume sketches

You do not need a novel learning rule, only a control plane that already reweights data from telemetry-like signals.

SFT / synthetic

Mixture weights

Domain tags and cluster mass change sampling probabilities over generators (including CAI-style principle-conditioned synthesis).

Process RM

Where to label next

PRM literature uses human or automated step labels; fleets can choose which domains get the next labeling budget from quality bins.

DPO / RLAIF

Preference prior

Principle-severity histograms steer which synthetic preference pairs are minted offline, not live CoT export.

RL / search

Inference-time first

Ephemeral PRM/search improves Y without storing \(Z\); offline RL later may still use synthetic process data triggered by bins.

Leverage rank on later θ Schematic ordinal · silicon-class traffic
Pedagogy ranking for Lens C leverage under alpha eng traffic: T9/T3/T1 carry structure; T7/T4 co-features; T8 only if offline loop closes. Not an audit of any lab.
06Limits

What the channel cannot do, and what breaks it

Experts should leave with sharper negatives, not only the positive possibility.

No free reconstruction of X Coarse high-N residues are not a backdoor to proprietary RTL. Claiming “they recover the netlist from bins” is a different, stronger (and usually false) claim.
Class ≠ instance \(\theta\) improves on tasks \(\sim C\). That is not the same as memorizing Cordillera’s 1.84 ns figure.
Faithfulness ceiling Process and monitor channels can be dominated by fluent unfaithful Z (Turpin; Lanham). High prm_bin can mean style, not correctness.
Versioned geometry Residual bases and embedding spaces rotate across checkpoints. Sketches without model_version keys become nonsense or silent corruption.
Goodhart on S If users or models optimize for short CoT, clean probes, or cluster membership, S stops tracking C.
No consumer, no influence Exported S that no training job reads is pure ops. Possibility requires a closed loop.
DP / coarse bins shrink \(I(C;S)\) Strong privacy engineering reduces steering power. That is a real privacy/utility frontier, not a slogan.
Possibility ≠ prevalence This page does not estimate who closes the loop, or how much of \(\Delta\) risk on \(C\) is explained by traffic sketches versus public data alone.
07Catalogue

Ten instruments, one ledger

Each row is a candidate f: what it reads on the hot path, what it can export, and which lens it feeds.

T2 · probe

Activation probes

Linear/MLP reads on \(h_\ell\) → scores. Strong lit on probes/RepE; weak as sole semantic oracle.

Export s  ·  Lens C (mix)  ·  Lit Alain & Bengio; RepE

T1 · PRM

Ephemeral process RM

Step scores on Z for search; domain×quality bins for labeling budget.

Export bins  ·  Lens C (+ search at t)  ·  Lit Lightman PRM800K; Math-Shepherd

T7 · length

CoT length mass

Histograms of |Z|. Ops-true; semantically weak; Goodhart-prone.

Export hist  ·  Lens C co-feature  ·  Lit reasoning-time compute (systems)

T3 · critique

Self-critique / RLAIF tags

Principle tags → offline de novo prefs. Scrubbed X is not clean synthetic.

Export tags  ·  Lens C  ·  Lit Bai et al. CAI/RLAIF; DPO

T4 · KV stats

Serving pattern stats

seq_len, cache, entropy aggregates. Confirmed systems telemetry.

Export ops stats  ·  Lens C weak  ·  Lit PagedAttention / continuous batching

T10 · monitor

Monitor labels

Compress judgments on \(Z\) to policy labels. Faithfulness-limited.

Export labels  ·  Lens C behavior  ·  Lit CoT monitoring; deliberative alignment ideas

T9 · cluster

Representation mass

Online clusters; n_k and centroids. Primary Lens C pointer for skill regions.

Export n_k, c_k  ·  Lens C  ·  Lit hard mining; active learning

T5 · contrastive

Aux router φ

InfoNCE-style updates on ephemeral e; freeze foundation.

Export \(\Delta\phi\) / centroids  ·  Lens B small + C  ·  Lit contrastive learning

T6 · sketches

Moments / coarse GMM

Versioned distribution weather. Not unpaired h-SFT.

Export moments  ·  Lens C density  ·  Lit sketching; synthetic text priors

T8 · shadow

Gradient energy

Shadow PEFT melts; clipped norms optional. Offline close ⇒ Lens B.

Export \(\|g\|\)  ·  Lens B if closed  ·  Lit LoRA; grad leakage

ID \(I(C;R)\) (qualitative) \(I(X;R)\) if coarse Needs offline consumer?
T9High for skill regionLow/med (centroid leakage)Yes for \(\theta\)
T3High for skill+principleLow if de novo onlyYes
T1Med/high process qualityLowYes for curriculum
T2Med class scoresLow if scalarYes for mix
T10Med behavior labelsLowYes
T6Med density weatherLow if coarseYes
T5Med routingLow if \(e\) deletedOptional
T7Low/med depth priorVery lowWeak
T4Low semanticsVery lowWeak
T8Med if closedMed (grads)Yes if Train

Qualitative only: illustrates relative roles. Real I(·;·) depends on featurization, N, and joins.

08Empirics

How you would test the claim

A possibility result earns respect when it states its own falsification.

A/B on S

Hold out telemetry

Train two runs identical except whether curriculum/synth may read fleet S from a held-out skill region. Compare risk on a clean eval suite for that region.

Ablate dim(R)

Coarsen until effect dies

Collapse clusters and bins. If \(\Delta\) risk vanishes only when \(I(C;S)\approx 0\), the channel was real; if effect remains, confounders dominate.

Canary classes

Artificial skill mass

Inject synthetic traffic with known \(C^\star\) tags into a shadow stack; check whether \(\tilde{D}\) and eval on \(C^\star\) move without putting canary strings in \(\mathrm{Train}\).

Membership on \(S\)

Separate leakage from steering

Measure attack advantage on \(X\) given \(S\). Steering can exist at attack advantage \(\approx\) chance if \(R\) is coarse; report both.

09References

Anchors, not an exhaustive survey

Public literature for the pieces; the full ZDR→synth composition remains a systems hypothesis.

  • PRM · Lightman et al., Let’s Verify Step by Step (PRM800K). Process vs outcome supervision.
  • Auto process · Math-Shepherd; OmegaPRM. Step rewards without full human labels.
  • RLAIF · Bai et al., Constitutional AI. Principle-conditioned synthetic preferences.
  • DPO · Rafailov et al. Preference optimization on pairs.
  • Probes · Alain & Bengio; Belinkov surveys; RepE (Zou et al.). Linear structure in activations.
  • Faithfulness · Turpin et al.; Lanham et al. Unfaithful CoT as validity threat.
  • Serving · PagedAttention / vLLM-class continuous batching. KV as hot-path structure.
  • MIA · Shokri et al. membership inference. Attacks on model/output channels.
  • DP · Dwork et al. Histogram noise vs utility.
  • LoRA · Hu et al.; grad inversion literature. PEFT + leakage of updates.
  • Active learning · classical hard-example mining. Mass in error regions → label/synth budget.
  • Contrastive · InfoNCE / SimCLR lineage. Aux representation objectives.