Reservoir Agent

A transformer with a genuine time axis. Standard transformers are stateless between forward passes — they re-derive everything from the context window each call. This project injects a fixed, randomly-initialized reservoir into a pretrained transformer's mid-layer attention so its state evolves across passes, accumulating a history of the model's own attention dynamics. This session is a feasibility + dynamics study at small scale: can the injection be done without breaking the base model, and what reservoir regime turns the carried state into usable signal rather than noise?

Research question Can a fixed, randomly-initialized reservoir injected into a pretrained transformer's mid-layer attention give the model genuine state between forward passes — a real time axis — without degrading its base capabilities, and what reservoir-dynamics regime (spectral radius, size, injection depth) makes that injected state usable signal rather than noise?
Reservoir Agent architecture: input tokens flow through frozen lower layers into a mid-depth injection layer Lk where attention runs jointly over tokens and reservoir nodes; the reservoir reads the attention matrix (fixed W_in) and writes its state back (learned readout) each forward pass, with state persisting across passes; upper layers are fine-tuned; reservoir weights are random and fixed, and the best of N seeds is kept.
The architecture. Every forward pass is one reservoir tick: at the injection layer Lk, attention runs over the token hidden states and the reservoir nodes (extra keys/values). The reservoir reads the attention matrix through fixed W_in and writes its state back through a learned readout — both at the same layer, every pass — so state accumulates across passes (“implicit elapsed time”). Only the transformer’s fine-tune changes; reservoir weights are random and fixed, and the best of N seeds is kept.
Read
The findings
The results of the investigation. ↓
Download
The report (PDF)
Full write-up, typeset. ↓
Source
GitHub repository
Code, data, and the literature review. ↗

What this project does

1 · The question

Whether a fixed random reservoir, injected into a pretrained transformer's mid-layer attention, can give it real cross-pass state without breaking the base model — and which reservoir-dynamics regime makes that state useful.

2 · Grounded in the literature

It relocates the proven reservoir-computing recipe (fix the recurrent weights, train only a readout — Jaeger’s echo state networks, Maass’s liquid state machines) into a pretrained transformer, and targets a gap the expressivity literature makes precise: a finite-precision transformer is bounded to TC⁰/FO(M) per forward pass, while state carried across passes is the documented lever past that ceiling. Every prior recurrence-augmented transformer (Transformer-XL, RMT, Block-Recurrent, Mamba, Titans…) uses trained recurrence carrying state within a sequence; none uses a fixed-random reservoir with state across independent passes. Full survey: literature/REVIEW.md.

3 · The finding

In progress. First result: the reservoir's echo state property breaks sharply at spectral radius ρ ≈ 1 (the edge-of-chaos boundary) in the autonomous regime — see Findings. Next: model surgery (H1 non-destruction) and the sweep on real GPT-2 attention streams.

How the reservoir enters attention

At the neuron level, the mechanism is small: the reservoir nodes simply join the attention layer's key/value sequence. The same attention that moves information between token residual streams now also reads from and writes to the reservoir — via a fixed projection in and a learned readout out — and unlike the token streams, the reservoir state carries across forward passes.

A residual-stream view: five token columns (The, capital, of, France, is) of gray residual nodes with green FFN-query neurons, a blue attention layer whose neurons attend toward the final column to predict 'Paris', and pink FFN-value neurons; a reservoir column of fixed random amber nodes joins the attention layer as extra keys/values, read via W_in and written via W_out, with a recurrence loop and state that persists across passes.
The mechanism, at the neuron level. A reservoir-augmented view of the classic residual-stream picture: the attention layer that predicts “Paris” also takes the fixed-random reservoir nodes as extra keys/values (read via W_in, written via the learned W_out), and the reservoir state — unlike the token streams — persists across passes.

The always-alive runtime

The architecture implies a different execution model from standard inference. A stateless transformer is a request–response handler; the Reservoir Agent is a persistent, always-alive process whose reservoir state and context buffer are owned by the runtime and never wiped between passes. A scheduler decides when to run a forward pass — prompted (new input arrives) or unprompted (an idle timer fires) — and an output gate decides whether to emit or stay silent. Forking a standard agent harness into this shape is long-horizon work (tracked in todo.md); this session targets the model surgery and dynamics beneath it.

The always-alive runtime: user or harness input feeds a pass scheduler that decides when to run a forward pass (prompted or unprompted, driven by an idle timer); a persistent context buffer and a GPU-resident reservoir state store (mutated in place each pass) feed the forward pass; an output decision either emits a high-confidence token or stays silent, updates state, and schedules the next pass.
The runtime. The reservoir state store (r(t), pinned in GPU memory, mutated in place each pass) and a never-wiped context buffer persist across passes. Unprompted passes let the agent keep processing with no new input; the output gate emits only when confident, otherwise it updates state and schedules the next pass. (Aspirational / compute-gated — see todo.md.)

Findings

In progress. The full write-up lives in FINDINGS.md and is built into a typeset PDF on every push. It states the question, architecture, literature grounding, and method now, and reports experimental results — the reservoir-dynamics characterization and the H1 non-destruction regression — as they land. No result is claimed here until it has been measured.

First result (synthetic-input dynamics sweep). Driving a 200-unit reservoir across spectral radius ρ ∈ [0.1, 2.0], the echo state property — the reservoir forgetting its initial condition — holds cleanly for ρ < 1 and breaks sharply at ρ ≈ 1 (gold curve), exactly the edge-of-chaos boundary the classical theory predicts, now measured in this injection-oriented setup. Saturation and effective dimensionality rise smoothly with ρ. A nuance worth flagging: under unit-scale input drive the reservoir forgets its initial state across all ρ (strong input enforces the ESP), so the ρ ≈ 1 boundary is the regime that matters for unprompted, input-free passes — precisely where the agent runs on reservoir state alone.

Reservoir dynamics versus spectral radius: init-forgetting (echo state property) is near zero below rho=1 and jumps sharply above it; saturation and participation ratio rise smoothly with rho; input separation rises steadily.
Reservoir dynamics vs. spectral radius (synthetic input, K=200). Autonomous init-forgetting transitions sharply at ρ ≈ 1 (the echo-state boundary); saturation stays low and dimensionality rises with ρ. Regenerate with python scripts/run.py sweep.