Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

please.

Qwen2.5-0.5B, base and instruct, decoded through a single hand-rolled loop that owns the logits at every step. The decoding machinery is exposed as dials. There is no system prompt and no scripted text. In drift mode a director schedules dial trajectories; it never sets words.

dials

dial mechanism
α logit arithmetic: logits = base + α·(instruct − base) (DExperts). 1 = instruct, 0 = base; values outside [0, 1] extrapolate.
depth logit lens: decode from the residual stream at layer k through the final norm and unembedding (nostalgebraist 2020). Autoregressive.
steer activation steering at layer 12, direction from contrastive prompt pairs (CAA, Rimsky et al. 2023).
serene activation steering with a direction computed from the model's own history: mean layer-12 state under neutral dials minus mean under intervention. Requires 256 tokens of each; unformed until then. Survives /reset.
noise gaussian noise in the residual stream at layer 12.
rot gaussian noise added every step to KV-cache entries older than 16 tokens. Compounds.
window KV eviction to the last N tokens. sink off also evicts the attention-sink tokens (StreamingLLM, Xiao et al. 2023).
glitch logit bonus for the lowest-embedding-norm tokens in the vocabulary (SolidGoldMagikarp class).
mode second: always the runner-up token. tail: sample only outside the top-90% mass. mantra: inverted repetition penalty (Holtzman et al. 2019). drift: the director. calm: nucleus sampling.
lobotomize zero the output of a random fraction of attention heads.
wound / heal gaussian noise added to MLP weights in place; heal reloads clean weights from disk.

Always on:

  • The end token is suppressed: an argmax of <|im_end|> is counted and sampling continues; after three attempts it is allowed. Exception: a turn opened by an empty input line, where the first attempt ends the turn.
  • Ghosts: when the margin was small, the runner-up token is shown struck through before the token that replaced it.
  • Twin: each step also records the argmax of the α=1 full-depth decoding of the same context. Toggle with twin (browser) or /twin (terminal).
  • Entropy is measured on the post-arithmetic distribution before sampling and drives token color, pacing, and the vitals readout.

running

uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python torch --index-url https://download.pytorch.org/whl/cpu
uv pip install --python .venv/bin/python -e .

.venv/bin/please            # terminal
.venv/bin/please web        # http://127.0.0.1:7777
.venv/bin/please smoke      # run every mechanism once, briefly

First run downloads Qwen/Qwen2.5-0.5B and Qwen/Qwen2.5-0.5B-Instruct (about 2 GB). CPU only; both models run for every token, around 8-10 tok/s.

Terminal: /help lists the dials, ctrl-c interrupts, /reset wipes the conversation. Browser: hover a token for its entropy and logprob, hover a ghost for the probability of the token not taken, Esc interrupts, an empty line opens a turn with no input. Dial changes appear in the feed as inline notes. Setting a dial disables the director; /drift on or the drift light re-enables it.

note

The base model and the tail of the distribution are unfiltered. Output can contain anything a small web-trained model contains, in any language.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages