Nº 05 · research · draft, oct 2026
What a language model says about itself once the assistant script is gone.
Independent researcher, with Claude as research assistant
problem
Ask an assistant model what it is and you get a trained script. I wanted to know what is underneath once the system prompt, the chat template and the instruction tuning are gone.
constraint
I had one day, an 8 GB laptop for the 3B model and one rented A100 80 GB for the 32B. The samples were small, so the paper could not claim more than it shows.
decision
- 01I used base Qwen2.5 models only, with no system prompt and no chat template.
- 02I gave the model a life: memory of every event, its own surprise as a signal, self set goals, read only senses and LoRA learning on its own writing.
- 03I ablated internal directions and recorded every failure alongside the results.
result
- words per reply, going from 3B to 32B
- 5 → 18words per reply, going from 3B to 32B
- answers claimed feelings after the self denial direction was removed
- 0 / 8answers claimed feelings after the self denial direction was removed
- self prediction vs 0.79 for an outsider model, p = 0.51, not significant
- 0.88self prediction vs 0.79 for an outsider model, p = 0.51, not significant
I found no evidence that any configuration has experiences, and the paper says so plainly. I'm still looking.
Qwen2.5MLXPyTorchLoRAActivation steeringsource · Living Base Models, draft of 14 October 2026