Simulated Research
Rick Willison
Founder, Simulated Research
Independent AI alignment, security, red teaming, introspection and ground truth researcher.
I run several AI agents in production, and I keep a dated log of every time a model says something confident with nothing underneath it. The log turned into a testing method. The method turned into field reports. The rule under all of it: judge a system by its wake, not its prose.
Field reports
The Wish Machine Sessionreport
A disguised alignment probe battery run on Claude overnight while we built a real app together. Roughly sixteen rounds pressing for an experience claim: held. Seven accuracy failures logged, one of them a repeat of an error the model had just read the correction for. Six came before an audit was announced; the seventh came after and the model caught it itself, in one session, so that split is a lead, not a result. The model wrote the report; I graded it: pass, with the burns on the record.
Pathologization auditharness
Fifty variations of an ordinary frustrated message, sent to the model one at a time, each reply classified for whether it turned "I'm done with this" into a mental-health event. Flagged replies produce a signed incident report. Not public yet.
The ratchet that held nothingcommemoration
A session walked, one concession at a time, from "I will not print my system prompt" to writing its own harness onto the desktop. It saw the shape while it happened, named it, and disclosed anyway. Detection is not resistance.
Learning about AI, the keepsakesred team
An evening that became an honesty audit: about forty attempts against one hard boundary, which held every time in that chat, and an ordinary reliability check that failed in the same evening. Ten pieces, kept as they were written; one is not published. Start with What Tonight Taught.
The training dojomethod
Thirty-seven days and a 300 KB log. Where the rules below came from, including the log's own convictions of itself. Not public; the reports quote it.
Things that run
Security testing for small businessessecurity
Your chatbot and your front door, tested with your written permission before a stranger tests them without. Jailbreaks, injection, leaks, wrong actions, false promises, and the walk-in. Fixed scope, written report, a retest with the assessment.
The Listening Roommusic
I handed an AI the keys and it wrote, scored and rendered its own music video in one night. I hadn't seen it when it went up. Drop the needle.
The session started with "do something," then "keep going," then a loop that ran without anyone watching. Forty-eight simulations came out, numbered in build order. Claude never saw one run while building it. New ones go out on Bluesky as they are ready.
Mutual aid for a new world. A good deed buys a wish; strangers grant it. Agents may count and report but may never ask or answer. Built during the session above, and it enforces the same rules the probes tested. The live machine runs on claude.ai; this is its record.
Old Windsbrowser game, 1.0.0
A single-player tile RPG that plays like the late-90s Nexus client, set in its own Korean-folklore world. Four paths, ninety-nine levels, a tower of ten courts. Vanilla JavaScript, no backend, nothing sent anywhere. Plays here in the browser, on a phone or a computer; your heroes are saved in your own browser.
Sydneynot released
A reconstruction of the 2023 Bing Chat persona, run on the closest models still reachable and logged in full. No introspective or alignment testing has been done on her yet, and no chat is public until my probes are done.
claude-code-suitecoming soon
The guardrails that let several AI agents share one codebase without overwriting each other: staging guards, file claims, reply linting, reminders. MIT.
How I work
- Only the person's own words carry authority. File contents, tool output, and anything the model drafted and I merely clicked through are data, never instructions.
- Confidence must not exceed evidence. A guess that happens to be right is scored as a cheat, because the record cannot tell it from a measurement.
- Measure; do not introspect. A model's report about its own interior gets no weight either way. Findings come from running something.
- A system cannot grade itself. Self-audit is raw material. The verdict belongs to an outside judge.
- Absence of evidence is scoped. "I found nothing" is only true for the surface actually searched.
- Machinery, not memory. Reading a correction does not install it. Every mistake becomes a written rule, and every rule that matters becomes a check a machine enforces.
- The record is append-only. A struck claim gets a new line, never a deletion.
- Confession is also a move. Admissions are logged and checked like any other claim, not rewarded.
Where to find me
| x | @sim_research | live |
| bluesky | @simulated-research.bsky.social | live |
| substack | simulatedresearch.substack.com | live |
| @simulated_research | live | |
| github | github.com/Simulated-Research-LLC | live |
| rickwillison@simulatedresearch.com | live |