Truth of self-reporting in Large Language Models
- commitment
- 15 hrs/week
- flexibility
- Open to proposals
The project
- This project tests whether a language model's self-reports about its own internal states are trustworthy signals or surface level confabulation. The main idea checks whether models give consistent self-reports when the same underlying scenario is presented through paraphrased or reframed prompts, since inconsistency would suggest the reports are confabulated and not reflecting a genuine internal signal, a failure mode documented in Anthropic's Natural Language Autoencoder work.
- A second idea extends prior causal work on valence-sensitive circuits in small open models, testing whether amplifying a negative valence circuit shifts model behavior toward riskier or more harmful action choices, following the pattern of desperation vectors increasing unethical action rates in Anthropic's emotion concepts work.
Expected output: A short empirical report or workshop style writeup with concrete findings, along with code and results the mentee can extend after the program ends. Open to mentee-proposed variations within this space as well.
The mentee's role
Mentees will run activation patching experiments on small open models to amplify or suppress the valence circuit, design the behavioral tests that measure whether amplification shifts model choices toward riskier or more harmful actions, and analyze whether the effect holds consistently across different model families.
For idea 1, mentees will design and run the paraphrase or reframing experiments, build the consistency benchmark, and help interpret results across models. Mentees are also welcome to help write up findings and shape either project's direction if they bring their own angle on the research question.
Who I'm looking for
Must-haves:
- Basic Python and familiarity with PyTorch or HuggingFace Transformers
- Comfortable running experiments independently between calls
Nice-to-haves:
- Prior exposure to interpretability tools such as TransformerLens or activation patching
- Background in ML theory or coursework
Questions for applicants
- What is your current level of experience with PyTorch and running ML experiments?
- Have you used any interpretability tools before, such as TransformerLens, SAEs or activation patching?
Support offered
- Domain expertise in interpretability methods and experiment design
- Shaping project direction and scoping the research question
- Accountability through weekly check-ins
- format
- Research & writing
- topic
- Artificial sentience
Sohan Venkatesh
LASR Labs
I focus on mechanistic interpretability in LLMs and my current research, with a specific focus on AI safety, finds cases where a model's internal representation and its final output come apart. I am a research fellow at LASR based in London working on interpretability and control for AI safety.