Agentic jailbreaking: Interactive research explainer
- commitment
- 8 hrs/week
- flexibility
- Open to proposals
The project
AI agents are moving into real workflows faster than they can be made robust against attacker exploits. This project builds an interactive, Distill-style explainer that maps the current landscape of agentic jailbreaking to expose the most crucial vulnerabilities to a broad technical audience.
The fellow will start by understanding how AI agents is actually deployed in practice, then survey the research that exposes weaknesses in these systems. That includes adversarial and optimization based attacks, black box and white box jailbreaking, indirect prompt injection, memory poisoning, and misaligned or reward-hacked behavior. From this survey we would build a clear picture of which techniques work, why they work, and what they reveal about the underlying fragility of current agent architectures. We would end by speculating on what would actually make future systems more robust.
The mentee's role
- Survey the literature on agentic attacks and different methods of jailbreaking.
- Sandbox a small number of methods on modern models
- Design and build the interactive components, using a coding agent.
- Prepare the final submission for a blogpost track or workshop, including any companion write up
Who I'm looking for
- Comfortable reading and distilling research papers quickly, and able to ingest research and write clearly
- Some hands-on experience with LLMs or agentic systems (feel free to share repos/past publications/etc)
- Use a coding agent (Claude Code/Codex/etc) to ship code
- Nice to have: prior exposure to red teaming, adversarial ML, or building interactive or data visualization work
Questions for applicants
- Describe three things (academic or in industry) you are proud of.
- Describe a piece of research you have done in the past. Explain it in a few sentences as you would to a technically aligned person outside the field.
- Tell us about a time you had to learn something technical quickly and from scratch. What was it, and how did you go about it?
- Any additional links to previous works that would be relevant.
Support offered
- Provide feedback on drafts, discuss research.
- Accountability through regular check-ins and clear milestones
- Co-work and contribute to preparing the submission
- Connect to others working in AI safety and security
- format
- Research & writing
- topic
- Near-term impact · Artificial sentience · Other · AI safety & alignment
Shivam Raval
Shivam Raval is a recent PhD graduate in Physics and AI at Harvard with an interest in Interpretability, safety alignment and visualization. He has extensive research experience, ranging from experimental physics, applied machine learning, data visualization to mechanistic interpretability and AI safety. His research has received awards and recognition in conferences such as ICLR and IEEE VIS. In his free time, he likes to make art, discuss philosophy, and read science fiction, and explore nearby areas for cute pets!