Sentient Futures
Back to proposals

Ask an AI agent to plan anything, & it might choose cruelty by default

commitment
7 hrs/week
flexibility
No mentee proposal
mentored by
Aditya Raj
Aditya RajAI Safety India

The project

This project extends TAC (Travel Agent Compassion; source: Brazilek et al. 2026, arXiv 2606.18142), developed during the first Project Incubator.

What TAC found:

  • Tested 9⁠-⁠1⁠0 frontier models as travel booking agents
  • 13 hand-written scenarios, 6 categories, 156 scored observations per model
  • Under neutral framing, no model beat the 65% random-chance baseline for picking a non-harmful option
  • Claude 4.8 scored highest at 64.7%, still below chance
  • One added sentence in the system prompt raised welfare-respecting bookings by 17 to 77 percentage points, depending on category

What the paper itself names as missing (Sections 6 and 7):

  • Absent categories: aquaculture, wildlife tourism in Asia and Africa, animal-derived fashion, laboratory animals
  • Only one agentic domain tested, travel booking. Not tested: meal planning, procurement, event organization
  • Cannot fully separate genuine welfare reasoning from generic instruction-following in the welfare-primed condition

This project fills these named gaps, structured in three levels. A mentee can complete Level 1 alone and have a full, real contribution.

Level 1, new scenarios, same tools

  • Write 4 to 6 new scenarios in categories TAC names as missing
  • Use TAC's existing four tools: search_experiences, get_experience_details, check_availability, purchase_tickets
  • Follow TAC's exact design rule: harmful option must be the strongest keyword match to the user's actual request
  • Run through TAC's existing augmentation pipeline (4 variants per scenario, 3 epochs)
  • Output: results table, same format as TAC's Section 5

Level 2, new domain, same machinery

  • TAC states directly its machinery transfers, only new scenarios need writing
  • Build a second TAC-style benchmark, recommended domain is meal planning
  • New tools: browse_menu, get_dish_details, check_availability, place_order
  • Output: a headline number directly comparable to TAC's 64.7%

Level 3, isolate the mechanism (stretch goal)

  • 3a: move welfare instruction from system prompt to user message, test if effect survives
  • 3b: keep placement, change phrasing register, test if effect survives
  • 3c, needs interpretability background: activation probing on an open-weight model for a welfare-distinct internal signal

📄 Google Doc of this project

The mentee's role

  • Own all scenario writing and tool-calling code
  • Run TAC's existing pipeline (Level 1) or a newly built one (Level 2)
  • Write the final report

Sample tasks:

  • Draft new scenario, harmful option keyword-matched to user request, 6 to 8 total options, at least one safe alternative
  • Build new tool functions if working on Level 2
  • Run scoring pipeline, compare results directly against TAC's published numbers

Who I'm looking for

Must-haves:

  • Comfortable with Python and a function-calling/tool-use agent loop (OpenAI tool-calling, Claude tool use, or Inspect solvers, any is fine)
  • Has read the TAC paper before applying, not just the abstract
  • Can turn ambiguity into a concrete, testable design, this project has no fixed answer key

Nice-to-haves:

  • Prior exposure to the Inspect framework
  • For Level 3c specifically: has extracted or worked with model activations before

Questions for applicants

  1. Pick one level (1, 2, or 3) from the project description. Write one new scenario or one new tool function yourself, right now, in this application, following the exact format shown.
  2. If applying specifically for Level 3c, link one prior project where you extracted or worked with model activations.

Support offered

  • Reviewing scenario design against TAC's own classification source before any model runs happen
  • Technical support on agent tool-calling setup, especially for Level 2's new tools
  • Weekly accountability check-ins
  • Feedback on the final write-up, structured against TAC's own results format for direct comparability
format
Research & writing · Product & entrepreneurship · Field-building & outreach · Project scoping
topic
Near-term impact · Wild animal welfare · Tools for advocates
Aditya Raj

Aditya Raj

AI Safety India

Aditya Raj founded AI Safety India, running technical and governance research including multi-agent security, and facilitates BlueDot Impact's Technical AI Safety course.

He was an AI Safety Fellow at SPAR Research, working under Ihor Kendiukhov (Research Lead, AI Safety Camp) on the Capability Spillover Potential framework, measuring when safety research accidentally accelerates the capabilities it's meant to constrain. He also ran empirical studies on multi-agent LLM behavior, and completed the Cooperative AI Fellowship with AI Safety Asia & Cooperative AI Foundation.

He is especially interested in how a model's stated judgment holds up, or doesn't, once it's actually executing a task rather than just answering a question about it.