Building AI safety infrastructure for all sentient beings
- commitment
- 8 hrs/week
- flexibility
- Open to proposals
The project
Project 1 (Founding & Field-Building)
This is a less structured, apprenticeship-style role for someone who wants to see what it actually takes to run an early-stage org at the intersection of AI safety and animal welfare. You'll work directly with me on whatever Mycelium is doing during the program window (Aug 31 – Nov 9), which will likely include some mix of:
- Fundraising & grants: researching funders, drafting grant applications, refining our pitch and theory of change
- Product & strategy: scoping new technical AI safety work, prioritizing the roadmap, brainstorming our long-term strategy
- Field-building & partnerships: coordinating with partner orgs (Sentient Futures, Arcadia Impact, CaML), community outreach, events
- Operations: comms, hiring pipelines, program logistics, etc.
This track is ideal for someone considering founding their own org, moving into ops/strategy roles in the AI safety or animal advocacy space, or who wants a realistic picture of nonprofit entrepreneurship before committing to it. The exact scope will flex with the org's needs and your interests.
Location: Remote-friendly (though in-person in the Bay Area / NYC is a plus)
Project 2 (Technical Research & Engineering)
Hands-on alignment research within Mycelium's agenda: building the benchmarks, evals, and fine-tuning research for AI systems to consider all sentient beings. We'll scope the specific project together based on your interests and where our research is at program start. Possible directions:
- Emergent alignment: The inverse of emergent misalignment (Betley et al.) — fine-tune an open-source model on a single narrow good value (compassion for animals) and test whether it becomes broadly more aligned toward humans OOD, without capability loss. Builds on Anthropic's "Teaching Claude Why," DeepMind's SDF experiments, and CaML's alignment midtraining for animals.
- Extending MANTA, our multi-turn adversarial benchmark for animal welfare values in frontier models
- New evals: Scoping and prototyping evaluations for underexplored areas - e.g. monitoring LLMs in agentic environments taking actions that affect animal welfare, red-teaming, etc.
Depending on direction, you'll build synthetic data pipelines, run SFT experiments, design eval scenarios and grading rubrics, analyze results across welfare/alignment/capability benchmarks, and contribute to a public write-up. Roles are flexible - from owning major components to executing specific tasks with guidance.
Location: Fully remote-friendly
The mentee's role
Project 1 (founding & field-building):
- Draft a grant application end-to-end - funder research, theory of change, budgeting, etc. - with my feedback at each stage
- Iterating on our Theory of Change and understanding why it's important for an organization
- Help scope and prioritize new technical AI safety work (benchmarks, evals, red-teaming, etc.) and potentially coordinating with researchers
- Coordinate a partnership workstream with orgs like Sentient Futures, CaML, etc.
- Own comms artifacts: social media, website copy, a launch post, a newsletter, outreach sequences, etc.
- Sit in on funder calls, strategy sessions, and hiring conversations (day-to-day of running an early-stage org)
Project 2 (technical research & engineering):
Tasks will differ depending on the specific project you're working on
- Develop or extend benchmarks & evals work from the current AIxAnimals landscape
- Red-team models and generate reports to inform future technical safety work
- Build and run the synthetic data generation pipeline (constitutional SDF + Q&A ethical dilemma datasets), modeled on Anthropic's "Teaching Claude Why" and DeepMind's follow-up experiments
- Fine-tune open-source models (Gemma, Qwen) across data conditions: documents-only, chat-only, combined
- Run evaluation suites across three axes: animal welfare benchmarks, human moral-consideration benchmarks (safety, bias, discrimination), and capability benchmarks (MMLU, HLE) to check for degradation
- Analyze OOD transfer results and contribute to a write-up (blog post, workshop paper, or arXiv preprint)
Who I'm looking for
Please indicate which project (1 or 2) you're applying to in the "Questions for applicants" section. Treat these as guides, not constraints - very open to mentees newer to the space!
Project 1 (founding & field-building):
- Strong written communication - sometimes, a lot of this work is writing (grants, comms, strategy docs)
- Self-directed and comfortable with ambiguity; scope will shift with org needs
- Prior ops / project management / fundraising experience, event or community organizing
- Comfortable networking at conferences, summits, events, etc. in EA, AI safety, or nonhuman welfare
- Familiarity with the EA / AI safety space and nonprofit entrepreneurship
Project 2 (Technical):
- Background or demonstrated interest in ethics, moral philosophy, CS
- Python proficiency, experience with git (version control)
- Experience building with LLM APIs (OpenAI, Claude, etc.)
- Comfortable using AI coding assistants and iterating on generated code
- Research basics: reading technical papers, identifying gaps, synthesizing findings
- Familiarity with evals frameworks (e.g., Inspect), prompt engineering, etc.
Across both:
- Self-starter who makes measurable progress between meetings
- Genuine interest in animal / nonhuman welfare
Questions for applicants
- Which project are you applying to: (1) the founding & field-building role or (2) the technical research & engineering project? You're welcome to apply to both (answer both questions!)
- If Project 1: Tell me about something you've built, organized, or shipped end-to-end - an event, a project, a fundraiser, anything. What did you own, and what would you do differently? (~150 words)
- If Project 2: Review MANTA, our existing eval for nonhuman biases in AI models. What's one limitation or gap in how it handles questions about animals or non-human entities? Propose a specific test scenario or methodology improvement - what would the prompt look like, what would you measure, and how would you know it's working? (~150 words)
Also huge plus if you live in either the Bay Area or NYC and can co-work in-person!
Support offered
- Shaping direction and scoping
- hands-on technical guidance (fine-tuning, evals, LLM pipelines)
- feedback on drafts, code, and writing
- domain expertise across AI safety × animal welfare
- facilitating connections across the AIxAnimals ecosystem (NYU CMEP, DeepMind advisors, Sentient Futures, Electric Sheep, CaML, Arcadia Impact)
- weekly accountability check-ins
- direct co-working exposure to founding, fundraising, and org strategy
- format
- Project scoping · Other · Research & writing · Product & entrepreneurship · Field-building & outreach
- topic
- Near-term impact · Longtermism · Other · AI safety & alignment
Allen Lu
Allen is the Founder and Executive Director of Mycelium, an organization working to make AI go well for all sentient beings through technical AI safety research & engineering. Currently, the org focuses on benchmarking, evals, and other alignment efforts through training data and SFT. Recent work involved developing MANTA, a multi-turn adversarial benchmark for measuring animal welfare values in frontier models.
Concurrently, Allen is working with the NYU Center for Mind, Ethics, and Policy as a Technical Evals Researcher with Jeff Sebo and advisors from Google Deepmind. Previously, he led animal welfare programming at Collider, NYC's coworking hub for AI safety and other high-impact work. He has also facilitated the AGI strategy course for Bluedot, as well as AIxAnimals fellowships for Sentient Futures and Electric Sheep. He was also a mentor in a previous iteration of SPAR. Before transitioning into AI safety, he worked as a software engineer in creative technology.