LLM-driven causal modeling of AI loss-of-control risk

commitment: 10 hrs/week
flexibility: Open to proposals

mentored by

The project

We have built a library of loss-of-control scenarios extracted from the AI safety literature. This project converts that library into causal loop diagrams by extracting decision-relevant variables and the relationships between them, using an LLM pipeline grounded in established literature-extraction methods (Rigorously Interpreted Quotation Analysis, Extract-Define-Canonicalize, and structured concept analysis). We then evaluate whether SaferAI's quantitative risk estimation methods can be applied on top of these diagrams to produce numeric risk estimates rather than qualitative scenario descriptions. The novel contribution is an automated pipeline that converts natural language stakeholder input directly into a structured causal risk model, removing the manual translation step that currently limits how fast new scenarios can be incorporated into formal risk analysis.

The mentee's role

Mentees will build the technical pipeline end to end. This includes implementing the literature-extraction methods to pull decision-relevant variables and relationships from the scenario library, building the LLM pipeline that converts these into causal loop diagrams, and testing the SaferAI quantitative risk estimation methods on top of the resulting diagrams. I will support each step closely

Who I'm looking for

Must-haves for mentees:

Strong Python coding skills, comfortable building and debugging multi-step pipelines
Background in quantitative modeling, ideally with exposure to causal inference or systems modeling
Hands-on experience with open-weight LLMs (Hugging Face, local inference, or API-based) and basic prompt engineering
Comfort reading and extracting structured information from dense academic literature
Strong willingness to learn and adapt to ambiguous, partially-specified research problems
Ability to honor the time commitment stated on the platform. This project targets a conference submission and requires concentrated, consistent involvement during active phases

Nice-to-haves (not required):

Familiarity with RAG or structured extraction pipelines
Exposure to graph or network-based modeling (for the causal loop diagram structure)
Background in AI safety or risk analysis literature

Questions for applicants

Describe a project where you built a pipeline using an open-weight LLM (Hugging Face, local inference, or similar). What made the pipeline reproducible and rigorous, not just functional?
Briefly describe how you would approach processing a loss-of-control scenario from an academic paper to extract its decision-important variables. Walk through your reasoning, not just the final method.

Support offered

I bring a background in complex systems modeling and social computational science, which I use to help shape ambiguous research questions into testable structures and to provide direction for developing the technical methodology.
On execution, I support drafting, from early methodology sections through conference submission.
On domain expertise, my background spans LLM evaluation, causal inference, and decision-making under deep uncertainty, directly relevant to converting literature into structured risk models.
I provide regular accountability check-ins to keep technical work on pace toward the conference deadline, and I am ready to facilitate connections to relevant researchers and literature where useful.
This will ultimately be a student-led process, with my strong commitment to supporting the project throughout.

format: Project scoping · Product & entrepreneurship · Research & writing
topic: Other · Tools for advocates · Post-AGI transition · Longtermism

Swaptik Chowdhury

Dun and Bradstreet

I am a researcher specializing in quantitative risk estimation, LLM evaluation methodology, and decision-making under deep uncertainty. My work centers on a simple question: which unsafe futures become more likely as AI capabilities scale, not just what a model can do today.
At RAND Corporation, I led human baselining for a study comparing five LLM grading methods across four expert benchmarks and five frontier LLMs, and built adversarial-prompting safeguards against misuse of LLM-enabled bioinformatics tools. At Dun & Bradstreet, I translate this research into production, building LLM-as-judge audit frameworks and agentic deliberation systems over enterprise data.
I mentor researchers working at the intersection of rigorous measurement and real-world AI risk.