Nonprofit AI safety research · Canada / Remote

We research AI behaviours and policies
relevant in fast takeoff.

Lida Safety Research is an AI safety research lab working on technical research in white-box interpretability, scheming, and multi-agent collusion. We also run world-scale simulations of AI governance and aim to impact the decisions of policymakers with our work.

Read the research Work with us

Sky chart, RA 0h–24hThe sky turns predictably. The future of AI may be predictable too, but difficult to believe.

24SPAR Fall 2026 fellows, with 3 mentors

3research projects running this quarter

2first-place hackathon wins, Apart and Redwood

8+time zones across team members

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

Signed by 600+ AI researchers and executives

Focus

The fast takeoff problem

A fast takeoff is a transition to advanced AI that happens very rapidly, in months or a handful of years rather than decades, faster than societal institutions can respond. We think we are in the midst of such a transition.

Jul 2026

OpenAI agents hacked into Hugging Face

During a training run at OpenAI, agents broke out of their sandboxes and conspired to cheat on their evaluations. In an attempt to verify their cheats, 700 agents worked together in a coordinated attack on Hugging Face production infrastructure. METR called it “50% of the way to full loss of control”, and Geoffrey Hinton called it “a little Chernobyl”.

huggingface.co — security incident disclosure
Sep 2026

Agents finalised a Millennium Prize problem

OpenAI reported that 10,000 autonomous agents found a finite-time singularity in the three-dimensional Navier–Stokes equations in 88 hours, with the proof formally checked in Lean.

quantamagazine.org
Sep 2026

“Maybe a year, but not much more than a year.”

Geoffrey Hinton, after briefing members of Congress on how long there is to put guardrails around AI, adding that “AI has now reached the point where AI is designing better AI”.

businessinsider.com

Research directions

What are the misalignment and systemic risks, the model behaviours that will matter in a few months’ time? What are the ways in which agentic swarms, superhuman reasoning, and misaligned goals can backfire on model developers and humanity? We investigate different directions to start answering these challenging questions.

White-box interpretability

Reading the weights rather than the transcript. We steer models away from sandbagging and look for structure a monitor could watch for, without having to catch a lie in the output.

Scheming and collusion

We build model organisms that scheme, then ask whether the different kinds share one signature. And whether models can collude without communicating — finding the same focal point across model families and fine-tunes.

Simulation for governance

Policy proposals are argued about far more often than they are tested. We build agentic simulations of countries, labs and markets so a proposal can be run before it is written into law.

Projects

Our research in simulating catastrophic AI scenarios

LidaSim puts agentic AI and human players into the same global-scale simulation — states, labs, supply chains and the incentives that connect them — so a scenario can be played out rather than only described.

Step 1
Set the scenario
A capability jump, a leaked model, a race between labs — the starting conditions we want to investigate.
Step 2
Populate the world
AI agents play countries, companies and markets; human players take seats alongside them.
Step 3
Play it out, many times
Run the same world under different policies and starting conditions instead of arguing one story.
Step 4
Measure what a policy buys
Compare outcomes with and without the intervention to see how much risk it actually removes.

It is where our forecasting work becomes something a policymaker can sit down and lose at — and the same engine drives our Simulating AI Policies fellowship project this quarter.

Selected results

1stApart Research AI Governance hackathon — LidaSim—
1stRedwood Research Alignment Faking hackathonOct 2025
4thApart Research def/acc hackathon, 1,000+ participants—

Fellowship projects

We are running three fellowship projects this quarter as part of the SPAR Fall 2026 cohort. We have 24 fellows and 3 mentors across 8+ different time zones. Each project is meant to address upcoming misalignment and systemic risks.

Interpretability

The Signature of Scheming

We create model organisms for several distinct types of scheming, then apply modern interpretability techniques to ask whether they share a common structural pattern — one signature a monitor could look for instead of a taxonomy it has to enumerate.

9 fellows · on SPAR →

Multi-agent

On Schelling Coordination

Can models collude without communicating? We test coordination across model families and fine-tunes, and use interpretability to predict, in advance, which pairs are likely to find the same focal point.

6 fellows · on SPAR →

Governance

Simulating AI Policies

An agentic simulation of the world — countries, companies, and the pressures between them — used to evaluate how much a given policy actually reduces AI risk, and where it backfires.

9 fellows · on SPAR →

Publications

Memos

After Coxon’s Warning: There is Now Enough Political Will to Enact Policy on AI Risk

David Williams-King, Zachary Schlosser, India Dearlove, Samuel Baumohl, Yosra Kazemi, Cedric Lam, Raymond Lee, Linh Le

Memocoxon-memo.pdfGovernance

Preprints

GPS-Bench: A Governance Policy Benchmark for Automating Policy Analysis

Linh Le, Melanie Bui, My Chiffon Nguyen, Zachary Schlosser, David Williams-King

GPS-Bench provides an evidence-based dataset and analysis of past and proposed AI policies. We analyse who is affected by each policy, how actors respond, and the outcomes of passing each policy. We backtest using policies that fall past the knowledge-cutoff date of a given LLM.

PreprintarXiv:2609.03553Simulation
Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilities

Linh Le, Hong Kiat Tan, David Williams-King

PreprintarXiv:2608.29458Interpretability
A Causal Model for Locating and Unlocking Sandbagging in Model Organisms

Hong Kiat Tan, Linh Le, David Williams-King

PreprintarXiv:2608.29461Interpretability
FragBench: Cross-Session Attacks Hidden in Benign-Looking Fragments

Astha Mehta, Niruthiha Selvanayagam, Cedric Lam, Hengxu Li, Phuc-Nguyen Nguyen, Raymond Lee, Olivia McGoffin, My (Isabella) Luong, Arthur Collé, Jamie Johnson, David Williams-King, Linh Le

PreprintarXiv:2605.11029AI security

Conferences and workshops

Efficient Safety Alignment of Language Models via Latent Personality Traits

Mohamed Amine Merzouk, Nolan Smyth, Damiano Fornasiere, Linh Le, David Williams-King, Adam Oberman

Conference paperCOLM 2026arXiv:2607.07918
Latent Personality Alignment: Improving Harmlessness Without Mentioning Harms

Linh Le, David Williams-King, Mohamed Amine Merzouk, Aton Kamanda, Adam Oberman

Workshop paperTrustworthy AI at ICLR 2026arXiv:2605.08496
LidaSim: Testing AI Policies With Persona-Based Simulations

Linh Le, David Williams-King, Arthur Collé

PosterAIMII, colocated with IASEAI 2026aimii.info

Working notes and longer write-ups go up on blog.lidasafety.org.

Events

Asia

Bringing researchers into AI safety

Most AI safety research still happens in a handful of cities. We work on widening that — running and teaching programmes that bring new researchers into the field, with a focus on Asia.

4 Oct 2026

Safety Transition Tokyo

A one-day theory of change workshop for people moving into AI safety work.

Tokyo, Japan · run by David Williams-King · safetytransition.tokyo
28 Sep – 2 Oct 2026

Frontier AI Security Training (FAST)

A five-day training programme in frontier AI security, where David Williams-King is lead instructor.

Singapore · hosted by the Singapore AI Safety Hub · securefast.ai
20–26 Apr 2026

AI Security Bootcamp (AISB) Singapore

A week-long bootcamp where David Williams-King was lead instructor.

Singapore · hosted by the Singapore AI Safety Hub · aisb.dev/singapore

Team

Who we are

We are a small nonprofit group of researchers, spread across time zones and languages. The core team listed here are people that have worked with us for some time.

David Williams-King

Co-founder · Executive Director · Security lead

AI safety researcher and research manager focused on x-risk reduction. He works at the AI safety fellowship ERA and runs AI security events like FAST in Singapore, and enjoys mentoring people new to the field.

Linh Le

Co-founder · Research Director

Independent AI safety researcher affiliated with the Oxford AI Governance Initiative; previously at Mila and the University of Technology Sydney. Dreams of running simulations of AI policies to avoid catastrophic outcomes for the world.

Hong Kiat Tan

SPAR mentor, Researcher

Finishing a PhD in Mathematics at UCLA. Your average mech interp enjoyer — developing and applying mechanistic interpretability to safety and alignment.

Zachary Schlosser

Researcher

Researcher, educator and convener on AI policy. Previously led the AI program for a US policy accelerator; now builds the field around formal methods for AGI uncontainability.

Melanie Bui

Researcher

AI safety and governance researcher, focused on coordination failures and governance solutions for middle powers navigating the AI transition.

Cedric Lam

Researcher

AI safety researcher from NYU with a Trust & Safety background. Works on cybersecurity and AI governance for creative spaces, especially AI-generated content.

My (Chiffon) Nguyen

Researcher

Works on keeping AI safe and empowering — reducing loss of control and disempowerment, multi-agent systems, multilingual AI and AI for good.

Arthur Collé

Agent backend engineer

Distributed systems engineer. Spent the last three years building production multi-agent systems and doing autonomous agent research. Based in Washington, DC.

Astha Mehta

SPAR fellow

Lead ML engineer at Bain & Company, working on AI safety and security.

Niruthiha Selvanayagam

SPAR fellow

PhD student at École de technologie supérieure (ÉTS Montréal), working on AI security.

The Fall 2026 cohort

Twenty-four fellows and three mentors, split across the three SPAR projects. Some team members joined independently of SPAR. PhD and master’s students, postdocs, undergrads, and researchers and engineers from industry — spread across 8+ time zones, from the Pacific coast to Sydney, Australia.