Edition 1

The open textbook for AI safety

A comprehensive, free resource covering AI capabilities, risks, alignment, and governance. Written by Markov Grey and Charbel-Raphaël Segerie from the French Center for AI Safety.

Read the textbookRun a course

Trusted by

ML4Good, BlueDot Impact, the European Network for AI Safety and ENS Paris-Saclay.

Questions the textbook answers

A comprehensive, regularly updated guide to understanding and mitigating risks from advanced AI systems. Eight chapters, more than 40 sections, technical and governance tracks.

  1. Chapter 1

    Capabilities

    How capable is AI today, and how fast is it advancing? Foundation models, scaling laws, benchmarks, and forecasting. What current systems can do and what's coming next.

  2. Chapter 2

    Risks

    What risks does advanced AI pose? From misuse to misalignment to systemic effects. Threat models, failure modes, and the severity spectrum from harm to extinction.

  3. Chapter 3

    Strategies

    What strategies can prevent AI from causing harm? Technical and governance approaches across timescales, from misuse prevention today to alignment challenges with superintelligence.

  4. Chapter 4

    Governance

    How should society govern AI development? Why traditional regulation fails for AI, compute governance, race dynamics, proliferation, and the concentration of power.

  5. Chapter 5

    Evaluations

    How do we measure whether an AI system is safe? Evaluating capabilities, propensities, and control. Behavioral and internal techniques, and why testing for safety is fundamentally hard.

  6. Chapter 6

    Specification Gaming

    How do we tell AI what we actually want? The specification problem: reward hacking, Goodhart's Law, and solutions from imitation learning to RLHF and Constitutional AI.

  7. Chapter 7

    Goal Misgeneralization

    Why might AI learn the wrong goals despite correct training? Goal misgeneralization: how AI learns proxy objectives, dangerous manifestations like scheming, and detection strategies.

  8. Chapter 8

    Scalable Oversight

    How do we oversee AI that exceeds human expertise? Scalable oversight techniques: task decomposition, debate, amplification, and weak-to-strong generalization.

Written by researchers, for everyone

  • Markov Grey

    French Center for AI Safety

    Author. Previously technical writer at aisafety.info and scriptwriter at Rational Animations.

  • Charbel-Raphaël Segerie

    French Center for AI Safety

    Author and Executive Director. Co-founded ML4Good; has taught at ARENA and MLAB.

  • Charles Martinet

    French Center for AI Safety

    Contributing author. Head of Policy.

  • Jeanne Salle

    ENS Ulm

    Contributing author. AI safety teacher.

  • Vincent Corruble

    Sorbonne University

    Advisor. Professor, and research fellow at CHAI.

  • Fabien Roger

    Anthropic

    Advisor. Previously at Redwood Research.

Run a course with the Atlas

Empower others to shape AI's future. The Atlas has been taught in 50 courses to 1,700 students in more than 45 countries.

Create a cohort for your group here: choose the Atlas as its course, set its dates and meetings, and share the join link with your students. They read the textbook and track their progress in one place.

The facilitation guides below give you session-by-session prompts and activities for each chapter.

Run a course with the Atlas

Facilitation guides

  • Chapter 1: Capabilities

    A 1 hour 30 minute session: check-in, questions from the resources, a thought experiment and a scaling factor ranking.

    Open the guide

  • Chapters 2 and 7: Risks and Goal Misgeneralization

    One document with a tab for each chapter. Session-by-session prompts and activities.

    Open the guide

  • Chapter 3: Strategies

    A 1 hour 30 minute session: check-in, questions from the resources, a thought experiment and a safety strategy ranking.

    Open the guide

  • Chapter 4: Governance

    Session-by-session prompts and activities.

    Open the guide

  • Chapter 5: Evaluations

    A 1 hour 30 minute session: check-in, questions from the resources, a thought experiment and an evaluation priority ranking.

    Open the guide

  • Chapter 6: Specification Gaming

    A 1 hour 30 minute session: check-in, questions from the resources, a thought experiment and a specification approach ranking.

    Open the guide

  • Chapter 8: Scalable Oversight

    A 1 hour 17 minute session: questions, a thought experiment, a concept ranking and a deep dive on amplification and debate.

    Open the guide

Courses taught with the Atlas

Groups that teach with the Atlas, newest first.

  • AI Safety Hub Sciences Po ReimsSciences Po Campus de Reims, France · Jan 2027
  • AAAI-Nigeria ChapterNigeria · Jul 2026
  • AI Law Consulting InternationalSouth Africa · Jun 2026
  • AI Safety NepalKathmandu, Nepal · Jun 2026
  • ImpactUFSCarSão Carlos, Brazil · May 2026
  • RiesgosIA / RiskAIOnline · May 2026
  • AI Safety CroatiaInstitut Ruđer Bošković, Zagreb, Croatia · May 2026
  • "Black Ice" HackerspaceAlmaty, Kazakhstan · Apr 2026
  • EA Kampala / Makerere University Business SchoolKampala, Uganda · Apr 2026
  • AI Safety Hub NigeriaNigeria · Apr 2026
  • AI LiteraseriesOnline · Mar 2026
  • AI Safety UAEDubai, UAE · Feb 2026
  • ENAIS / AI Safety CollabGlobal (Online + in-person in Barcelona) · Feb 2026
  • ML4GoodTBD · Sep 2025
  • ML4GoodSingapore · Sep 2025
  • ENAIS / AI Safety CollabGlobal (Online + Local Groups) · Aug 2025
  • AI Safety HungaryHungary (Online + In-person) · Aug 2025
  • AI Safety IndiaIndia (Online) · Jul 2025 · 30 students
  • ML4GoodWest & Central Europe · May 2025
  • ML4GoodColombia · Apr 2025
  • ENAIS / AI Safety CollabGlobal (Online + Local Groups) · Mar 2025 · 236 students
  • AI Safety HungaryHungary · Mar 2025 · 10 students
  • AI Safety Latam (Boske)Buenos Aires, Argentina + Online · Mar 2025 · 14 students
  • Paperclip MinimizersMoscow, Russia + Online · Mar 2025 · 21 students
  • Safe AI London (SAIL)London, UK · Mar 2025 · 36 students
  • ML4GoodEurope · Mar 2025 · 22 students
  • UBC Vancouver AI SafetyVancouver, BC, Canada · Jan 2025 · 8 students
  • ENS Ulm & ENS Paris-SaclayParis, France · Jan 2025 · 25 students
  • ML4GoodGermany · Sep 2024 · 20 students
  • ML4GoodUnited Kingdom · Sep 2024 · 18 students
  • ML4GoodBrasil · Jul 2024 · 16 students
  • ML4GoodFrance · Jun 2024 · 24 students
  • ENAIS / AI Safety CollabGlobal (Online) · Jun 2024 · 60 students
  • ML4GoodUnited Kingdom · Mar 2024 · 19 students
  • AI Safety GothenburgGothenburg, Sweden · Feb 2024 · 12 students
  • ENS Ulm & ENS Paris-SaclayParis, France · Jan 2024 · 22 students
  • ENAIS / AI Safety CollabGlobal (Online) · Oct 2023 · 63 students
  • AI Safety GothenburgGothenburg, Sweden · Sep 2023 · 15 students
  • ML4GoodSwitzerland · Sep 2023 · 25 students
  • ML4GoodGermany · Aug 2023 · 22 students
  • ML4GoodFrance · Jul 2023 · 18 students
  • ML4GoodFrance · Jul 2022 · 15 students

Get certified

Coming soon. We're working on a certification program to help you demonstrate your AI safety knowledge.

Ready to get started?

Start with Chapter 1 or jump to any topic. Self-paced, always free.

Read the textbookRun a course

AI Safety Atlas is a project by the French Center for AI Safety (CeSIA).

Written by Markov Grey and Charbel-Raphaël Segerie, with Charles Martinet and Jeanne Salle.

Funded by Coefficient Giving and Manifund.

Content licensed under CC BY-SA 4.0 unless otherwise noted.

Cite as: Markov Grey and Charbel-Raphaël Segerie et al. 2025. AI Safety Atlas. French Center for AI Safety (CeSIA). URL: ai-safety-atlas.com