The AI Safety
Talent Pipeline

A Few Notes on How to Read This

I originally created this document as a “personal checklist” of fieldbuilding tasks that seem good to complete, and I created the associated website because I thought it would be an efficient way to store that information. I am publishing this because I've learned a lot about fieldbuilding needs along the way, and I thought some people would appreciate seeing my (albeit imperfect) top-down perspective. So, here are some things to consider as you read through this:

• I am uncertain about the magnitude and tractability of many of these problems, and with a few exceptions, I am intentionally abstaining from making claims about how important or tractable they are. I am also uncertain how much of a problem many of the “leaks” are: some are generally agreed upon as important by many people in AI safety, while others are less-evidenced speculation by people I know who have completed or run programs.
• The “pipeline” visualization is imperfect. Org locations in here are often imperfect approximations, so I don't think this should be taken 100% literally. Additionally, some orgs have been forgotten or excluded. For more detailed info, you can look at the Field Map on aisafety.com.
• The number of dots moving from section to section is just a rough representation of when and how people get lost. I don't have real data on the true ratios, so please don't read the flow as a precise measurement of anything.

General Concerns About This Whole Framing

• While the pipeline makes for a nice visual metaphor, I don't think it's the most accurate or good way to think about people. We care about impact, but at the end of the day, people aren't just dots that we push through our funny little impact machine; they're human beings that deserve respect. They deserve to be invested in and cared about, not just pushed through some pipes.
• The world is complex and messy! In general, I don't think we have super-efficient markets in the charity space, even with Effective Altruists trying their best, because a lot of people thinking about these leaks are too busy with other projects to fix them. However, sometimes there's a good reason why something isn't happening, even if it's not immediately obvious. Most people trying to address leaks should check for these before investing themselves too heavily in a project.
• The pipeline framing might seem weird and cult-y. I'm uncertain what should be done about this, but I suspect the best thing for us to do is be as transparent as possible: we're trying to get more people to work on these issues because we're seriously concerned about the possibility, even a small possibility, that AI could be very destructive. We think that by getting more people to work on these issues, we can make the future better, which is why we're making it as easy as possible to motivate people to work on these problems.
• Alternatively, I don't think it's much weirder than the “high school → college → internship → job” pipeline that the rest of the world interacts with, so maybe it's not that crazy. The difference is that in AI safety, you probably have to care about what you're doing, while many people in other jobs don't.
• One of my favorite writings on having an impactful career is a piece called “Your Goal Isn't Really to Get a Job”, by Matt Beard. To articulate this pipeline clearly, it's helpful for me to put everything in terms of jobs, but at the end of the day, we care about impact. I included the theory of change for each role people make it to because that's what we should be thinking about. Still, I slightly worry that the whole “career pipeline” framing will push some people towards thinking that the goal is just to land a solid job.

Thank You's

Many of the ideas listed are not fully my own. Many of them come from random discussions with other people in AI safety or other EA Forum posts. It's hard for me to rigorously give credit to everyone, but here's a list of people who deserve thanking: Manon Kempermann, Weronika Żurek, Sam Smith, Spencer Kitts, Sophie Kim, Jacob Brinton, Afitab Iyigun, Eliana Du, Sam Anschell, Neav Topaz, Aaron Gertler, Harry Waterman, Seth Lifland, Alexandra Bates, Helena Tran, jteichma, Agustin Covarrubias, Aris Richardson, Aryan Bhatt, Clarissa Lam, SvA, and probably many others.

This post was written during my time in the Generator Residency, a program dedicated to helping people learn generalist skills through real-world experience. Most of what I learned to write this post came from trying to choose my Generator project.