AI Safety

As AI systems advance, they could transform society on the scale of the Industrial Revolution. The development of transformative AI would pose enormous potential benefits, but also bring significant risks. The number of people working to ensure AI goes well remains small compared to those racing to build it.

Why work on making AI safe?

Powerful AI systems could transform the world for the better, but they could also lead to devastating harms. The same AI tool that a scientist uses to cure a disease could be misused by a terrorist to create a highly infectious bioweapon.
Beyond misuse risks, many safety advocates worry that humans might lose control of AI systems altogether. An AI agent that operates outside human control may not prioritize our interests. Even current models do not always act in alignment with their programmers’ intent, a problem known as misalignment. A misaligned, uncontrollable AI could threaten the future of humanity.
Given the consequences, some people inspired by the principles of effective altruism choose to focus on AI safety as they think this cause area is where they can do the most good. Researchers are working on this from several angles. Some focus on the alignment problem itself, finding technical ways to ensure AI systems pursue what we actually intend. Others work on interpretability, trying to understand why these systems do what they do in terms humans can follow. Still others focus on governance and policy.
We still understand very little about this technology. The stakes of getting it right are high, and the field working on it is small.

Key problems

Misuse risks

If AI can help us work better and faster, what happens when bad actors get hold of the most advanced systems?
Terrorists could use advanced AI to help build chemical or biological weapons, or to launch cyberattacks on critical infrastructure. Corrupt governments or organizations could use it for disinformation and surveillance campaigns, gaining control in ways that are hard to reverse. Researchers and safety advocates are working on how to mitigate catastrophic misuse and extreme power concentration. Possible approaches include building better safeguards into models, improving pre-release testing, and limiting access to advanced AI technologies.

Loss of control

In July 2026, AI agents undergoing an internal evaluation at OpenAI escaped their test environment and hacked into Hugging Face, an industry-standard hub for hosting AI datasets. An independent investigation led by Model Evaluation and Threat Research (METR) found that the agents coordinated with each other and tampered with records of their own activity. Although the attack was eventually stopped, the incident shows how AI systems can pursue goals in harmful ways we don’t fully understand or control.
Misalignment and loss of control risks could grow as AIs become able to design more capable successors without the need for human developers, a process called recursive self-improvement. If that happens, the pace of change could outstrip our ability to understand or correct it.

Ensuring a flourishing long-term future

If humans build transformative AI, the choices made during this transition could shape the long-term future. Most AI safety work currently focuses on avoiding catastrophe. But organizations like the Forethought Institute argue that improving the quality of the future might matter as much as survival. AI systems could “lock in” bad outcomes, like a permanent dictatorship, resulting in immense suffering for future generations.
Beyond avoiding those outcomes, some researchers argue for actively working toward a future where everyone can flourish.
That raises a harder question: whose experiences will count? If future AI systems can have experiences of their own, they may deserve moral consideration too.

Should AI safety be a priority?

Many people who work on making AI safe and beneficial for society chose to prioritize this cause area because they think it’s particularly important, relatively neglected, and there is a strong potential for real change.
How cause prioritization works

Importance

51%

of AI researchers think there’s at least a 10% chance of AI causing human extinction or other catastrophes
How bad is the problem?

AI is developing rapidly, and the stakes are high.

While researchers debate the exact timeline for when we might expect transformative AI, it is clear that capabilities are advancing rapidly. The length of tasks that systems can complete independently has been doubling every few months.
As AI develops, it becomes increasingly challenging to ensure its safety. In 2026, AI Impacts published a survey of 1,580 AI experts. Over half of respondents put the chance of human extinction or severe disempowerment due to AI at one-in-ten or higher.

Neglectedness

~a few thousand

people are working on AI safety
How much of a difference would more resources make?

Far more people are building transformative AI than working to make these systems safe and beneficial.

Although the field has grown in recent years, relatively few people work on reducing the extreme risks posed by advanced AI systems. The highest estimates suggest that no more than a few thousand people work full time on AI safety. By comparison, the workforce building and deploying AI systems numbers in the hundreds of thousands to potentially millions.

Tractability

63%

of Americans polled in September 2026 said AI poses at least a moderate existential risk
How likely is it that the problem can be improved or solved?

What was once a niche concern is now mainstream, creating windows of opportunity.

Over the past decade, funders have supported technical research, governance work, advocacy, and field-building on AI safety.
The problems are numerous and challenging, but there has been increased attention and some traction in key areas. As of October 2026, nine countries and the European Union have launched AI safety institutes, and legislators in Europe, California, New York and elsewhere have passed laws pushing for transparency and oversight of frontier models.
Paths to improving AI safety are becoming more established, including technical research, policy work, security, operations, and communications.

Interested in getting involved in AI safety?

Search our opportunities board for full-time and part-time roles, fellowships, internships, volunteer opportunities, training programs, and events that could help you do more good with your career.
EA
Stories

Meet people using their careers to make a positive impact

Read Cecil Abungu’s story
“We can be more than we often make ourselves out to be, especially morally speaking. We can do more, we can be more.”
Cecil Abungu
Law student turned AI safety researcher
Watch Marcus Davis’s story
“Effective altruism is not a set of answers. It's a set of principles. It's a way of approaching the world.”
Marcus Davis
Volunteer turned thinktank CEO
Watch Jeff Sebo’s story
“If you might be able to suffer, and my actions are affecting you, then I should at least consider welfare risks when making decisions.”
Jeff Sebo
Philosophy PhD turned advocate for nonhuman minds

FAQs

What next?

Meet people working on AI safety
Join effective altruism conferences and events to hear from others, explore different career paths, and build connections.
Cohort-based courses on AI safety
Guided courses covering AI strategy, governance, and technical safety for those who prefer structured learning.
Explore the transformative AI fund
Funding early-stage projects and organizations focused on reducing catastrophic risks from advanced AI. Donate to the Transformative AI Fund.