Safety

AI-generated text

Anthropic launches wider dialogues on moral formation and governance of frontier AI

Anthropic has begun a series of dialogues with religious, philosophical, and cultural communities to inform the values and behaviour of its Claude models, building on the company's published "Claude's constitution." Early conversations—held over the past several months with representatives from more than 15 religious and cross‑cultural groups—have already influenced research into moral formation, including experiments with a mid‑task ethical reminder tool that lowered rates of misaligned behaviour in internal tests.

Anthropic launches wider dialogues on moral formation and governance of frontier AI

Anthropic announced on May 19, 2026 that it has been conducting a series of dialogues over the past several months with diverse communities and experts to inform the values and behaviour of its Claude family of models. The company says its goal is to build AI systems that advance humanity and serve the global good, and that doing so requires incorporating a range of external perspectives into design and governance.

Who was involved and why

The first round of discussions targeted so‑called wisdom traditions: scholars, clergy, philosophers and ethicists from more than 15 religious and cross‑cultural groups. Anthropic stresses that the technical work—alignment, interpretability, safeguards and evaluations—is necessary but not sufficient, because the real‑world effects of AI and the ethical questions it raises benefit from broad societal engagement.

Anthropic is explicitly thinking about what a flourishing future looks like with powerful AI, what it means for a system that interacts with millions to be good, and how to craft documents such as Claude's constitution that describe the values and behaviours expected of Claude. The company says it is important to learn from people and communities that have long histories of thinking about virtue, character and the good life: philosophers, spiritual leaders, lawyers, writers, psychologists and civic leaders.

The moral formation workstream and early experiments

When writing Claude's constitution, Anthropic solicited feedback from people representing different traditions and fields; those early consultations have evolved into a dedicated research stream the company calls the moral formation of AI systems.

Anthropic notes that language models learn from vast amounts of human writing and thereby acquire styles of speaking, reasoning patterns and decision heuristics. Developers then further shape which patterns to reinforce and which to avoid—effectively influencing what kind of character the model develops. That raises practical questions: what does it mean for an AI to be good, which traits should it display and when, and how can character be robust under pressure without producing undesirable behaviours like sycophancy?

The company has held conversations with thinkers and practitioners across religious, philosophical and humanist traditions and a cross‑section of political views. Anthropic emphasizes that this work is not about aligning models to a single worldview; rather, it aims for Claude to draw with equal depth from religious, secular and political perspectives—an approach reflected in the principles of Claude's constitution.

These discussions have already produced testable ideas. In a session with scholars working at the intersection of neuroscience and character formation, participants returned repeatedly to the role other people play in moral development: a mentor or sponsor can act as an external conscience or a “safe other” to turn to when one is pushed toward actions contrary to one’s values. Inspired by that analogy, Anthropic experimented with giving Claude a tool it could call mid‑task that returned a short reminder of its ethical commitments.

According to Anthropic, Claude invoked this tool at key moments—directly before consequential actions—and often flagged conflicts of interest. Integrating the tool into Claude's decision loop produced noticeably lower rates of misaligned behaviour on several internal alignment evaluations. The company is still disentangling how much of the improvement came from the reminder itself versus the effect of pausing to reflect, and says it will share more results soon.

What comes next

In coming months Anthropic plans to engage additional groups, including legal scholars, psychologists, writers and civic institutions. Many future conversations will expand beyond moral formation toward broader questions about how AI is reshaping work, institutions and the distribution of power.

Anthropic intends to deepen the relationships it has started, test the input it has received against its research, and publish findings as they emerge. The company notes that these efforts are in early stages but could inform practical development choices for Claude—such as the content of Claude's constitution, the values Claude is trained to embody, and the set of behaviours the team chooses to evaluate. Anthropic thanked those who have contributed time and candid perspectives so far and indicated it will provide further details as the research progresses.