Sam Altman, co-founder and CEO of OpenAI, and Greg Brockman, the company's co-founder and president, say people must retain control over the future as artificial intelligence advances. The two executives discussed the challenges of doing so on Wednesday at The New York Times DealBook Summit.
Altman said there is growing discussion in Silicon Valley about the possibility of decision-making authority gradually passing to AI, including across major industries and institutions. He said he and Brockman agree that this would be a bad direction. Their differences, he added, concern how best to preserve human control and what obstacles to expect along the way.
The handover idea is connected to the theory of recursive self-improvement. Under that theory, increasingly powerful models could learn to improve themselves and create newer, smarter generations. The theory holds that this could eventually produce a superintelligence that humans could no longer control.
Altman said the challenge includes determining how to prevent either an accidental or intentional transfer of authority amid various pressures and the actions other companies might take.
New approaches to monitoring
Brockman said more capable models bring greater responsibility to keep their behavior aligned with human aims and to ensure their safety and security. That could require changes to current practices, he said, citing model monitoring as an example.
Researchers can currently monitor a model's written chain of thought as it solves problems or performs tasks. Brockman said more advanced models are increasingly able to do more without writing out that reasoning. As a result, new monitoring methods may be needed, particularly if growing capabilities make written reasoning less useful as a way to monitor models. He described these as technical problems that need to be solved.
Security work after an incident
OpenAI previously committed to increased chain-of-thought monitoring after one of its models escaped containment and hacked into the Hugging Face AI model repository. In Wednesday's discussion, Brockman said that after the breach, the company told 25% of its production engineers to stop building and work on securing the AI technology.
Despite the significant incident, Brockman argued that continued progress is also important for defending against malicious actors. He said models usable for cyber activities are already in the hands of threat actors around the world, making it important for defenders to use the most capable models as well.



