Scott Wiener, who recently won the congressional seat formerly held by Nancy Pelosi in San Francisco, says he intends to press for strong regulation of advanced artificial intelligence at the federal level. I spoke with Wiener a few hours after he was declared the victor; even then he was focused on AI risk.
Why AI is front of mind
On the day of our conversation, the AI nonprofit METR released a report that many interpreted as suggesting Anthropic had demonstrated signs of “recursive self‑improvement,” a process where an AI system substantially contributes to its own advancement. Wiener called the report “very concerning” and warned that time is running out to regulate AI.
"It was my hope that Congress would take decisive action before [the end of] this year," he said, adding, "That does not appear to be happening." He repeatedly used the term "recursive self‑improvement," and argued: "We need to slow this down. Because it’s moving very quickly, and we could sleepwalk into a disaster."
His track record in California
Wiener has already shaped AI policy at the state level. In 2024 he authored SB 1047, which was poised to be the nation’s first substantial frontier AI law in California before Governor Gavin Newsom vetoed it. Wiener then sponsored SB 53, a lighter bill that Newsom signed. Both measures have informed proposed federal legislation.
Campaign funding and criticism
Wiener’s congressional campaign received over $3 million from Public First Action, a nonprofit that favors AI regulation and receives significant funding from Anthropic. Opponents used this contribution to argue he was too close to the AI industry. Wiener disputes that characterization, saying Public First supported him because he was the first lawmaker in the country to introduce AI regulation and the first to get it implemented, and calling the attacks “shockingly cynical.”
What he wants from federal policy
Wiener says he would welcome a Congressional select committee on artificial intelligence and would be eager to serve on it. He supports many ideas circulating in Congress: a strong, empowered, and nimble federal regulator (a concept proposed by Representative Ro Khanna), whistleblower protections, testing and auditing requirements, and a statutory kill switch. He also wants Congress to use its oversight powers to hold AI labs accountable. "This is about the future of humanity, and we want to make sure that the public has a say and that the public knows what’s going on," he said.
States’ role and preemption concerns
As a longtime state legislator, Wiener opposes broad federal preemption of state AI laws. He invoked Louis Brandeis’s notion that states act as "laboratories" and argued that states should remain able to experiment and move more nimbly than Congress at times.
Risk framing and political hurdles
Wiener frames the AI threat in terms familiar to AI safety experts. "We’re in a moment that is very dangerous in terms of the risk of loss of control over AI," he said, warning that recursive self‑improvement could lead to a total loss of control.
He will have some allies — Senator Bernie Sanders, for example, has met influential AI safety figures and spoken about existential AI risk — but Wiener acknowledged a gap between his Bay Area, AGI‑focused perspective and broader national concerns, which often center on opposition to local data centers. On the ballot, voters approved 40 out of 41 anti‑data‑center measures on Tuesday; Wiener said those votes mostly reflect local worries about impacts on neighborhoods, electric bills and water use, rather than national debates about catastrophic AI risk.
Wiener called himself "cautiously optimistic" about Congress’s ability to rein in AI given its past record regulating tech. He raised concerns about getting legislation through President Donald Trump, who in September called catastrophic AI risk a "hoax" and has shown reluctance to cooperate with China — something Wiener regards as essential to preventing the worst outcomes.
Still, Wiener said Trump’s unpredictability offers a fraught kind of hope: "So far what we’ve seen are some problematic things, but not complete disasters," Wiener said. "I hope that it doesn’t take a disaster to spur the president into acting. And I hope if that’s what it does take—which, like I said, I hope it doesn’t—but if it does, I hope it’s as small as possible."
Summary
Scott Wiener’s victory in Pelosi’s former San Francisco district positions him to push for federal AI rules mirroring the frontier‑focused measures he promoted in California. He advocates a strong federal regulator, whistleblower protections, mandatory testing and audits, and a kill switch, while opposing wholesale federal preemption of state laws. Wiener emphasizes the urgency of addressing recursive self‑improvement and calls for meaningful, enforceable Congressional action.
