Nathan Lambert and Florian Brand used their Interconnects podcast to walk through several intertwined debates from a busy week in AI: the mixed reception to Kimi K3.1 Pro’s benchmark results, the trade-offs around publishing open model weights, and how those choices interact with ongoing cybersecurity trends. They closed with lighter topics such as using consumer AI agents like Muse and the hiring practices at Anthropic.
Kimi K3.1 Pro benchmarks and reception
A central topic was why Kimi K3.1 Pro attracted mixed reactions despite very strong results on some leaderboards. The hosts noted that on Artificial Analysis the model ranks among the top models and appears ahead of certain published GPT-6 Astra results, while on other benchmarks — Terminal-Bench Science was cited as an example — the K3.1 Pro trails GPT-6 Astra by roughly 20 points. ProgramBench and other evaluations produced more variable outcomes.
Florian pointed out that different benchmarks probe different time horizons and problem styles: some are exploitative and reward focused, short-horizon goals, while others (like ProgramBench) expect longer exploration and more general problem solving. The result is that a single model can look excellent on one benchmark and less dominant on another.
They also discussed the release and marketing around K3.1 Pro. The podcast mentioned that Kimi planned to publish the model weights by November 4, 2024, and that the release included a period of restricted access (about two weeks of exclusivity) for some partners. The participants criticized certain aspects of the promotional campaign and disputed disclosures about paid posts.
Hosting costs and engineering trade-offs: three trillion vs 700 billion
The episode discussed practical hosting implications: large models—examples discussed included models in the ~3 trillion parameter range versus ~700 billion parameter models—demand substantially more infrastructure. The hosts mentioned needing NVL72-like rack capacity and B300 nodes to fit some of the larger model replicas, whereas smaller models can often run on H100 or H200 GPUs that are more commonly available. Model sharding is operationally awkward compared with running a single model replica per node, which affects latency, cost, and deployability.
Open weights and cyber safety design choices
Kimi’s public rationale for limiting offensive cyber capability in K3.1 Pro was discussed: the team said they intentionally down-weighted offensive cyber capabilities so that the published weights would not extend the absolute frontier of cyber capability and would remain behind frontier proprietary models. Florian argued that such a design makes sense from a responsible-disclosure perspective: broad public access helps find and patch bugs, while intentionally restraining offensive abilities reduces the risk of enabling high-impact attacks.
Cybersecurity practitioners, the hosts noted, often emphasize that real-world large-scale intrusions require long-horizon planning, persistence, and chaining multiple steps; those skills are still more available to closed models and to actors with long-term, privileged access.
The ‘‘you can hack everything’’ narrative vs reality
Nathan and Florian debated the popular narrative that once open models reach a certain capability, mass exploitation and rapid large-scale hacking will follow. They discussed the PHP2Shell vulnerability as a prominent example: Patch the Planet and other researchers reported a remote code execution bug affecting many PHP-based sites (including large WordPress ecosystems), coordinated a patch, and many sites temporarily went offline to apply fixes. Despite fear that ready access to models would cause immediate mass exploitation, the hosts observed that mass automated exploitation had not materialized at scale.
They also covered the LiteLLM compromise: LiteLLM (a multi-provider endpoint library) was breached again, and preliminary investigation suggested an API key left in a public GitHub repository for five months allowed later exploitation. This is an example where human operational error (exposed credentials, unpatched systems) played a decisive role.
Trusted-access programs and rapid patching
Both OpenAI and Anthropic have “trusted access” or defensive programs that provide closed-model capabilities to vetted security researchers and organizations. Florian argued that these programs have contributed to many timely vulnerability disclosures and fixes, because companies can trace and hold accountable misuse when they require identity verification and hardware keys for access. Nathan added that many of the notable high-profile fixes likely used closed-model capabilities behind trusted programs.
They also raised concerns about so-called abliteration or abliterated-model APIs (companies such as Abliteration.ai and Morph were named), noting that offering abliterated models over an external API raises questions about the real security benefit versus marketing claims.
Communication, politics, and Mythos
The podcast revisited the Mythos episode and its media fallout. Nathan argued that some early communications created a near-apocalyptic frame (“the boy who cried wolf” was invoked), which influenced public and policy conversations. Both hosts stressed how hard it is to assign causality: if a dramatic offensive incident happens, proving how much open weights versus closed-model access or other factors contributed will be difficult.
Anthropic and junior hiring
A short segment discussed Anthropic’s statement that it would not hire junior employees. Florian quipped that other employers will welcome that talent; Nathan warned that training and employing juniors matters for culture and for preparing the workforce for longer-term technological transitions. He suggested that declining to hire juniors might have reputational and societal consequences during larger job-market adjustments.
Practical agent usage: Muse, Dots and personal assistants
Florian described his experience using Muse after moving to the U.S.: Muse helped with bureaucratic onboarding tasks (forms, appointments, shopping), generating one-time payment cards via Stripe, and proactively reminding him about required documents. He noted the convenience but also mentioned issues: speed, browser crashes that force rebuilds, and occasional site blocks. Both hosts emphasized that they prefer human oversight when a model would contact real people, but they praised the efficiency gains for mundane, time-consuming tasks.
What’s next
Nathan and Florian expect more open-model releases in the coming weeks and months, continued vulnerability disclosures and patches, and intermittent incidents. They also expect increased activity from U.S. and Chinese model developers and a continued pace of public debate about open vs. closed models, cybersecurity implications, and operational practices. The hosts invited listeners to submit questions and topics for future episodes.
Chapters and timestamps (from the episode)
- 00:00 Kimi K3.1 Reaction
- 08:15 Kimi’s open-weight cyber principles
- 11:23 You can hack everything era
- 15:54 PHP2Shell, LiteLLM, and the new internet
- 24:00 Abliterated APIs, safeguards, and trusted access
- 31:41 Anthropic not hiring junior employees
- 36:16 Florian’s Muse usage
- 45:30 Wrap-up
The episode is available on Apple Podcasts, Spotify and other podcast platforms.
