ResearchSWE-Serve reveals gaps between local tests and real-model serving in inference pipelinesSWE-Serve is a new benchmark that evaluates repository-scale changes to inference-serving stacks using 53 executable tasks derived from 83 merged SGLang pull requests.5 min read
ResearchGoogle's ERA and John Platt: automating scientific optimization with AIJohn Platt, a long-time researcher at Google with an unusual résumé (an Academy Award, two textbook algorithms, two named asteroids and an Erdős–Bacon number of 6), leads work on Empirical Research Assistance (ERA), a system that uses large language models and evolutionary search to automate optimization for ‘‘scoreable’’ scientific problems.5 min read
ResearchDebating RSI, robotics and China’s lead: JS Denain on current uncertainties in AINathan Lambert interviewed Jean‑Stanislas “JS” Denain, senior researcher and head of the Insights Team at Epoch AI, about recent signals around rapid self‑improvement (RSI), the role of robotics in AI acceleration, and the gap between Chinese and US frontier models.6 min read
ResearchUK AI Security Institute publishes evaluation data via EvalEval's open infrastructureThe UK AI Security Institute (AISI) is using the EvalEval Coalition's infrastructure to publish evaluation methods, configurations, and verified results in a machine-readable format to improve reproducibility and comparability of AI evaluations.4 min read
ResearchEvaluating AI Agents by Tool Use and Task Completion, Not Just Model OutputsAgent evaluation has shifted from scoring single outputs to measuring full task execution in a live environment, with tool calling and environment state as central elements.8 min read
ResearchOpenAI works with independent mathematician advisory group on responsible reporting of mathematical resultsOpenAI announced it is working with an advisory group of independent mathematicians to responsibly share results in AI and mathematics, improve evaluation and communication, and build tools to support research and education.1 min read
ResearchHow post-training changed ChatGPT’s behavior and safetyPost-training techniques—supervised fine-tuning, reward modeling, and reinforcement learning with human feedback—are what turned pretrained models into conversational, safer, and tool-capable systems like ChatGPT.6 min read
ResearchToby Ord’s model suggests resource and time limits could curb an RSI-driven intelligence explosionToby Ord models how recursive self-improvement (RSI) might unfold and argues that physical, informational and temporal constraints will likely prevent unbounded, arbitrarily fast growth.3 min read
ResearchMapping the Community Around Uncensored Open-Weight AI ModelsStartup 10a Labs mapped the ecosystem of so-called “uncensored” open-weight AI models and found thousands of repositories on HuggingFace, a concentration of redistribution activity, and a shift toward Chinese-origin models.3 min read
ResearchStudy finds LLMs overestimate social punishment, risking biased predictions in applicationsResearchers from the University of Pennsylvania and the World Bank report that large language models systematically predict harsher social sanctions for norm violations than human respondents expect.3 min read
ResearchVals aims to set a new standard for AI evaluationVals, a startup founded in 2024, builds proprietary benchmarks to measure modern AI models on real-world, domain-specific tasks rather than public academic tests.3 min read
ResearchAdaptyv Bio and partners launch competition to experimentally validate over 5,000 protein designsAdaptyv Bio, in partnership with Anthropic, Modal and Twist Bioscience, is announcing a protein-design competition in which more than 5,000 designs will be experimentally validated.1 min read
