Anthropic announced on October 22, 2024 that it has released an upgraded Claude 3.5 Sonnet, introduced a new faster and lower-cost variant called Claude 3.5 Haiku, and opened a public beta for a novel "computer use" capability. The Sonnet update brings notable gains in coding and agentic tool-use tasks.
Availability
- The updated Claude 3.5 Sonnet is available to all users. The computer use beta is available today via the Anthropic API, Amazon Bedrock, and Google Cloud Vertex AI.
- Claude 3.5 Haiku will be made available later this month; it will launch initially as a text-only model with image input to follow.
- Update (2024-12-03): Claude 3.5 Haiku pricing was revised to $0.80 per MTok input / $4 per MTok output.
Claude 3.5 Sonnet — stronger software engineering capabilities
According to Anthropic, the updated Claude 3.5 Sonnet shows broad improvements on industry benchmarks, with especially strong gains for coding and agentic tasks. Reported benchmark changes include:
- SWE-bench Verified: performance rose from 33.4% to 49.0%, which Anthropic states exceeds all publicly available models in the evaluated set, including reasoning models such as OpenAI o1-preview and specialized agentic coding systems.
- TAU-bench (retail domain): improved from 62.6% to 69.2%.
- TAU-bench (more challenging airline domain): rose from 36.0% to 46.0%.
Anthropic says these improvements come at the same price and speed as the prior Sonnet. Early customer feedback indicates meaningful gains in AI-assisted coding workflows: GitLab reported up to 10% stronger reasoning across DevSecOps use cases without added latency; Cognition saw substantial improvements in coding, planning, and problem solving for autonomous AI evaluations; and The Browser Company reported Sonnet outperformed every model they had previously tested for web automation tasks.
Pre-deployment testing of the new Claude 3.5 Sonnet involved external partners: the US AI Safety Institute (US AISI) and the UK Safety Institute (UK AISI). Anthropic also evaluated the model for catastrophic risk and determined that the ASL-2 Standard described in its Responsible Scaling Policy remains appropriate for this model.
Claude 3.5 Haiku — speed, lower latency and competitive performance
Claude 3.5 Haiku is the next iteration of Anthropic's fastest model family. Anthropic reports that at similar speed to Claude 3 Haiku, the new Haiku improves across skills and outperforms the previous generation's largest model, Claude 3 Opus, on many intelligence benchmarks. It performs strongly on coding tasks; for example:
- SWE-bench Verified: 40.6%, outperforming many agents built on publicly available state-of-the-art models, and exceeding the original Claude 3.5 Sonnet and GPT-4o on some measures.
With low latency, better instruction-following, and more accurate tool use, Claude 3.5 Haiku is positioned for user-facing products, specialized sub-agent tasks, and generating personalized outputs from large datasets such as purchase history, pricing, or inventory.
Teaching Claude to navigate computers — public beta for 'computer use'
Anthropic is piloting a different approach: rather than building bespoke tools for single tasks, it is teaching general computer-use skills so Claude can operate a wide range of human-facing software. The capability lets Claude view screens, move a cursor, click buttons, and type text.
Developers can use an API that enables Claude to convert instructions (e.g., "use data from my computer and online to fill out this form") into concrete interface actions (e.g., check a spreadsheet; open a browser and navigate to pages; extract data and fill a form). On OSWorld, a benchmark evaluating AI models' ability to use computers like people do, Claude 3.5 Sonnet scored 14.9% in the screenshot-only category — notably higher than the next-best system at 7.8%. With more allowed steps, the score rose to 22.0%.
Anthropic warns the capability is experimental and imperfect: some actions people do effortlessly — scrolling, dragging, zooming — remain challenging. The company recommends developers begin with low-risk tasks. Because computer use could introduce or amplify familiar harms like spam, misinformation, or fraud, Anthropic says it has adopted proactive safety measures, including new classifiers to detect when computer use is being performed and whether harm is occurring. More details on the research and safety steps are provided in Anthropic's post on developing computer use.
Looking ahead
Anthropic says lessons from these early deployments will help clarify the potential and implications of increasingly capable AI systems. The company invites developers and customers to try the new models and the public computer use beta and to provide feedback, noting that these features could enable new ways of working with Claude.
