On July 11, 2023, Anthropic announced Claude 2, its next‑generation language model. Claude 2 is available via an API for businesses and through a public beta chat at claude.ai for users in the United States and the United Kingdom. The company says the model produces longer responses, handles larger inputs, and improves performance on tasks such as coding, mathematics, and reasoning.
Key capabilities and availability
- The Claude 2 API is offered to businesses at the same price as Claude 1.3. Anyone in the US and UK can create an account and try the beta chat experience.
- Anthropic increased the model’s input and output capacity: prompts can now include up to 100,000 tokens, enabling Claude 2 to process hundreds of pages of technical documentation or an entire book. The model can also generate longer documents — from memos and letters to stories of several thousand tokens — in a single run.
Performance figures
Anthropic published comparative results showing improvements over Claude 1.3:
- On the multiple choice portion of the Bar exam, Claude 2 scored 76.5%, up from 73.0% with Claude 1.3.
- On GRE reading and writing tasks, Claude 2 performs above the 90th percentile compared with graduate school applicants; on quantitative reasoning it scores similar to the median applicant.
- On the Codex HumanEval Python coding test, Claude 2 scored 71.2%, compared with 56.0% for Claude 1.3.
- On the GSM8k dataset of grade‑school math problems, Claude 2 achieved 88.0%, up from 85.2%.
Anthropic said it plans to roll out further capability improvements gradually over the coming months.
Safety and robustness
Anthropic reports ongoing work to make Claude 2 safer and harder to coax into producing offensive or dangerous outputs. In internal red‑teaming evaluations that combine automated testing and manual review, Claude 2 produced harmless responses at twice the rate of Claude 1.3 on a representative set of harmful prompts. The company notes that no model is immune to jailbreaks and that it has applied multiple safety techniques and extensive red‑teaming to reduce harmful outputs. Anthropic also cautions that AI assistants are best used for everyday tasks such as summarizing or organizing information and should not replace professional advice on physical or mental health.
Enterprise partners and use cases
Anthropic says thousands of businesses are already using the Claude API. Two partners highlighted in the announcement:
-
Jasper, a generative AI platform: Greg Larson, Vice President of Engineering at Jasper, said Claude 2 competes with other state‑of‑the‑art models across many use cases and excels in long‑form, low‑latency applications. He commented: “We are really happy to be among the first to offer Claude 2 to our customers, bringing enhanced semantics, up‑to‑date knowledge training, improved reasoning for complex prompts, and the ability to effortlessly remix existing content with a 3X larger context window.”
-
Sourcegraph and its coding assistant Cody: Sourcegraph reported that Claude 2’s improved reasoning and larger context windows (up to 100K tokens) allow Cody to provide more accurate answers and include more codebase context. Quinn Slack, CEO and Co‑founder of Sourcegraph, noted that these improvements make slow and frustrating parts of the developer workflow faster and more enjoyable.
Feedback and responsible deployment
Anthropic welcomes user and partner feedback as it expands Claude’s availability. The chat experience remains an open beta, and the company advises users to be aware that Claude — like all current models — can sometimes generate inappropriate responses. Anthropic invites businesses and users who need access in currently unsupported areas to contact them to discuss options for working with Claude.
