Riley Goodside and Spencer Papay write:
Anthropic, an AI startup co-founded by former employees of OpenAI, has quietly begun testing a new, ChatGPT-like AI assistant named Claude.
[…]
Anthropic’s research paper on Constitutional AI describes AnthropicLM v4-s3, a 52-billion-parameter, pre-trained model… Anthropic tells us that Claude is a new, larger model with architectural choices similar to those in the published research.
For context, GPT-3 has 175 billion parameters.
Claude can recall information across 8,000 tokens, more than any publicly known OpenAI model, though this ability was not reliable in our tests.
This is, effectively, how much “short-term memory” an AI model has. You definitely don’t want any information to be pushed out of memory during a normal chat session. Ideally, an AI model would remember information across multiple chat sessions although neither GPT-3 nor Claude have this ability at this time.
Later in the article, the authors preform some comparisons between Claude and ChatGPT (GPT-3.5). Here are the big takeaways:
- Both models are bad at math but Claude, at least occasionally, recognizes this fact and refuses to answer math problems when asked.
- ChatGPT is quite good at code generation. The code Claude generates contains significantly more errors.
- Both models appear to be broadly equivalent at logical reasoning tasks.
- Both models are good at text summarization.
The article concludes:
Overall, Claude is a serious competitor to ChatGPT, with improvements in many areas. While conceived as a demonstration of “constitutional” principles, Claude feels not only safer but more fun than ChatGPT.
And this is all with a model with somewhere around one third of the parameters of GPT-3? I have a feeling this is going to be an exciting year for LLM developments.