Here’s how Claude 3.5 Sonnet and GPT-4o stack up in a direct comparison

In the ever-growing large language model (LLMs) landscape, two front-runners stand out from the rest of the race: Anthropic’s Claude 3.5 Sonnet and OpenAI’s GPT-4o (the “o” stands for “Omni”). Both AIs boast impressive capabilities, but which reigns supreme? This guide dives deep into Claude 3.5 Sonnet and GPT-4o, dissecting their strengths and weaknesses across various tasks.

We’ll not only explore their relative accuracy and response speeds, but also unpack their pricing structures and service tiers. By the end of this article, you’ll be armed with the knowledge to make an informed decision about which LLM best suits your specific needs and budget.

Pricing and tiers

ChatGPT users can access the GPT-4o model for free, but the number of queries you’re allowed to ask it within a three-hour span is limited. Exceeding that limit will move you down to using GPT-3.5, OpenAI’s older, less capable model, until the cooldown timer has reset.

User’s will need to pay for the $20-per-month ChatGPT Plus subscription, a $30-per-month Teams subscription, or a market rate Enterprise sub if they want to use GPT-4o to their hearts’ content. Paying for access not only drastically increases the rate cap for using the AI model, but it also grants access to additional features like Dall-E image generation.

Access to Claude is set up in much the same way. At the free tier, users can converse with the chatbot on either the web or through the iOS app. They cal also upload images and documents to the Anthropic server and query the AI about their contents, as well as enjoy limited use of the new Claude 3.5 Sonnet model.

Paying $20 a month for a Pro account gets you everything from the free tier, as well as higher usage limits, access to both Claude 3 Opus and Haiku, priority bandwidth and availability, and the ability to create Projects with the AI centered on a set of documents or files. Paying $30 a month per person (minimum 5 people) for a Teams account nets you even higher usage limits and the ability to share chats between teammates.

Advantages of using Claude

While Claude may not enjoy the same degree of name recognition as GPT-4o, Anthropic’s latest AI model holds a number of advantages over its rival — and not just in performance benchmarks. For one, Claude boasts a significantly larger context window than ChatGPT, (200,000 characters versus 128,000).

This means it can remember and analyze a much larger chunk of previous conversation or text, leading to more nuanced and relevant responses, especially across longer interactions. Claude is also trained to prioritize accuracy over creativity in its responses.

It may struggle a bit in generating amusing haiku, but Claude excels at summarizing research papers and generating reports based on uploaded data. Claude also has a more recent knowledge cutoff date than GPT-4o: April 2024 versus October 2023.

In addition, Anthropic trained its Claude models as Constitutional AIs. This means that the system implements a set of principles to make judgments about its generated outputs, effectively guiding the model toward the sort of ethical behavior dictated by the constitution. That includes avoiding toxic or discriminatory outputs, as well as avoiding describing or advocating for illegal or unethical activities.

Advantages of using GPT-4o

GPT-4o, on the other hand, offers users a wider variety of functions, a larger knowledge base, and more flexible log-in options than Claude.

Unlike Anthropic’s offering, ChatGPT can generate images (thoughthat feature requires a subscription) and access the web to find the most up-to-date information available, not just the data it was trained on. And contrary to the sign-up process with Claude, GPT-4o users won’t have to provide their phone number for verification, or really any credentials at all, in order to start using the model And, thanks to its larger knowledge base, GPT-4o is more adept at generating creative content across images, text, and audio than Claude.

The ChatGPT website also tends to overload with user requests at a far lower rate than Claude’s.

Speed and accuracy comparison

Both models are capable of generating a wide variety of text outputs, from refining user-written prose to creating computer code and spreadsheet algorithms. When asked to generate a “piece of Python code that will measure the area of a circle and generate a graph of the function,” both returned similarly structured snippets. GPT-4o produced the following:

import numpy as np
import matplotlib.pyplot as plt

# Function to calculate the area of a circle
def calculate_area(radius):
return np.pi * radius ** 2

# Generate a range of radius values
radii = np.linspace(0, 10, 100) # radii from 0 to 10 with 100 points

# Calculate the corresponding area values
areas = calculate_area(radii)

# Plotting the function
plt.figure(figsize=(10, 6))
plt.plot(radii, areas, label=’Area = πr²’)
plt.title(‘Area of a Circle as a Function of Radius’)
plt.xlabel(‘Radius’)
plt.ylabel(‘Area’)
plt.legend()
plt.grid(True)
plt.show()

Meanwhile, Claude’s response was a bit more fleshed out:

import math
import numpy as np
import matplotlib.pyplot as plt

def circle_area(radius):
return math.pi * radius**2

# Generate radii from 0 to 10 with 100 points
radii = np.linspace(0, 10, 100)

# Calculate areas for each radius
areas = [circle_area(r) for r in radii]

# Create the plot
plt.figure(figsize=(10, 6))
plt.plot(radii, areas, ‘b-‘, linewidth=2)
plt.title(‘Area of a Circle as a Function of Radius’)
plt.xlabel(‘Radius’)
plt.ylabel(‘Area’)
plt.grid(True)

# Add some interesting points
plt.plot(0, circle_area(0), ‘ro’, label=’r = 0′)
plt.plot(1, circle_area(1), ‘ro’, label=’r = 1′)
plt.plot(5, circle_area(5), ‘ro’, label=’r = 5′)
plt.plot(10, circle_area(10), ‘ro’, label=’r = 10′)

plt.legend()
plt.show()

In terms of image identification, the two models are closely matched. When asked to identify the various aspects of the following image, both GPT-4o and Claude 3.5 were easily able to find and describe each of the 21 items present.

GPT-4o has the advantage of delivering information about recent events, given its ability to search the web for news that’s more recent than its training data. Asking Claude “what happened in Luxembourg yesterday” only gets you a note reading: “I apologize, but I don’t have access to real-time news or information about specific events that occurred yesterday in Luxembourg. My knowledge cutoff is in April 2024, and I don’t have information about events after that date.” The AI does offer a number of helpful recommendations as to where you might be able to find the information you requested.

I did find comparing the two systems to be a bit of a challenge, mostly because I was only able to make a half-dozen requests of Claude before the system locked me out for three hours. Even on the free tier of ChatGPT, I won’t be frozen out of the system entirely — I’ll just have to converse with a slightly inferior model for a while.

Which is better?