OpenAI API rate limits: tiers, 429s, and retries
OpenAI API rate limits explained: requests and tokens per minute, usage tiers, rate limit headers, and backoff that stops 429 errors in production.
Aymeric Zhuo10 min read
On this page
- What we'll cover
- What the limits actually measure
- Usage tiers and how you move up
- Reading the rate limit headers
- The retry pattern that works
- Controlling concurrency before you hit the wall
- Monitoring, so you find out before your users do
- Sharing an allowance between workloads
- Bulk work belongs somewhere else
- Frequently asked questions
- Related reading
Rate limits are the one category of OpenAI API limit that waiting actually fixes. They restrict how much you may do per minute rather than what you may do at all, which means the right response is a pause and a retry rather than a change to your request.
That makes them a concurrency problem more than an error-handling problem. This page covers how the limits are measured, how to read what the API tells you about your remaining allowance, and the patterns that keep throughput high without producing a stream of 429s. For context windows, output ceilings, and spend caps, which retrying cannot help with, see OpenAI API limits.
What we'll cover
What the limits actually measure
Several meters run at once, and you are limited by whichever runs out first.
Requests per minute counts calls regardless of size. Many small requests exhaust this while barely touching the token allowance.
Tokens per minute counts input and output tokens together across all your requests. Few large requests exhaust this while the request count stays low.
Requests per day applies on some models and tiers, and is the one that catches overnight batch jobs which were comfortably within the per-minute limits.
Images per minute and similar meters apply to the non-text endpoints.
Which meter binds depends entirely on the shape of your traffic, and this is worth working out before optimizing. A workflow classifying short messages is almost always request-bound, and adding concurrency makes it worse. A workflow summarizing long documents is token-bound, and the fix is different: fewer tokens per request, or a slower rate of requests.
Limits apply per model and per organization rather than per API key, so issuing more keys does not increase your allowance. Different models carry different limits, so moving to a smaller model often relieves pressure without any change to your code.
Usage tiers and how you move up
Accounts sit in a usage tier, and the tier determines the limits. Progression is based on cumulative spend and the time elapsed since the first payment, and it happens automatically rather than on request.
New organizations start at the bottom, with limits low enough that a modest load test will hit them. This is worth knowing before concluding something is wrong with your code.
Two practical points. Tier advancement requires elapsed time as well as spend, so a large prepayment does not immediately grant the top tier. And the specific numbers attached to each tier change often enough that the only trustworthy source is your own account's limits page, which shows what applies to you per model right now.
Reading the rate limit headers
Every response carries your current standing, and using it is the difference between reacting to failures and avoiding them.
x-ratelimit-limit-requestsandx-ratelimit-limit-tokensgive your ceilings.x-ratelimit-remaining-requestsandx-ratelimit-remaining-tokensgive what is left in the current window.x-ratelimit-reset-requestsandx-ratelimit-reset-tokensgive when each replenishes.
A 429 response may also carry retry-after, and where it does that value should be used directly rather than computing your own delay.
The useful pattern is to watch the remaining counts and slow down as they approach zero, rather than running at full speed until the wall arrives. Logging remaining tokens alongside your request volume also makes capacity planning straightforward, because you can see how close you run at peak rather than discovering it during an incident.
The retry pattern that works
Three properties matter, and omitting any one of them causes a specific problem.
Exponential backoff. Each retry waits longer than the last, so a service under pressure is not hammered. Fixed-interval retries against a rate limit reproduce the original load.
Jitter. A random component on each delay. Without it, a hundred requests that failed together retry together, and fail together again. Jitter is the part most often left out and it matters more than the backoff curve.
A ceiling on attempts. Retrying forever turns a temporary problem into a permanent one, with a queue that never drains.
import random
import time
import openai
def call_with_retry(client, max_attempts=5, **kwargs):
for attempt in range(max_attempts):
try:
return client.chat.completions.create(**kwargs)
except openai.RateLimitError as error:
if attempt == max_attempts - 1:
raise
retry_after = getattr(error, "retry_after", None)
if retry_after:
delay = float(retry_after)
else:
delay = min(2**attempt, 60) + random.uniform(0, 1)
time.sleep(delay)
Catch the rate limit error specifically rather than catching everything. A context window error retried five times is five identical failures and a five-times bill for the attempts that did reach the model.
Distinguish the quota 429 too, since its message concerns billing rather than rate and no amount of waiting resolves it. Treating that as retryable produces an application that appears to hang.
Controlling concurrency before you hit the wall
Retrying handles the limit being reached. Not reaching it is better, and that is a matter of controlling how many requests are in flight.
Cap parallelism explicitly. A semaphore around your calls, sized to your actual limits, is the single most effective change available. Unbounded parallelism over a list of two thousand items produces two thousand simultaneous requests and a great many 429s.
Queue rather than fan out. A worker pool pulling from a queue gives steady, predictable throughput and a natural place to apply backpressure. Firing everything at once and sorting out the failures is slower in practice as well as noisier.
Estimate tokens before sending. Where the token meter binds, track your consumption against the remaining allowance and pace accordingly, rather than counting requests.
Prefer a smaller model where it suffices. Smaller models carry more generous limits and cost less. A classification step rarely needs the largest model available, and moving it can relieve an entire pipeline.
Cache aggressively. The fastest request is the one not made. Identical inputs recur more often than people expect, particularly in retrieval pipelines.
This is a meaningful amount of infrastructure for something incidental to the work you actually wanted to do. Building on a platform that handles the model access removes it: on CodeWords you describe the automation in plain language and Cody, the automation builder, builds it, with pacing, retries, and model access handled as part of the platform rather than as code you maintain. Automations connect to more than 3,000 integrations. The free plan covers light use, with Pro at $39 per month and Business at $100 per month as usage grows; details are on the pricing page.
Monitoring, so you find out before your users do
Rate limiting is one of the few production problems with a reliable early warning, since the headers tell you how close you are running on every single response. Most teams discover this after the first incident.
Log the remaining allowance, not just the failures. Recording remaining requests and tokens on each call gives you a utilization curve. A service running at ninety percent of its token allowance at peak is one busy Monday from failing, and nothing in an error-only log shows that.
Alert on the approach rather than the arrival. A warning when sustained utilization passes a threshold is actionable. An alert on 429s tells you it has already happened.
Count retries separately from requests. A system that appears healthy while quietly retrying half its calls is doing twice the work for the same output, and the latency your users see is the sum. Retry rate is the metric that exposes this.
Watch the daily meters as well as the per-minute ones. Jobs comfortably within requests per minute can still exhaust a daily allowance, and that failure arrives late in the run when the most work has already been done.
Record which meter bound. Knowing whether you exhausted requests or tokens points directly at the fix, and the two have opposite remedies.
Sharing an allowance between workloads
Rate limits apply per organization and per model, so everything you run competes for one pool. This is the cause of the most annoying version of the problem: a backfill script started by a colleague quietly consuming the allowance that production needed.
Separate projects where the platform allows it, so development, testing, and production draw on distinct budgets and a load test cannot take down a live service.
Give production priority explicitly. Where several workloads share an allowance, interactive traffic should win over background work. A queue that batch jobs yield to is straightforward to build and prevents a whole category of incident.
Schedule background work off-peak. Enrichment, backfills, and scheduled summaries rarely care when they run, and moving them away from your busy hours costs nothing.
Publish what the allowance is. Engineers who know the ceiling design within it. Engineers who do not will write the unbounded parallel loop, and reasonably so, because nothing told them otherwise.
Bulk work belongs somewhere else
Processing a large backlog through the real-time API is the wrong shape of work for it, and it is where most rate limit pain originates.
OpenAI's Batch API accepts a file of requests and returns results within a turnaround window, at lower cost and against a separate, much larger allowance. For anything not needed in the next few seconds, this is the correct tool: overnight enrichment, backfilling a dataset, classifying an archive.
The decision is simply whether a person is waiting. If somebody is watching a screen, use the real-time API and pace it. If the results are wanted by morning, batch them and stop competing with your own production traffic.
Frequently asked questions
Why am I rate limited when my request volume is low?
The token meter is probably binding rather than the request meter. A handful of requests carrying long documents can exhaust tokens per minute while the request count stays trivial. Check which allowance the headers show as depleted.
Do multiple API keys give me more capacity?
No. Limits apply at the organization level per model, so additional keys share the same allowance. Keys are for separating and tracking usage, not for increasing it.
How long should I wait after a 429?
Use the retry-after header when it is present. Otherwise back off exponentially with jitter, starting around a second and capping at a minute. Waiting a fixed interval reproduces the collision that caused the failure.
Can I request a limit increase?
Tiers advance automatically with spend and account age rather than through a request process. Organizations with genuinely unusual requirements can talk to OpenAI, though the usual answer is that the tier will advance on its own.
Are rate limits shared across models?
No. Each model carries its own allowance, which is why moving a step to a smaller model can relieve pressure without reducing what you are doing overall.
Does streaming affect rate limits?
Streaming changes how the response reaches you rather than what it costs. The tokens count the same. It does improve perceived latency, which sometimes removes the temptation to add parallelism that would make the limits worse.
What is the difference between a 429 for rate and a 429 for quota?
The status code is shared and the message differs. A rate 429 refers to requests or tokens per minute and resolves by waiting. A quota 429 refers to billing or spend limits and resolves only when someone changes a setting. Retry logic should treat them differently.
Should I retry on a 500 or 503 as well?
Yes, and with the same backoff and jitter, since server errors are usually transient. Keep the attempt counts separate in your logging, because a rising rate of server errors indicates something quite different from a rising rate of rate limit errors and the responses differ.