Repository Public
1# Rate limits23> For the complete documentation index, see [llms.txt](/llms.txt). Markdown versions of documentation pages are available by appending `.md` to the page URL.45Rate limits are restrictions that our API imposes on the number of times a user or client can6access our services within a specified period of time.78## Why do we have rate limits?910Rate limits are a common practice for APIs, and they're put in place for a few different reasons:1112- **They help protect against abuse or misuse of the API.** For example, a malicious actor could flood the API with requests in an attempt to overload it or cause disruptions in service. By setting rate limits, OpenAI can prevent this kind of activity.13- **Rate limits help ensure that everyone has fair access to the API.** If one person or organization makes an excessive number of requests, it could bog down the API for everyone else. By throttling the number of requests that a single user can make, OpenAI ensures that the most number of people have an opportunity to use the API without experiencing slowdowns.14- **Rate limits can help OpenAI manage the aggregate load on its infrastructure.** If requests to the API increase dramatically, it could tax the servers and cause performance issues. By setting rate limits, OpenAI can help maintain a smooth and consistent experience for all users.1516Please work through this document in its entirety to better understand how17 OpenAI’s rate limit system works. We include code examples and possible18 solutions to handle common issues. We also include details around how your19 rate limits are automatically increased in the usage tiers section below.2021## How do these rate limits work?2223Rate limits use metrics such as **RPM** (requests per minute), **RPD** (requests per day), **TPM** (tokens per minute), **TPD** (tokens per day), **IPM** (images per minute), and audio minutes per minute for some streaming audio models. Rate limits can be hit across any of the options depending on what occurs first. For example, you might send 20 requests with only 100 tokens to the ChatCompletions endpoint and that would fill your limit (if your RPM was 20), even if you didn't send 150k tokens (if your TPM limit was 150k) within those 20 requests.2425[Batch API](https://developers.openai.com/api/reference/resources/batches/methods/create) queue limits are calculated based on the total number of input tokens queued for a given model. Tokens from pending batch jobs are counted against your queue limit. Once a batch job is completed, its tokens are no longer counted against that model's limit.2627Other important things worth noting:2829- Rate limits are defined at the [organization level](https://developers.openai.com/api/docs/guides/production-best-practices) and at the project level, not user level.30- Rate limits vary by the [model](https://developers.openai.com/api/docs/models) being used.31- For long context models like GPT-5.5, there is a separate rate limit for long context requests. You can view these rate limits in [developer console](https://platform.openai.com/settings/organization/limits).32- OpenAI sets an approved monthly usage limit for each organization. This is separate from the [spend limits](https://developers.openai.com/api/docs/guides/spend-limits) that you can configure for an organization or project.33- Some model families have shared rate limits. Any models listed under a "shared limit" in your [organizations limit page](https://platform.openai.com/settings/organization/limits) share a rate limit between them. For example, if the listed shared TPM is 3.5M, all calls to any model in the given "shared limit" list will count towards that 3.5M.34- Vector store ingestion is also rate limited per vector store ID. `/vector_stores/{vector_store_id}/files` and `/vector_stores/{vector_store_id}/file_batches` share a limit of 300 requests per minute for each vector store. For larger ingests, prefer `/vector_stores/{vector_store_id}/file_batches`.3536## Usage tiers3738You can view the rate and usage limits for your organization under the [limits](https://platform.openai.com/settings/organization/limits) section of your account settings. As your spend on our API goes up, we automatically graduate you to the next usage tier. This usually results in an increase in rate limits across most models.3940| Tier | Qualification | Usage limits |41| ----------- | --------------------------------------------------------------------- | ---------------- |42| Free | User must be in an [allowed geography](https://developers.openai.com/api/docs/supported-countries) | $100 / month |43| Tier 1 | $5 paid | $100 / month |44| Tier 2 | $50 paid | $500 / month |45| Tier 3 | $100 paid | $1,000 / month |46| Tier 4 | $250 paid | $5,000 / month |47| Tier 5 | $1,000 paid | $200,000 / month |4849To view a high-level summary of rate limits per model, visit the [models page](https://developers.openai.com/api/docs/models).5051### Rate limits in headers5253In addition to seeing your rate limit on your [account page](https://platform.openai.com/settings/organization/limits), you can also view important information about your rate limits such as the remaining requests, tokens, and other metadata in the headers of the HTTP response.5455Responses can include the following header fields:5657| Field | Sample Value | Description |58| ------------------------------------ | ------------ | ------------------------------------------------------------------------------------------------- |59| Retry-After | 56 | The minimum number of seconds to wait before retrying a temporary rate-limit error, when present. |60| x-ratelimit-limit-requests | 60 | The maximum number of requests that are permitted before exhausting the rate limit. |61| x-ratelimit-limit-tokens | 150000 | The maximum number of tokens that are permitted before exhausting the rate limit. |62| x-ratelimit-remaining-requests | 59 | The remaining number of requests that are permitted before exhausting the rate limit. |63| x-ratelimit-remaining-tokens | 149984 | The remaining number of tokens that are permitted before exhausting the rate limit. |64| x-ratelimit-reset-requests | 1s | The time until the rate limit (based on requests) resets to its initial state. |65| x-ratelimit-reset-tokens | 6m0s | The time until the rate limit (based on tokens) resets to its initial state. |66| x-ratelimit-limit-project-tokens | 60000 | The token limit for the project. |67| x-ratelimit-remaining-project-tokens | 57000 | The remaining number of tokens permitted before exhausting the project-scoped token rate limit. |68| x-ratelimit-reset-project-tokens | 3s | The time until the project-scoped token rate limit resets to its initial state. |6970Project-token headers may be present when a project-scoped token limit applies. `Retry-After` may be present on `429` responses caused by a temporary rate limit and `503` responses caused by temporary model overload. It does not mean that quota, billing, or other errors that require user action can be resolved by retrying.7172### Fine-tuning rate limits7374The fine-tuning rate limits for your organization can be [found in the dashboard as well](https://platform.openai.com/settings/organization/limits), and can also be retrieved via API:7576```bash77curl https://api.openai.com/v1/fine_tuning/model_limits \78 -H "Authorization: Bearer $OPENAI_API_KEY"79```808182## Error mitigation8384### Handle rapid traffic increases and model overload8586The API can return `slow_down` when your request rate increases too quickly, or `server_is_overloaded` when the requested model is temporarily overloaded. Check the HTTP status and `error.code` to tell these conditions apart:8788| HTTP status | Error type | Error code | What it means | What to do |89| ----------- | --------------------------- | ---------------------- | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |90| `429` | `rate_limit_error` | `slow_down` | Your request rate increased too quickly. | Follow `Retry-After` when it's present, reduce your request rate, and then increase it gradually. |91| `503` | `service_unavailable_error` | `server_is_overloaded` | The requested model is temporarily overloaded. | Follow `Retry-After` when it's present, then retry. If the error continues, increase the delay between retry attempts. |9293If `Retry-After` is missing, increase the delay between retries and add a small random delay.9495A `slow_down` error can occur even when your traffic is within its requests-per-minute and tokens-per-minute limits. It reflects how quickly traffic increased, not whether you exhausted those limits.9697As a rule of thumb, once your traffic reaches 1 million input tokens per minute (TPM), increase it by no more than 50% every 15 minutes. The exact point at which the ramp-rate limit applies can vary by model and traffic conditions.9899Enterprise customers whose pay-as-you-go traffic routinely hits ramp-rate limits can consider [Scale Tier](https://openai.com/api-scale-tier/) for more predictable capacity on eligible models. For GPT-5.6 and later models, see [Reserved Tier](https://openai.com/api-reserved-tier/). Capacity tiers don't change how you should handle a `slow_down` response: follow `Retry-After` when it's present, reduce traffic, and ramp gradually.100101#### Update existing error handlers102103If your application handled earlier throttling and overload responses, check both the HTTP status and `error.code`:104105- On endpoints that previously returned `503` with the `slow_down` code for both conditions, rapid traffic increases now return `429` with `slow_down`. Model overload remains `503` but uses `server_is_overloaded`.106- Video requests rejected before creating a job previously returned `429` with the `invalid_request_error` type and `rate_limit_exceeded` code for these conditions. Rapid traffic increases now return `429` with `rate_limit_error` and `slow_down`; model overload returns `503` with `service_unavailable_error` and `server_is_overloaded`. Errors reported in a video job's status are a separate case.107108Handle both `429` and `503` in your SDK error handlers. For example, Python, TypeScript, and Ruby use `RateLimitError` for `429` and `InternalServerError` for `503`; Java uses `RateLimitException` and `InternalServerException`. Keep support for earlier response codes while your application can still receive them. Other errors can use the same HTTP statuses, so inspect the error body before choosing a recovery action.109110For streaming requests, these HTTP error responses apply before the stream starts. An error after streaming begins can arrive as a stream event; don't automatically replay a request after consuming output.111112### What are some steps I can take to mitigate this?113114The OpenAI Cookbook has a [Python notebook](https://developers.openai.com/cookbook/examples/how_to_handle_rate_limits) that explains how to avoid rate limit errors, as well an example [Python script](https://github.com/openai/openai-cookbook/blob/main/examples/api_request_parallel_processor.py) for staying under rate limits while batch processing API requests.115116You should also exercise caution when providing programmatic access, bulk processing features, and automated social media posting - consider only enabling these for trusted customers.117118To protect against automated and high-volume misuse, set a usage limit for individual users within a specified time frame (daily, weekly, or monthly). Consider implementing a hard cap or a manual review process for users who exceed the limit.119120#### Retrying with exponential backoff121122When a request exceeds a temporary rate limit, the API returns a `429` error. The response can include a `Retry-After` header that tells you how many seconds to wait before trying again. Treat this value as a minimum: wait at least that long and add a small random delay so multiple clients don't retry at the same time.123124Each [official OpenAI SDK](https://developers.openai.com/api/docs/libraries#install-an-official-sdk) automatically retries eligible `429` and `503` responses, subject to its retry settings. Handling of `Retry-After`, especially long delays, varies by SDK version and configuration. Check the retry behavior of your installed version rather than assuming every server delay is supported.125126If a valid server delay exceeds the supported or configured maximum retry delay, stop retrying and defer the request rather than retrying sooner. An SDK can return the original HTTP error when it declines a delay above its limit. Continue to handle cancellation and timeout errors separately: a canceled request or expired deadline can stop retries without returning that HTTP error. A timeout for each attempt isn't necessarily a deadline for the entire operation.127128If you're using your own HTTP client, follow `Retry-After` when the header is present and contains a valid value. If it's missing or invalid, fall back to exponential backoff with jitter. Limit both the number of attempts and the total time spent retrying. If you manage retries in your application, disable SDK retries or account for them in those limits so nested retry loops don't multiply requests. Don't retry quota, billing, or other errors that require you to take action.129130Exponential backoff means waiting briefly after an unsuccessful request, then increasing the delay after each unsuccessful retry. This continues until the request succeeds or reaches a configured retry limit.131132This approach has many benefits:133134- Automatic retries means you can recover from rate limit errors without crashes or missing data135- Exponential backoff means that your first retries can be tried quickly, while still benefiting from longer delays if your first few retries fail136- Adding random jitter to the delay helps retries from all hitting at the same time.137138Note that unsuccessful requests contribute to your per-minute limit, so continuously resending a request won’t work.139140The Python examples below demonstrate fallback backoff. They don't inspect `Retry-After`: before using them, add handling for valid server hints so the wrappers don't retry sooner than requested. Disable SDK retries or account for them in your application's retry limits.141142143144##### Example 1: Using the Tenacity library145146147148Tenacity is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just about anything.149To add exponential backoff to your requests, you can use the `tenacity.retry` decorator. The below example uses the `tenacity.wait_random_exponential` function to add random exponential backoff to a request.150151Using the Tenacity library152153```python154from openai import OpenAI155from tenacity import (156 retry,157 stop_after_attempt,158 wait_random_exponential,159) # for exponential backoff160161client = OpenAI()162163164@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))165def completion_with_backoff(**kwargs):166 return client.completions.create(**kwargs)167168169completion_with_backoff(170 model="gpt-3.5-turbo-instruct",171 prompt="Once upon a time,",172)173```174175176Note that the Tenacity library is a third-party tool, and OpenAI makes no guarantees about177its reliability or security.178179180181182183184185##### Example 2: Using the backoff library186187188189Another python library that provides function decorators for backoff and retry is [backoff](https://pypi.org/project/backoff/):190191Using the Tenacity library192193```python194import backoff195import openai196from openai import OpenAI197198client = OpenAI()199200201@backoff.on_exception(backoff.expo, openai.RateLimitError)202def completions_with_backoff(**kwargs):203 return client.completions.create(**kwargs)204205206completions_with_backoff(207 model="gpt-3.5-turbo-instruct",208 prompt="Once upon a time,",209)210```211212213Like Tenacity, the backoff library is a third-party tool, and OpenAI makes no guarantees about its reliability or security.214215216217218219220221##### Example 3: Manual backoff implementation222223224If you don't want to use third-party libraries, you can implement your own backoff logic following this example:225Using manual backoff implementation226227```python228# imports229import random230import time231232import openai233from openai import OpenAI234235client = OpenAI()236237# define a retry decorator238239240def retry_with_exponential_backoff(241 func,242 initial_delay: float = 1,243 exponential_base: float = 2,244 jitter: bool = True,245 max_retries: int = 10,246 errors: tuple = (openai.RateLimitError,),247):248 """Retry a function with exponential backoff."""249250 def wrapper(*args, **kwargs):251 # Initialize variables252 num_retries = 0253 delay = initial_delay254255 # Loop until a successful response or max_retries is hit or an exception is raised256 while True:257 try:258 return func(*args, **kwargs)259260 # Retry on specific errors261 except errors:262 # Increment retries263 num_retries += 1264265 # Check if max retries has been reached266 if num_retries > max_retries:267 raise Exception(268 f"Maximum number of retries ({max_retries}) exceeded."269 )270271 # Increment the delay272 delay *= exponential_base * (1 + jitter * random.random())273274 # Sleep for the delay275 time.sleep(delay)276277 # Raise exceptions for any errors not specified278 except Exception:279 raise280281 return wrapper282283284@retry_with_exponential_backoff285def completions_with_backoff(**kwargs):286 return client.completions.create(**kwargs)287```288289Again, OpenAI makes no guarantees on the security or efficiency of this solution but it can be a good starting place for your own solution.290291292293294295#### Reduce the `max_tokens` to match the size of your completions296297Your rate limit is calculated as the maximum of `max_tokens` and the estimated number of tokens based on the character count of your request. Try to set the `max_tokens` value as close to your expected response size as possible.298299#### Batching requests300301If your use case does not require immediate responses, you can use the [Batch API](https://developers.openai.com/api/docs/guides/batch) to more easily submit and execute large collections of requests without impacting your synchronous request rate limits.302303For use cases that _do_ requires synchronous responses, the OpenAI API has separate limits for **requests per minute** and **tokens per minute**.304305If you're hitting the limit on requests per minute but have available capacity on tokens per minute, you can increase your throughput by batching multiple tasks into each request. This will allow you to process more tokens per minute, especially with our smaller models.306307Sending in a batch of prompts works exactly the same as a normal API call, except you pass in a list of strings to the prompt parameter instead of a single string. [Learn more in the Batch API guide](https://developers.openai.com/api/docs/guides/batch).