It depends on where you send the request. On Anthropic's own API, max_tokens does not count against your rate limits at all: Anthropic's rate limits page says "The max_tokens parameter does not factor into OTPM rate limit calculations, so there is no rate limit downside to setting a higher max_tokens value." That has been the rule only since February 2026; before then Anthropic estimated output from the cap too. On OpenAI, Azure OpenAI, Amazon Bedrock and Databricks the cap does count: each charges it, or something that includes it, when the request is admitted, before the model has written anything. A request that will write 400 tokens but sets max_tokens to 32,000 can be refused with an HTTP 429 while your dashboard shows plenty of room.

That is one common cause of the complaint "I got a 429 but I'm under my limit", which turns up on the OpenAI forum, on Microsoft Q&A and in a Claude Code issue about Bedrock. Bursts, shorter enforcement windows and temporary limits cause it too, and the last section of this guide shows how to tell them apart. The same Claude model can behave both ways: called through Anthropic's API, the cap is free; called through Bedrock or Databricks, it is reserved.

This guide carries one illustrative request through every platform: 3,000 input tokens, none of them cached, and a reply of 400 tokens, sent once with max_tokens at 1,000 and once at 32,000. Every rule below is quoted from the platform's documentation as read on 13 October 2026. We sent no requests, so where a platform's documentation is silent, this guide says so rather than guessing. On our ladder for grading AI claims, every rule here is documented, not measured.

Why a cap the reply never reaches can still use up a limit

A rate limiter has to decide whether to admit a request before the model runs, and at that moment nobody knows how long the reply will be. A platform can handle that in one of three ways:

  • Reserve the cap. Charge max_tokens at admission, then return the unused part when the reply finishes. This is safe for the provider and pessimistic for you: a large cap blocks capacity for as long as the request is in flight.
  • Charge the cap and keep it. Count max_tokens (or input plus max_tokens) against a per-minute counter and never correct it. The documentation for OpenAI and for Azure's standard deployments describes the charge but not a correction.
  • Count output as it is produced. Charge nothing for the cap and count each output token as it is generated. This is Anthropic's rule.

Input is easier to count in advance, because the prompt is in the request, but some platforms still estimate it rather than tokenise it. OpenAI says its estimate is based on the request's character count and Azure says "based in part on the character count", and Anthropic says its input limit is "estimated at the beginning of each request, and the estimate is adjusted during the request to reflect the actual number of input tokens used."

Where a cap is reserved and returned, the caps you declare set how many requests can be in flight at once. Where it is charged and kept, they set how many you can send in a minute. Either way, the length of the replies you actually get does not.

Anthropic API: max_tokens is not charged

On the Claude API, output is limited in output tokens per minute (OTPM), and Anthropic's rate limits page says OTPM limits "are evaluated in real time as output tokens are produced, counting only the actual tokens generated." The sentence quoted at the top of this guide follows: max_tokens does not factor into the calculation.

This is a recent change, and older answers still give the old rule. Until February 2026 the same page said "OTPM rate limits are estimated based on max_tokens at the beginning of each request", adjusted at the end to actual output, and advised reducing max_tokens if you hit OTPM limits early. The Internet Archive's copy from 18 February 2026 has that wording; its copy from 20 February 2026 has the current one. Anything written about Claude's rate limits before then, including answers from assistants trained on it, may describe a reservation that no longer exists.

Input has its own limit, input tokens per minute (ITPM), which is estimated at the start of the request and then corrected. For most Claude models only uncached input counts: input_tokens and cache_creation_input_tokens count toward ITPM, while cache_read_input_tokens do "NOT count toward ITPM for most models". The exception the page names is Claude Haiku 3.5, which also counts cache reads.

Both limits are enforced with a token bucket: "your capacity is continuously replenished up to your maximum limit, rather than being reset at fixed intervals." The same page lists the standard limits by tier. On the Start tier, Claude Opus 5.5 and Claude Haiku 5.5 each have 1,000 requests per minute, 2,000,000 ITPM and 400,000 OTPM.

For the example request, the admission charge is 3,000 tokens against ITPM whether max_tokens is 1,000 or 32,000, and the request then uses 400 tokens of OTPM as it writes them.

No rate-limit downside is not the same as no downside. Anthropic's errors page says to avoid "setting a large max_tokens value without using the streaming Messages API or Message Batches API", and its SDKs check that a non-streaming request is not expected to run past 10 minutes. A generous cap is free for the rate limiter, but it means you should stream.

The rate limits page also says its limits apply to Claude Platform on AWS, which is Anthropic's API sold through AWS Marketplace. That is a different product from Amazon Bedrock, whose rules come below.

OpenAI API: the larger of max_tokens and the input estimate

OpenAI's rate limits guide gives its rule under the heading "Reduce the max_tokens to match the size of your completions": "Your rate limit is calculated as the maximum of max_tokens and the estimated number of tokens based on the character count of your request." The cookbook notebook on rate limits states the same rule and its effect: "If you set max_tokens too high, your usage can be overestimated, even if the actual response is much shorter."

Read literally, the rule is a maximum, not a sum. For the example request that means 3,000 tokens at max_tokens 1,000 (the input estimate is larger) and 32,000 at max_tokens 32,000, with the prompt adding nothing. That literal reading is ours; OpenAI gives no worked example.

Three things the guide does not say, as of 13 October 2026:

  • Whether the unused part of the charge is returned when the reply finishes, and when.
  • Whether the rule applies unchanged to max_completion_tokens on Chat Completions and max_output_tokens on the Responses API. The guide names only max_tokens; that the newer parameters behave the same way is our assumption, not something OpenAI has written. A forum thread asking exactly this has no answer from OpenAI staff.
  • What is charged when no cap is set.

The question matters more on reasoning models. OpenAI's reasoning guide recommends "reserving at least 25,000 tokens for reasoning and outputs when you start experimenting with these models", and if you reserve that space by setting max_output_tokens to 25,000, the rule above would charge 25,000 tokens per request at admission. OpenAI's rate limits guide lists 1,000,000 tokens per minute for its Astra, Sol and Terra models on the Build tier, which is room for 40 such requests a minute.

Cached input does not help with OpenAI's limit. Its prompt caching guide says "Cached input tokens still count toward tokens-per-minute limits." OpenAI also warns that "unsuccessful requests contribute to your per-minute limit", so retrying immediately after a 429 makes it worse, and it documents a separate 429 with the code slow_down that "can occur even when your traffic is within its requests-per-minute and tokens-per-minute limits" because it "reflects how quickly traffic increased".

Azure OpenAI: prompt plus max_tokens, in a counter that resets each minute

Microsoft's Azure OpenAI quota page spells the rule out in full: "TPM rate limits are based on the maximum number of tokens that are estimated to be processed by a request at the time the request is received." The estimate includes the prompt text, "The max_tokens parameter setting" and "The best_of parameter setting", and its troubleshooting section repeats that limits "are calculated using the estimated maximum token count (prompt + max_tokens), not the actual tokens generated." Each increment of best_of "multiplies the token count against your rate limit."

On a standard deployment the estimate "is added to a running token count of all requests that is reset each minute. If at any time during that minute, the TPM rate limit value is reached, then further requests will receive a 429 response code until the counter resets." That describes a fixed one-minute window, not a bucket that refills, and the page describes no correction to actual usage. For the example request the charge is 4,000 tokens at max_tokens 1,000 and 35,000 at 32,000. Requests rejected with a 400 for being too long "might still count toward rate limiting", the page adds.

Provisioned (PTU) deployments work differently. Microsoft's provisioned throughput guide says the service estimates each request's cost by "combining the prompt token count (less any cached tokens) and the specified max_tokens in the call", then "corrects the utilization estimate using actual token counts" when the request finishes, using "a variation of the leaky bucket algorithm". A request is refused only "if current utilization is at 100%", so a large cap there does not get its own request refused; it raises utilisation and lowers how many requests after it get in. Cached tokens "don't contribute to utilization", and if max_tokens is not specified, "the service estimates a value", which the guide warns can lower concurrency.

Amazon Bedrock: reserved at the start, then output weighted up to 15 times

Yes: on Bedrock, input plus max_tokens is deducted from your quota when the request is admitted. The details differ between its two endpoints, and on the older one a request can use more quota than you are billed for.

On the bedrock-runtime endpoint, AWS's token burndown page says that at the start of a request "Total input tokens + max_tokens" is deducted from your tokens-per-minute (TPM) and tokens-per-day (TPD) quotas, and "The request is throttled if you exceed a quota." During processing the deduction is adjusted, and at the end the request is charged:

InputTokenCount + CacheWriteInputTokenCount + (OutputTokenCount x burndown rate)

with "any unused tokens" returned. Cache reads are not counted. As of 13 October 2026 the page sets the burndown rate at 15 times for output tokens on Claude models of version 4.8, 10 times on Claude Opus 5.5, Claude Sonnet 5, Claude Opus 5 and Claude Fable 5.1, five times on other Anthropic models of version 4.7 and below, 10 times on OpenAI's GPT-5.6 Sol, Terra and Luna, and one for every other model. You are billed for actual tokens; the burndown applies only to the quota.

For the example request on Claude Opus 5.5, admission costs 4,000 tokens of quota at max_tokens 1,000 and 35,000 at 32,000, and once the reply is written both settle at 7,000: 3,000 input plus 400 output times 10. A small cap therefore helps you get requests admitted, but it cannot lower what a request finally uses, because the burndown on real output sets that.

This is the mechanism behind the Claude Code issue opened on 24 May 2025, when Claude Code's default output cap of 32,000 tokens met low Bedrock quotas for Claude Sonnet 4 and users saw "429 Too many tokens". One commenter reported that Claude Sonnet 4 requests with max_tokens above 39,200 failed; another put the threshold "between 35000 and 40000". Lowering CLAUDE_CODE_MAX_OUTPUT_TOKENS fixed it, and the issue was closed on 22 August 2025. Those are user reports from that time, not a current measurement.

The newer bedrock-mantle endpoint, which serves the OpenAI-compatible and Anthropic Messages APIs, has separate input and output quotas and no requests-per-minute quota. Its quota page says the request's input tokens "plus the value of max_tokens (or the model-specific maximum if max_tokens is not set)" are checked against the input-tokens-per-minute quota, and admission that would exceed it returns a 429. The cap is reserved against the input quota, not the output one. Output is then counted against the output quota as it is generated, and "If the quota is reached during generation, generation stops and the response is returned with a finish reason indicating the cutoff." Afterwards the page says "the difference between max_tokens and the actual output" is returned to the input quota, so the input quota ends up holding the input plus the actual output: 3,400 tokens for the example request.

Google: Gemini API limits count input only

The Gemini API rate limits page lists its token limit as "Tokens per minute (input)", alongside requests per minute and requests per day, so max_output_tokens plays no documented part in it. The page also describes spend-based limits for paid tiers, from $10 to $200 per 10 minutes on a rolling window, which return 429 RESOURCE_EXHAUSTED and for which Google suggests "smaller context windows or shorter outputs". The page does not say whether the spend estimate uses the output cap, and it documents no rate-limit headers.

On Google Cloud, Gemini's pay-as-you-go tier, now called Standard PayGo, has no fixed per-customer token quota: Google says a 429 there "doesn't indicate that you've hit a fixed quota. It indicates temporary high contention for a specific shared resource." Provisioned Throughput, by contrast, "estimates the output token size" when a request arrives and reconciles afterwards, without saying whether the estimate uses max_output_tokens. A request whose estimate exceeds the remaining provisioned quota is not refused by default: it "is processed as pay-as-you-go".

Claude models on Google Cloud have separate input and output quotas on newer models. Google's Claude quotas page mentions "a complex token estimation and refund system for Anthropic's Claude models" but does not describe it, so whether max_tokens is reserved there is not documented.

Databricks: max_tokens reserved against the output limit

Databricks' Foundation Model APIs limits page states the reservation model most plainly: "If you provide max_tokens in your request, Databricks uses this value to estimate and reserve output token capacity before the request is admitted for processing." If that would exceed the output-tokens-per-minute limit, "Databricks rejects the request immediately with a 429 error", and "if the actual token usage is less than the reserved max_tokens, Databricks credits the difference back". Without max_tokens, it "uses a default reservation". Input tokens are counted from the actual prompt, against a separate input limit.

The same request on six platforms

The table applies each documented rule to the example request: 3,000 uncached input tokens and a 400-token reply, with Claude Opus 5.5 wherever the rule depends on the model. The figures are our arithmetic, done with a short script that applies the quoted rules, not measurements.

Platform Limit charged At admission, max_tokens 1,000 At admission, max_tokens 32,000 Once the reply is written
Anthropic API input tokens per minute 3,000 3,000 3,000
Anthropic API output tokens per minute 0 0 400
OpenAI API tokens per minute 3,000* 32,000 not documented
Azure OpenAI, standard tokens per minute 4,000 35,000 not documented
Bedrock runtime tokens per minute 4,000 35,000 7,000
Bedrock Mantle input tokens per minute 4,000 35,000 3,400
Bedrock Mantle output tokens per minute 0 0 400
Databricks input tokens per minute 3,000 3,000 3,000
Databricks output tokens per minute 1,000 32,000 400
Gemini API input tokens per minute 3,000 3,000 3,000

* On our literal reading of OpenAI's rule as a maximum: the input estimate is larger than the cap, so the cap adds nothing.

Against an illustrative limit of 1,000,000 tokens, raising the cap from 1,000 to 32,000 changes how many of these requests fit, but what that number means depends on the platform. On OpenAI and Azure's standard deployments, where nothing is documented as returned, it is how many you can send in a minute: 333 falling to 31 on OpenAI, and 250 to 28 on Azure. On Bedrock and Databricks, which return the unused reservation, it is how many can be in flight at once: 250 to 28 on both Bedrock endpoints, and 1,000 to 31 against Databricks' output limit. On Bedrock runtime each finished request still settles at 7,000 tokens, so over a full minute you get about 142 of them whatever the cap. On Anthropic's API the number does not change, because the cap is not charged.

If you leave max_tokens out

What happens without a cap varies as much as the rule itself:

  • Anthropic API: you cannot leave it out; max_tokens is required.
  • OpenAI and Azure standard deployments: not documented.
  • Azure provisioned deployments: "the service estimates a value", which can lower concurrency.
  • Bedrock runtime: not stated on the burndown page. In the Claude Code issue, one user reported that leaving it unset gave 64,000 for Claude Sonnet 4.
  • Bedrock Mantle: the model's maximum is reserved, the largest possible charge.
  • Databricks: "a default reservation", size not stated.
  • Gemini API: not applicable, because the token limit counts input only.

How to tell which rule a 429 came from

A 429 under your limit usually means a reservation, a burst, a temporary limit or a different limit. The response and the documentation tell you which:

  • Anthropic returns a 429 that names the limit exceeded and a retry-after header, plus anthropic-ratelimit-requests-*, -input-tokens-*, -output-tokens-* and -tokens-* headers with limit, remaining and reset values. Token remaining values are "rounded to the nearest thousand". Limits can bite over shorter intervals than a minute: Anthropic says "a rate of 60 requests per minute (RPM) might be enforced as 1 request per second." New organisations may start in an Evaluation tier with limits below the published ones, and the 429s for acceleration limits and spend caps described above are separate again.
  • OpenAI returns x-ratelimit-limit-tokens, x-ratelimit-remaining-tokens and x-ratelimit-reset-tokens (and request and project-token versions), and Retry-After on some 429s. A slow_down error code means you ramped up too fast, not that you ran out; OpenAI's rule of thumb is that "once your traffic reaches 1 million input tokens per minute (TPM), increase it by no more than 50% every 15 minutes."
  • Azure OpenAI returns x-ratelimit-* headers and retry-after-ms on 429s. The most direct sign of a 429 under your limit is a temporary rate limit adjustment on a standard deployment: Microsoft says you will see "x-ratelimit-limit-tokens in response headers is lower than your deployment's configured TPM". Requests-per-minute limits are also checked "in small time windows (1–10 seconds)", so a burst can be refused while the minute's total is fine.
  • Bedrock documents no rate-limit headers. A ThrottlingException that says "Too many tokens" points to the token quotas, which include a per-day quota as well as the per-minute one; CloudWatch metrics record input and output token counts per model.
  • Claude on Google Cloud: Google warns that because of its estimation and refund system, the token usage on the console's Quota page "might be inaccurate", so check the token count metrics instead.
  • Gemini API documents no rate-limit headers. Its limits "are applied per project, not per API key", so two keys in one project share them.
  • Databricks puts limit_type, limit, current and retry_after in the 429 body.

None of these platforms documents whether its remaining header reflects the reservation or actual use. That can be tested: send a short prompt with streaming on (so the headers arrive before the reply is written), once with a cap of 16 and once with 32,000, and compare the remaining-tokens header on each response. If it drops by roughly the cap, the cap was reserved. On Anthropic, rounding to the nearest thousand means only a large gap between the caps will show.

How to set max_tokens

Set the cap from where you send the request, not from habit:

  • On Anthropic's API, set max_tokens high enough that replies are not cut off, and stream when it is large. Anthropic says there is no rate-limit downside, so a truncated reply costs you more than a generous cap does. Watch ITPM instead, and cache long stable prefixes, since cache reads do not count toward it on most models.
  • On OpenAI, Azure, Bedrock and Databricks, set the cap to the longest reply you actually expect, plus a margin. Your logs give you this: the 99th percentile of output tokens per request type is a reasonable starting point. Handle a reply that stops at the cap (stop_reason: max_tokens on Claude, an incomplete status or length finish reason on OpenAI) by continuing it, rather than setting every cap to the model's maximum.
  • On Bedrock Mantle, always send max_tokens. Leaving it out reserves the model's maximum.
  • On Bedrock runtime with Claude, plan quota around the output burndown, 10 times on Claude Opus 5.5, because a small cap only helps admission.
  • On reasoning models, remember that the output cap includes reasoning tokens, so a cap tight enough to please the rate limiter can leave the model no room to answer. Our guide to what reasoning effort controls covers how the cap and effort interact.

If you run the same model on two platforms, keep two settings. The open question worth testing is OpenAI's: whether a large cap is returned when the reply ends, or only counted once and left to age out of the window. Until OpenAI documents it, size caps there as if it is not returned.