How to reduce AI API costs: caching, batching, routing and output limits

A large share of many AI API bills comes from a few habits: sending the same long prompt at full price, letting a model write and reason more than the task needs, using a large model for simple requests and paying for speed nobody needs. This guide covers six levers against them, with catalog prices as of October 2026. It builds on AI token pricing explained, which explains the price types.

In short:

• Measure first: fixed and variable input, visible and reasoning output.
• Prompt caching bills repeated prefixes at 2% to 25% of the input rate across the catalog.
• Output costs a median of four times as much as input: lower the effort and ask for shorter answers before switching models.
• Route routine requests to small models, which can cost a twentieth of mid-sized ones.
• Batch APIs halve token prices at Anthropic, OpenAI and Google, and OpenAI and Google sell flex processing at batch rates.
• In the worked example below, caching, lower effort and routing cut an $8,100 monthly bill to about $1,590.

Start by measuring where the tokens go

Log token counts per request type for a few days. Anthropic reports cache reads, cache writes and uncached input separately ^[1], and OpenAI lists cached and cache-write tokens in its usage details ^[2]. Four numbers per request type are enough:

1. Fixed input: system prompt, tool definitions and reference text that are the same on every call.
2. Variable input: the user's message, retrieved passages and earlier conversation turns.
3. Visible output: the answer itself.
4. Reasoning output: thinking tokens, billed as output even though the API does not return them ^[3].

Multiply each by its rate; the largest line shows where to start. OpenAI's cost guide groups the options into fewer requests, fewer tokens and smaller models ^[4].

Cache the parts of the prompt that repeat

Prompt caching bills a repeated prompt prefix at a reduced rate: across the catalog as of October 2026, between 2% and 25% of the normal input price, most often 10%. On Claude Sonnet 5.5 and GPT-6.1 Sol, a cached token costs $0.10 per million instead of $2. What decides whether you get that rate:

• Order. A hit needs an identical prefix. Anthropic builds the cache in the order tools, system prompt, messages, and a change at one level invalidates everything after it ^[1]. OpenAI advises putting stable instructions and reference material first and timestamps and user details at the end ^[2]. A date stamp at the top of a system prompt can prevent every hit.
• Minimum length. Short prefixes are not cached: below 512 tokens on current Claude models, with no error returned ^[1], below 1,024 tokens on GPT-5.6 and later ^[2] and below 4,096 tokens on Gemini 3.8 Flash and Gemini 3.1 Pro ^[5].
• Lifetime. Anthropic's default cache lasts five minutes and is refreshed free on each hit ^[1]. On GPT-5.6 and later, a prefix stays reusable for 30 minutes after its last use by default, or for 12 hours with the 12-hour option in the Responses API ^[2]. Low-traffic endpoints may never keep a cache warm.
• Writes. Anthropic and OpenAI (GPT-5.6 and later) bill default cache writes at 1.25 times the input rate, Anthropic's one-hour cache at 2 times and OpenAI's 12-hour cache at 2.5 times ^{[1] [2]}. On Claude, a five-minute write pays for itself after one cache read ^[6], so writes only matter when the prefix keeps changing. Google adds an hourly storage price, $0.50 per million tokens per hour for Gemini 3.8 Flash through 2026 ^[7].
• Settings that reset the cache. On Claude, changing tool definitions, the speed setting or the top-level effort level invalidates cached prefixes ^[1]. Choose these per workload, not per request. Current Claude models can change effort per message instead, which keeps the cache ^[8].

Track the hit rate (cached input divided by all input). If it stays low, look for content that changes near the top of the prompt.

Cut output and reasoning tokens

Among text models in the catalog, output costs a median of four times as much as input, and reasoning tokens count as output ^[3].

• Lower the effort level. Anthropic's effort parameter applies to all output tokens, including tool calls and thinking, and low is its most efficient level, meant for simpler tasks. Claude Sonnet 5.5 defaults to high, Claude Opus 5.5 and Claude Haiku 5.5 to medium ^[8]. With OpenAI's reasoning.effort, lower effort likewise favors speed and fewer tokens ^[3].
• Ask for less. Request a fixed format, such as a JSON object with only the fields you use, and tell the model not to repeat the input. On Claude Opus 5, lower effort does not reliably shorten the visible answer, and Anthropic recommends prompting for length instead ^[8].
• Cap output, with headroom. max_tokens (Anthropic) and max_output_tokens (OpenAI) limit all generated tokens, reasoning included. If a reasoning model hits the limit before it answers, you still pay for the input and reasoning tokens ^[3]. Use the cap against runaway output, not to squeeze normal answers.

Route each request to the cheapest model that handles it

As of October 2026, Claude Haiku 5.5 and GPT-6 Luna cost $0.10 per million input tokens and $0.50 per million output tokens, a twentieth of Claude Sonnet 5.5 and GPT-6.1 Sol at $2 and $10. Across the whole catalog, base input prices run from $0.035 to $10 per million tokens.

Routing works when requests differ in difficulty and you can tell them apart cheaply: by request type, simple rules or a small classifier model. Anthropic describes two starting points: a small model such as Claude Haiku 5.5, upgrading only where tests show a gap, or a stronger model such as Claude Opus 5.5, moving to lower effort or smaller models once quality is confirmed ^[9].

Anthropic's own measurements add two cautions ^[10]:

• Try lower effort first. A multi-model setup that looked cheaper than the default single model cost more than that model at lower effort.
• Keep escalation rare. If a small model consults a larger one on most tasks, running the larger model directly is the cheaper route to the same result.

Shrink the uncached input

Once the fixed prefix is cached, the variable part drives input cost. In the worked example below, 5,000 cached tokens cost $0.0005 per request on Claude Sonnet 5.5, while the 1,000 uncached tokens cost $0.002, four times as much. Trim there:

• Retrieve less. Send the passages that answer the question, not every match.
• Trim history. Each turn resends the whole conversation. Anthropic's context editing clears the oldest tool results past a threshold, and compaction replaces earlier history with a summary. Clearing invalidates cached prefixes, so clear enough tokens at once to make it worthwhile ^[11].
• Compress text. Methods such as LLMLingua drop low-information tokens; its authors report up to 20x compression with little performance loss on their test sets ^[12]. Test on your own data, as compression can drop details that matter, and leave the cached prefix alone: its tokens are already cheap.

Use batch and flex processing for work that can wait

• Batch APIs. Anthropic, OpenAI and Google take 50% off ^{[6] [13] [14]}. xAI takes 20% off, on Grok 4.7, 4.6, 4.5 and 4.3 and the Grok 4.20 models only ^[15]. At Anthropic, most batches finish within an hour ^[16]. OpenAI completes batches within 24 hours, with rate limits separate from standard ones ^[13]. Google's target is 24 hours ^[14].
• Batch plus caching. At Anthropic the discounts stack, but cache hits inside batches are best effort, typically 30% to 98%, and the one-hour cache is suggested for batches with shared context ^[16]. Google also supports caching for batch requests ^[14].
• Flex processing. OpenAI's flex tier bills normal requests at batch rates, in return for slower responses and occasional 429 errors that are not billed ^[17]. Google lists flex rates equal to its batch rates, such as $0.375 input and $1.875 output per million tokens for Gemini 3.8 Flash ^[7]. Flex suits evaluations, data enrichment and background jobs that can retry.

Check which price tier you are paying

Several modifiers apply on top of the base rate:

• Speed tiers. Anthropic's fast mode for Claude Opus 5.5 costs twice the standard rate ^[6]. On GPT-6.1 Sol, OpenAI's Fast tier (formerly Priority) also costs twice the standard rate, and Ultrafast six times as much ^[18]. xAI bills priority requests at 2x ^[15], and Google's Priority rates are 1.8 times its standard rates ^[7]. Keep them for latency-critical paths.
• Off-peak rates. DeepSeek charges half price outside its peak hours (01:00–04:00 and 06:00–10:00 UTC, Monday to Friday), so weekends are off-peak all day ^[19].
• Regional processing. US-only inference on recent Claude models costs 1.1 times the standard rate ^[6], and OpenAI adds 10% for regional processing on models released on or after March 5, 2026 ^[18]. Use it only where data rules require it.
• Prompt-length thresholds. Some models bill the whole request at a higher rate from a certain prompt size: above 272K input tokens on GPT-6.1 Sol ^[18], from 200K on Grok 4.7 ^[15], above 200K on Gemini 3.1 Pro ^[7] and above 100K on Claude Haiku 5.5 ^[6]. Splitting a document to stay below the threshold can cost less than one large request; context windows explained covers these limits.
• Introductory prices. Gemini 3.8 Flash costs $0.75 input and $3.75 output per million tokens through 2026-12-31; from 2027-01-01 the rates double to $1.50 and $7.50 ^[7]. Budget for the later rate.

The table lists the fast-mode and off-peak rates recorded in the catalog.

Worked example: one workload, three changes

A support team drafts replies to 300,000 tickets a month with Claude Sonnet 5.5. Each request sends a fixed 5,000-token prefix (instructions, tool definitions, policies) plus a 1,000-token ticket. Replies average 1,500 output tokens, thinking included. Prices are catalog prices as of October 2026; the effects of the second and third changes are assumptions to confirm with your own tests.

• Baseline. 1.8 billion input tokens cost $3,600, and 450 million output tokens cost $4,500, for $8,100 a month.
• Cache the prefix. Assuming 5,000 cache writes a month (after quiet periods or prompt edits), 300 million uncached tokens cost $600, 1.475 billion cache reads $147.50 and 25 million cache-write tokens $62.50. Input falls to $810.
• Lower effort and shorter replies. If quality holds at medium effort with a length instruction and replies average 900 tokens, output falls to $2,700.
• Route routine tickets. A Claude Haiku 5.5 call (1,200 input and 10 output tokens) classifies each ticket for $37.50 a month in total. If 60% are routine and go to Haiku 5.5, with the same caching and reply length, the Sonnet share costs $1,440 and the Haiku share about $111.

If the drafts could wait, the Batch API would roughly halve the rest to about $800, as long as cache hits hold up. As of October 2026, GPT-6.1 Sol and GPT-6 Luna have the same token and cache rates as Claude Sonnet 5.5 and Claude Haiku 5.5, so the same plan gives similar figures on OpenAI.

The calculator below starts from the baseline volumes. The full cost calculator adds the 83% cached share and the 300,000 requests and compares every model at once, without cache-write charges.

Estimate the monthly API cost

Input tokens per month

Output tokens per month

Compare costs Open the calculator

Where to go next

Start with the levers that need no new model, caching and output limits, then test routing and batch processing. Compare list prices on the AI pricing page and models on the comparison page. The methodology explains where our prices come from; more guides are in the guides section.