USE CASE · VERIFIED 24 JUL 2026
The cheapest AI API for RAG pipelines
Retrieval-augmented generation — retrieve relevant chunks from your own documents, feed them in as context, ask the model to answer using only that material — is one of the more input-heavy workloads on this site, but it has a second variable most of our other use-case guides don't: how many chunks you can afford to stuff into a single call depends on context window, not just price per token.
Worked example: 80M input, 20M output tokens/month
A realistic shape for a RAG system answering questions against a document set — several retrieved chunks per query as input, a synthesized answer as output.
Why context window is the second axis here
Every other use-case guide on this site is mostly a price question. RAG adds a real second constraint: how many retrieved chunks fit in one call before you have to start trimming or summarizing your retrieved context, which costs quality. A model with a small context window forces you to retrieve fewer chunks or truncate them, which risks leaving out the passage that actually answers the question. Gemini's models lead the tracked set on this specifically — Gemini 3.1 Pro at 2M tokens and Gemini 2.5 Flash / Gemini 3 Flash at 1M — with Claude Sonnet 5 and Claude Opus 4.8 also at 1M. GPT-4o and GPT-4o mini sit at 128K, the same ceiling as DeepSeek V3 and Grok 4.1, which is enough for most RAG setups but leaves less room to be generous with how many chunks you retrieve per query.
Does a bigger context window actually improve answers?
Not automatically, and it's worth being honest about that. Stuffing more retrieved chunks into context can also mean more irrelevant material for the model to sift through, which sometimes hurts precision rather than helping it — a well-tuned retrieval step that returns fewer, more relevant chunks often beats a sloppier retrieval step compensated for by a bigger context window. Context headroom is insurance against a retrieval step that isn't perfectly tuned yet, not a substitute for tuning it.
The volume pattern that actually works here
Because RAG input tends to dwarf output — you're feeding in several retrieved passages to produce one synthesized answer — the input price matters more than it does in output-heavy use cases like content generation. That's exactly why GPT-4o mini and Gemini 2.5 Flash, tied cheapest on input at $0.15/M, come out so far ahead in the worked example above. The calculation changes if your RAG system does heavier synthesis — comparing multiple sources, producing a longer structured answer — which shifts more of the cost to the output side and narrows the gap.
How we'd actually decide
- Standard Q&A over a document set, cost is the priority: GPT-4o mini or Gemini 2.5 Flash — tied cheapest, and 128K is enough context for most retrieval setups.
- Retrieval step isn't tuned yet, or you need to stuff in unusually large documents whole: Gemini 3.1 Pro — 2M context gives the most headroom on this tracked set.
- Answers need to read in a consistent, on-brand voice across many queries: Claude Sonnet 5 — worth the premium for the same voice-consistency reason covered in our provider comparison.
Worked example uses standard (non-batch, non-cached) list pricing verified 24 July 2026. Use the calculator with your own volume for an exact estimate.