cosmodev ~/tools/embedding-chunk-planner

Document tokens

Chunk size (tokens)

Overlap (tokens)

Embedding model

Dimensions

Sample — a 10,000-token document (~40 pages) at these settings:
23 chunks · 11,408 tokens embedded (1,408 overlap overhead)
23 vectors · $0.000228 with text-embedding-3-small

Prices flow from the shared model table (USD per 1M input tokens). All math runs 100% client-side.

cosmodev ~/docs/embedding-chunk-planner.md

What it does

The Embedding Chunk Planner works out how a document splits into overlapping chunks before you send it to a retrieval-augmented generation (RAG) pipeline. Give it a token count, a chunk size, and an overlap, and it computes the number of chunks, the total tokens actually embedded (overlap re-embeds tokens at every seam), the overhead that adds, and the dollar cost for your chosen embedding model and dimensions. Prices come from the shared model table — OpenAI, Cohere, and Voyage AI — so the estimate matches real per-million-token billing.

It answers the questions that come up every time a RAG index is sized: “How many vectors will this corpus produce?” and “What does the overlap actually cost me?”

How to use it

Enter your document’s token count (a tokenizer or a tokens ≈ chars / 4 estimate both work).
Set Chunk size and Overlap in tokens — the 512/64 defaults are a common starting point.
Pick an Embedding model from the dropdown, then its Dimensions (only sizes the model offers are listed).
Read the plan: chunks, total tokens with overlap, overlap overhead, vector count, and cost.
Copy the plan, or share the URL — every input is encoded in the link.

Examples

A 1,000-token document at the 512/64 defaults, text-embedding-3-small @ 1536 dims

Chunks:            3
Tokens embedded:   1,128 (incl. overlap)
Overlap overhead:  128
Vectors:           3
Cost:              $0.000023

Same document, overlap pushed to 600

The overlap clamps to 256 (half the chunk size), so the plan becomes:

Chunks:            3
Tokens embedded:   1,512 (incl. overlap)
Overlap overhead:  512
Cost:              $0.000030

Switching to text-embedding-3-large @ 3072 dims

Same 1,128 tokens embedded, at $0.13 per million: 1128 / 1e6 * 0.13 = $0.000147.

Good to know

Overlap clamp: overlap never exceeds half the chunk size (floor(chunkSize / 2)). An overlap that large would make consecutive chunks stop advancing, so the planner clamps it — 512-token chunks cap overlap at 256 no matter what you enter. Negative overlap is treated as 0.
Why overlap costs money: every chunk boundary re-embeds overlap tokens on the next chunk. A 3-chunk plan with 64 overlap embeds 128 extra tokens — small per document, significant across a large corpus.
Shareable: all inputs live in the URL (?t=...&cs=...&ov=...&m=...&d=...), so a link reproduces the exact plan.
Private: runs 100% client-side — no document text or counts are ever sent anywhere.
Related tools: Token Generator, Regex Explainer, JSON Formatter.