Chunk Tester

Q: What is a content chunk in AI search?

A chunk is a small passage — usually a few hundred tokens grouped under the nearest heading — that a retrieval system stores and searches over instead of your whole page. When someone asks an AI answer engine a question, it retrieves the individual chunks that best match and cites those, not the full URL. If a chunk reads badly on its own, it is less likely to be retrieved or quoted, even when the surrounding page is excellent.

Free, no signup. AI answer engines don't read your page — they read chunks of it, stripped of surrounding context, and a well-written page can still retrieve badly once it's been split that way. See how yours splits, and which chunks would confuse a retrieval system. Pairs with the chunking guide.

Built by Patrick Stox

Chunking strategies
Strategy	Best for	Trade-off
Structure	Headed documents	Keeps semantic sections together
Fixed	Simple baseline	Can split lists and ideas
Paragraph	Short-answer content	Less surrounding context

Note: every retrieval system chunks differently — this models a common structure-aware strategy so you can spot problems, not reproduce one vendor exactly. Token counts are a chars/4 estimate. Pasted content never leaves your browser.

Strategy Unit Target chunk size 300 tokens Overlap 15%

Runs entirely in your browser — nothing you paste is uploaded or stored. Fetch a URL sends only that address to our server to retrieve the HTML; extraction and chunking still happen in your browser. Anonymous run-level outcome counters may be used for aggregate research; URLs, domains, IPs, and identifiers are never included, and no statistic is released below 100 runs.

Feedback

Sample report

Say you paste this section:

## Redirects

A 301 redirect tells search engines and browsers that a page has moved permanently to a new URL, and it should be used whenever content is retired for good so link equity and users both land in the right place.

However, they may keep the old URL indexed for a while, continue crawling it periodically to reconfirm the redirect, and pass only most (not all) of the original ranking signals to the new destination.

…and, at this tool's smallest allowed target size (50 tokens — the engine clamps anything lower), it returns:

Example data — captured by running this page's own chunker against the text above at a 50-token target

Retrievability

2 chunks 50% stand alone 1 with issues

Median chunk ≈ 53 tokens; 106 tokens total. “Stand alone” = has heading context and no context-dependent opener.

Chunks & issues

Chunk 1 — no heading context

41 words · ~55 tokensclean

Chunk 2 — Redirects · +overlap

34 words · ~51 tokens1 issue

dangling opener

Opens with “however” — refers back to text that may be in another chunk, so it doesn't stand alone. Fix: Start with a self-contained topic sentence that names the subject.

Chunk 1 swallows the heading itself — its breadcrumb reads “no heading context” because the label only shows headings above a chunk, not one inside it. See what the chip labels mean ↓.
Chunk 2 opens with “However” — pulled out on its own, it no longer says what “they” refers to, which is the one thing keeping this report from 100% stand-alone. Rewriting the opener to name the subject (“Search engines may keep the old URL indexed…”) clears the flag.
Both bars sit under the amber “too long” line (85% and 78% of the over-length threshold) — close, but neither chunk is flagged as too long at this target.
Striped overlap bands are copied into the following chunk — they show the trailing source blocks a retriever would prepend, which is exactly why a dangling opener like “However” can retain context. More on the concept in Chunking for AI search.

How to use it

Stay on Paste content and drop in article text, Markdown, or raw HTML — or switch to Fetch a URL to pull a live page's main text (client-rendered pages may extract little; paste in that case).
Set the Target chunk size (100–800 tokens) and Overlap (0–40%) to match the retrieval system you have in mind. The default 300 tokens / 15% is a reasonable middle ground.
Press Chunk it. The content is split under its nearest headings into token-sized windows.
Read the Retrievability pills first, then scan the Chunks & issues column. Hover any chunk card to highlight exactly which text it covers in the banded preview on the left.

What the results mean

The Retrievability summary shows three pills:

Chunk count — how many pieces your content split into at the current target size.
% stand alone (green at 70%+, otherwise amber) — the share of chunks with no standalone-blocking issue, i.e. they have heading context and don't open with a context-dependent word.
N with issues — how many chunks carry at least one flag.

Each chunk card shows its heading breadcrumb, word/token count, a size bar (which turns amber once the chunk passes 1.3× the target), and any of these issue chips:

Too long — over 1.3× the target; a retriever is likely to truncate it. Split the section under more sub-headings.
No heading (hard) — no heading in or above the chunk, so a retriever can't easily rank or attribute it. Add a descriptive heading.
Dangling opener (hard) — starts with a pronoun or bare continuation (this, it, they, however, therefore…) that points back to another chunk. Open with a self-contained topic sentence.
Split structure (hard) — a list, table, or code block is cut across a boundary, so half its rows land in a different chunk. Keep the whole structure in one section.
Orphan heading — the chunk ends on a heading whose content falls into the next chunk. Keep a heading with the paragraph that follows it.

The three hard issues (no heading, dangling opener, split structure) are the ones that stop a chunk from standing alone, so they're the ones that pull the “% stand alone” number down.

How it works

The tool first turns your input into a list of blocks — headings, paragraphs, list items, table rows, code, and quotes. Pasted HTML is parsed with the browser's DOMParser (skipping nav, header, footer, and script), while Markdown and plain text go through a lightweight fallback parser. It then walks those blocks and packs them into windows near your target token size, tracking the heading breadcrumb above each block so every chunk knows its context — and it never breaks a heading away from the paragraph that follows it.

Each finished chunk is checked against the five issue rules, and the median token size, total tokens, and stand-alone percentage are rolled up into the summary. Token counts are a chars/4 estimate, not a real tokenizer, so treat them as directional. All of this runs in your browser; only the optional URL fetch touches a server (an SSRF-guarded, cached endpoint that returns the page's HTML for local extraction).

Features

Structure-aware chunking that groups blocks under the nearest heading and keeps headings with their first paragraph.
Adjustable target size (100–800 tokens) and overlap (0–40%) — re-chunks live as you drag the sliders.
Colored preview bands that map each chunk to the exact text it covers, with hover-to-highlight linking the band to its chunk card.
Five per-chunk issue flags (too long, no heading, dangling opener, split structure, orphan heading) with a specific fix on each.
Paste text/Markdown/HTML, or fetch a public URL and extract its main content.
Runs client-side for pasted content — nothing is uploaded or stored.

Limitations

This models one common structure-aware chunking strategy — it is not a specific vendor's pipeline, so real boundaries at Google, OpenAI, or any other engine will differ. Token counts are a chars/4 estimate rather than a true tokenizer. For the structure-aware strategy, overlap is copied from trailing source blocks and shown as a striped band; fixed windows overlap at character boundaries. The URL fetch reads server-rendered HTML only: client-rendered (JS-only) pages may extract little, so paste in that case. And the tool spots passages that read badly out of context — it can't tell you whether an answer engine will actually rank or cite them.

AI answers may continue to reflect cached or previously retrieved material for days to weeks after a fix. Recheck over time before treating an unchanged answer as proof that the fix failed.

Frequently asked questions

What is a content chunk in AI search?

A chunk isChunking is splitting a document into smaller passages so AI systems can embed, index, and retrieve the single most relevant piece — not the whole page — in response to a query. It's a foundational step in RAG pipelines and the conceptual cousin of Google's passage ranking. a small passage — usually a few hundred tokens grouped under the nearest heading — that a retrieval system stores and searches over instead of your whole page. When someone asks an AI answer engine a question, it retrieves the individual chunks that best match and cites those, not the full URL. If a chunk reads badly on its own, it is less likely to be retrieved or quoted, even when the surrounding page is excellent.

How many tokens should a chunk be?

There is no universal number because every retrieval system chunks differently. This tool defaults to a 300-token target with 15% overlap, which is a common middle ground, and lets you slide the target from 100 to 800 tokens. It flags a chunk as "too long" once it passes 1.3× your target, since a retriever is then likely to truncate it. The token figure is a chars/4 estimate, not a real tokenizer count.

Why does a chunk get a "dangling opener" flag?

The tool flags a chunk whose first sentence starts with a context-dependent word — a pronoun or bare continuation such as "this", "it", "they", "however", or "therefore". Those words point back to text that may sit in a different chunk, so the passage does not stand on its own once retrieval pulls it out of the page. The fix is to open the section with a self-contained topic sentence that names its subject.

Does the Chunk Tester reproduce a specific vendor like Google or OpenAI?

No. It models one common structure-aware chunking strategy — grouping blocks under the nearest heading, packing them into token-sized windows with overlap, and never splitting a heading from its first paragraph. The goal is to surface passages that would retrieve badly under almost any system, not to reproduce one vendor exactly. Treat the chunk boundaries as representative, not identical to any single engine.

Is my content sent anywhere when I paste it?

No. In paste mode the extraction and chunking run entirely in your browser and nothing is uploaded. Only the optional "Fetch a URL" mode calls a server endpoint, and that endpoint fetches the public page you name, not your pasted text. Nothing is stored either way.

Next stepQuotability & Entity-Preserving Rewriter — generate the corrected version.

Feature requests for Chunk Tester

Upvote what you want most. New ideas can be submitted from the floating Feedback menu; requests appear here once approved, and the most-wanted rise to the top.

Loading…

➕ Request a feature

New requests are reviewed before they appear here.

Where this tool helps

Common use cases

Preview retrieval chunks

See how pasted content or a public page may split into bounded text chunks at a chosen size and overlap.

Find weak standalone passages

Flag chunks that lose heading context, begin with dangling pronouns, or split a list or table awkwardly.

Tune chunking settings

Compare token size and overlap choices before configuring a retrieval or knowledge-base pipeline.

Improve answer-ready structure

Rewrite headings and passages so important facts remain understandable when retrieved away from the full page.

Watch the full workflow

Chunk Tester walkthrough

Read the transcript