Compression (Gzip, Brotli, Zstd)
How HTTP text compression works — Gzip vs Brotli vs Zstd, what Googlebot actually supports, how it feeds Core Web Vitals and crawler fetch limits, and how to enable and verify it by server. The web-performance deep dive on shrinking bytes on the wire.
Compression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they cross the network, negotiated per request via Accept-Encoding / Content-Encoding, with Vary: Accept-Encoding needed to keep cached variants straight. Brotli usually does better than Gzip on text (often cited around 15–20% smaller, but that's directional, not a guaranteed ratio) and is Google's stated preference when the client supports it; Gzip is the universal fallback; Zstd is a registered but not universally supported third codec, and Google's crawler docs don't state Zstd support for Googlebot. Googlebot explicitly supports gzip, deflate, and Brotli (br) — stated plainly in Google's current crawler docs, informally confirmed by Gary Illyes in 2020, and rooted in Google's 2008 'First date with the Googlebot' post. Compression isn't a ranking factor and doesn't guarantee a TTFB/LCP or Search improvement on its own; it reduces transfer size, which can help TTFB and LCP if that's actually your bottleneck, and it helps pages stay under Google's fetch limit (2 MB for HTML per Illyes' 2026 'Inside Googlebot' post). Don't blanket-exclude formats by extension alone — decide from MIME type and measured output; already-compressed formats (images, video, WOFF2, most PDFs) rarely benefit. Use higher compression for static/pre-compressed assets and a lower/mid-range level for dynamic content, benchmarked against your own traffic and CPU headroom — no single level is universally best. BREACH-style risk is scoped to responses that mix a secret with attacker-reflected content, not a reason to disable compression sitewide. And yes — the Sitemaps protocol explicitly allows gzip-compressed sitemaps.
Evidence for this claim HTTP content coding compresses transferred representations and is negotiated with Accept-Encoding and Content-Encoding. Scope: Current official or standards documentation. Confidence: high · Verified: MDN: HTTP compression Evidence for this claim Brotli is an HTTP content coding supported through the br encoding token. Scope: Current official or standards documentation. Confidence: high · Verified: RFC 7932: BrotliTL;DR — CompressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. shrinks your text files — HTML, CSS, JavaScript — before they’re sent over the internet, so the browser downloads fewer bytes and the page loads faster (assuming that’s actually your bottleneck). The two you’ll hear about are Gzip (works everywhere) and Brotli (usually smaller, and Google’s preference) — exactly how much smaller varies by file, so don’t treat any one percentage as a promise. If PageSpeed InsightsPageSpeed Insights (PSI) is a free Google tool at pagespeed.web.dev that reports two kinds of data for a URL: real-user field data from the Chrome UX Report and a single Lighthouse lab run with the 0–100 Performance score. Only the field Core Web Vitals are what Google uses for ranking. ever told you to “Enable text compression,” this is the fix.
What compression is
When someone loads your page, their browser has to download all the files that make it up. Compression is a way to make those files smaller while they travel, then have the browser unpack them at the other end. Nothing about the page changes for the visitor — they just get it faster.
It works like a quick handshake on every request:
- The browser says, “here’s what I can unpack” — it sends a header called
Accept-Encodinglisting the algorithms it understands (likegzipandbr). - The server picks one, compresses the file with it, and sends back a
Content-Encodingheader saying which one it used. - The browser unpacks it and shows the page.
That’s it. It happens automatically, per file, and you turn it on once in your server or hosting settings.
The two you need to know
- Gzip — the old reliable. Every browser and search engine supports it. It’s the safe default.
- Brotli — newer, developed at Google, and usually makes text files a bit smaller than Gzip. When a browser supports Brotli, Google says to use it — with Gzip as the fallback for anything that doesn’t.
You’ll also start seeing Zstandard (Zstd), a third option that’s fast to unpack, but it’s still early and not widely used yet.
What compression is not
- It’s not minificationMinification is the process of removing unnecessary characters — whitespace, line breaks, comments, and (for CSS/JS) redundant syntax or long identifiers — from CSS, JavaScript, or HTML source, without changing how the browser parses or executes it. It's distinct from compression (Gzip/Brotli): the two are complementary, applied minify-first, then compress.. Minifying strips out spaces and comments from your code; compression re-packs the bytes for transfer. They’re different jobs, and you do both — minify first, then compress.
- It’s not cachingCaching stores a copy of a page or resource — in a browser, a CDN edge node, or a search crawler's own cache — so it can be served again without regenerating or re-downloading it. It isn't a direct ranking factor, but it feeds page speed and crawl efficiency.. Caching saves a copy so a file doesn’t have to be sent again. Compression makes the file smaller when it is sent. (See cachingCaching stores a copy of a page or resource — in a browser, a CDN edge node, or a search crawler's own cache — so it can be served again without regenerating or re-downloading it. It isn't a direct ranking factor, but it feeds page speed and crawl efficiency. for that side.)
What to compress — and what to skip
Compress your text: HTML, CSS, JavaScript, JSON, SVG, and XML files (including sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.).
Don’t bother compressing files that are already compressed — JPGs, PNGs, GIFs, most videos, WOFF2 web fonts, and most PDFs. They won’t get smaller, and squeezing them again just wastes effort. Google made this exact point back in 2008.
Does it help SEO?
Indirectly. Compression is not a ranking factor on its own. But smaller files mean faster loads, and speed feeds Core Web VitalsGoogle's three real-user UX metrics — LCP (loading), INP (responsiveness), and CLS (visual stability) — used by Google's ranking systems, with no official weight attached, measured on field data., which is part of how Google judges page experience. So you enable compression because it makes your site faster — not because Google gives you points for the setting itself.
Want the real version — Gzip vs Brotli numbers, exactly what GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. supports and how we know, the PageSpeed audit thresholds, compression levels, and how to enable and verify it on your server? Switch to the Advanced tab.
Evidence for this claim HTTP content coding compresses transferred representations and is negotiated with Accept-Encoding and Content-Encoding. Scope: Current official or standards documentation. Confidence: high · Verified: MDN: HTTP compression Evidence for this claim Brotli is an HTTP content coding supported through the br encoding token. Scope: Current official or standards documentation. Confidence: high · Verified: RFC 7932: BrotliTL;DR — CompressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. is HTTP content-encoding negotiation: the client sends
Accept-Encoding(which can carryqweights andidentity), the server replies withContent-Encoding, per request, per resource — and a cacheable response that varies by coding needsVary: Accept-Encoding. Brotli usually beats Gzip on the same text (often cited around 15–20%, though the exact margin depends on content, codec version, and level) and is Google’s stated preference when supported; Gzip is the universal fallback; Zstd is a registered but not universally supported third codec (Google’s crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. docs don’t state Zstd support for GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer.). GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. supports gzip, deflate, and Brotli (br) — plainly documented today, informally confirmed by Gary Illyes in 2020, and rooted in Google’s 2008 “First date with the Googlebot” post. Compression isn’t a ranking factor and doesn’t guarantee a TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good./LCPLargest Contentful Paint — render time of the largest visible image or text block, relative to when the page started loading. ≤2.5 s (at the 75th percentile) is good. or Search change on its own; it shrinks transfer size, which can help TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. → LCPLargest Contentful Paint — render time of the largest visible image or text block, relative to when the page started loading. ≤2.5 s (at the 75th percentile) is good. if that’s your actual bottleneck, and it helps pages stay under Google’s fetch limit (2 MB for HTML per the 2026 “Inside Googlebot” post) — a crawl-efficiency win, not just a speed one. Decide what to (not) compress from MIME type and measured output, not a blanket extension list; BREACH-style risk is scoped to responses mixing a secret with attacker-reflected content, not a reason to disable compression sitewide. Use higher compression for static/pre-compressed assets, a lower/mid-range level for dynamic content — benchmarked against your own traffic and CPU, since no single level is universally best. And the SitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. protocol explicitly allows gzip-compressed sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing..
What compression actually is
Compression — properly, HTTP content encoding — shrinks a text-based response
body before it crosses the network, so the client downloads fewer bytes and
decompresses them locally. It’s negotiated per request. As Google’s LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. docs
describe it: “When a browser requests a resource, it will use the Accept-Encoding
HTTP request header to indicate what compression algorithms it supports.” A typical
header looks like Accept-Encoding: gzip, deflate, br. The server picks one and
signals its choice back in the Content-Encoding response header.
Two boundaries worth being precise about, straight from the HTTP specification
(RFC 9110): content coding is a transformation of the representation
data — the response body — and that’s a different thing from transfer coding,
which operates on the message as it moves across a connection. Compression as
discussed on this page is content coding, identified by Content-Encoding. The
negotiation itself isn’t just a flat list — Accept-Encoding can carry weights
(q values) ranking a client’s preference among the codings it’s willing to
accept, and it can also express a refusal (q=0); identity is the tokenA token is the smallest unit of text (or image/audio/video) an LLM processes — roughly 4 characters, or about ¾ of an English word. A context window is the maximum number of tokens (input plus output) a model can hold at once, like its short-term memory. for “no
coding applied.” (RFC 9110, §8.4 and §12.5.3.) And if a cacheable response’s body
actually differs by the negotiated coding — a gzip variant and a Brotli variant of
the same URL, say — the response needs Vary: Accept-Encoding so a cache doesn’t
serve the wrong variant to a client that can’t decode it (RFC 9110, §12.5.5; Apache’s
mod_deflate docs describe this exact Vary requirement for compressed cache
variants).
It applies to text: HTML, CSS, JavaScript, JSON, SVG, and XML (sitemaps included). Both Gzip and Brotli can shrink text assets substantially — Google’s own PageSpeed guidance cites a ceiling: “Enabling gzip compression can reduce the size of the transferred response by up to 90%.” Treat that as a ceiling, not a typical number — the actual savings on any given response depend on the content’s redundancy, its size, whether it was already minified, the codec/version and dictionary used, and the compression level, so they can’t be generalized from one site’s numbers to another’s.
Two things it is not, and both matter:
- Not minificationMinification is the process of removing unnecessary characters — whitespace, line breaks, comments, and (for CSS/JS) redundant syntax or long identifiers — from CSS, JavaScript, or HTML source, without changing how the browser parses or executes it. It's distinct from compression (Gzip/Brotli): the two are complementary, applied minify-first, then compress.. MinificationMinification is the process of removing unnecessary characters — whitespace, line breaks, comments, and (for CSS/JS) redundant syntax or long identifiers — from CSS, JavaScript, or HTML source, without changing how the browser parses or executes it. It's distinct from compression (Gzip/Brotli): the two are complementary, applied minify-first, then compress. removes characters from the source (whitespace, comments); compression re-encodes the resulting bytes for transfer. They stack — minify first, then compress — and skipping either leaves savings on the table.
- Not caching. Compression is about encoding (shrinking bytes on the wire);
caching is about storage and reuse (
Cache-Control,ETag, CDNs). They’re complementary “how bytes get to the browser efficiently” levers. See cachingCaching stores a copy of a page or resource — in a browser, a CDN edge node, or a search crawler's own cache — so it can be served again without regenerating or re-downloading it. It isn't a direct ranking factor, but it feeds page speed and crawl efficiency. for the storage side.
This page is the compression deep dive under the critical rendering pathThe critical rendering path (CRP) is the sequence of steps a browser must complete before it can paint the first pixel: parse HTML into the DOM, parse CSS into the CSSOM, combine them into a render tree, run layout, then paint. Optimizing it shortens time to first render and improves Core Web Vitals. hub, which lists “compress text” as one bullet in its broader critical-bytes checklist. This is where that bullet gets its full treatment.
Gzip vs Brotli vs Zstd
Compression ratio
Brotli generally does better on text, but don’t treat any specific number as
universal. Google’s LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. doc is explicit that the audit’s reported savings
are computed with Gzip, and “if Brotli is used, even more savings are possible.”
Google’s own
web.dev Brotli codelab
shows one concrete example: a main.bundle.js that was 225 KB uncompressed came down
to roughly 61.6 KB with Gzip and about 53.1 KB with Brotli — about 14% smaller with
Brotli on that one file. Numbers like that (and the “~15–20%” figure you’ll see
elsewhere) are directional, not a guarantee — the real margin on any given asset
depends on the codec/library version, the compression level chosen, the content’s
redundancy, and whether it was already minified. No codec or quality level is
“always best” independent of what you’re compressing; benchmark your own assets
rather than porting someone else’s percentage.
Gzip is the universal fallback. It’s supported everywhere, which is exactly why Google recommends it as the safety net: “Use GZIP as a fallback to Brotli. GZIP is supported in all major browsers, but is less efficient than Brotli.”
Zstd is a third registered codec, not yet a safe default. IANA’s HTTP content
coding registry lists gzip, br, and zstd — but registration in that list isn’t
the same as universal client, crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index., origin, or CDN support, so only advertise
what a given request has actually asked for and keep a working fallback path.
For HTTP interoperability specifically, RFC 9659 (which updates the earlier RFC 8878
zstd registration) requires decoders to support windows through 8 MB and forbids
encoders from requiring a larger window than that — worth knowing if you’re
configuring a server or CDN for it. That HTTP content coding, zstd, is also a
distinct thing from dictionary-based compression coding dcz; don’t conflate the
two when reading vendor docs. Real-world adoption is still early (Caddy, for
instance, exposes encode zstd gzip), and Google’s current crawler documentation
lists gzip, deflate, and Brotli for its crawlers and fetchers — it does not
state Zstandard support for Googlebot, so don’t assume it’s part of that lineup.
Treat Zstd as one to watch and enable where your stack supports it and the request
asks for it, not as a Gzip/Brotli replacement yet.
Browser and crawler support
Brotli support is now effectively universal in browsers, but that wasn’t always true — and the historical gap is worth knowing because it’s why Gzip fallback still matters. Google’s Lighthouse doc notes: “As of December 2022 Brotli is supported in all major browsers except Safari on iOS.” That Safari-on-iOS gap is the canonical example of why you keep Gzip in the chain.
Google recommends Brotli whenever the client can take it: “If the browser supports Brotli (br) you should use Brotli because it can reduce the file size of the resources more than the other compression algorithms.”
When to still fall back to Gzip
Always keep Gzip configured alongside Brotli. Content-encoding is negotiated per
request, so a properly configured server serves Brotli to clients that advertise
br and Gzip to everyone else automatically — you’re not choosing one or the
other, you’re offering both and letting the handshake decide.
Does Google (and Bing) support Brotli and Gzip?
This is the question most compression articles assert without evidence. Here it is with a dated, three-point trail of Google’s own statements.
The 2008 “First date with the Googlebot” post
Google has explained compression in its own voice since 2008. In
First date with the Googlebot: Headers and compression
(Maile Ohye and Jeremy Lilley), Google wrote that all major search engines and web
browsers support gzip compression to save bandwidth, and that you might also see
x-gzip (the same as gzip), deflate (which Google also supports), and identity
(none). The same post is where Google explains that many file formats — Flash, JPG,
PNG, GIF, PDF — are already compressed, so there’s little to gain from compressing
them again. It even states a mild preference for gzip over deflate on robustness
grounds (gzip carries a checksum and full header, leaving less guesswork). It’s an
old post with an “outdated” banner, but the body is live and directly on-topic, and
almost no current article cites it.
#:~:text= fragment links weren’t constructed for that legacy template, so they’re
paraphrased rather than presented as verbatim pull-quotes.Gary Illyes’ 2020 Brotli confirmation
Brotli support for Googlebot was confirmed informally before it was formally documented. In August 2020, Google’s Gary Illyes said, after meeting with the Googlebot team, that he’d been asked a few weeks earlier whether Googlebot supports Brotli compression — and it does. Barry Schwartz reported it the same day at Search Engine Roundtable. That predates the current crawler-overview documentation by about four years — a nice reminder that a rep can confirm crawler behavior long before it lands in the docs.
The current official crawler documentation
Today it’s stated plainly. Google’s Crawler (User Agent) Overview says: “Google’s crawlers and fetchers support the following content encodings (compressions): gzip, deflate, and Brotli (br).” And it explains the per-request negotiation the same way a browser does: “The content encodings supported by each Google user agentA user agent is the HTTP request header a client (browser, crawler, or bot) sends to identify itself. For crawlers, a short user-agent token — a substring of that string — is what robots.txt rules actually target. is advertised in the Accept-Encoding header of each request they make. For example, Accept-Encoding: gzip, deflate, br.” I say the same thing in my Ahrefs Googlebot guide, which has a dedicated content-encoding section: “Googlebot supports gzip, deflate, and Brotli (br).”
So the lineage is: 2008 blog post → 2020 Illyes confirmation → current crawler docs. This is well-established, not speculative.
The Bing documentation gap
Bing is the honest asterisk here. There’s no detailed, public Bing document
stating which content encodings BingbotBingbot is Microsoft Bing's primary web crawler — the bot that discovers, fetches, and renders pages to build the Bing index. That index also powers Yahoo, DuckDuckGo, Ecosia, and Microsoft Copilot, so Bingbot's reach is far wider than Bing's own search-market share. negotiates when crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. ordinary HTML —
nothing equivalent to Google’s crawler-overview page. What Bing does document is
narrower: its Content Submission API supports Gzip, and you can submit
gzip-compressed sitemap files (.xml.gz) to Bing. Because HTTP compression is a
standard mechanism that essentially all modern clients negotiate via Accept-Encoding,
it’s reasonable to assume Bingbot handles Gzip at minimum — but treat that as an
inference from how HTTP works, not as a documented Bing statement. It’s a genuine
documentation-transparency gap between the two engines, not a knock on Bingbot’s
actual behavior.
Why compression matters for performance and Core Web Vitals
The causal chain (state it precisely)
Compression is not a direct ranking factor. The mechanism it feeds is: smaller transfer size can shorten download time, which can help Time to First Byte and downstream Largest Contentful Paint, which factor into Core Web VitalsGoogle's three real-user UX metrics — LCP (loading), INP (responsiveness), and CLS (visual stability) — used by Google's ranking systems, with no official weight attached, measured on field data. — a page-experience signal. Note the “can”: Google’s own guidance bounds the guaranteed outcome to fewer network bytes, not a guaranteed TTFB/LCP win, better Core Web Vitals, or a Search change — if your load is bottlenecked elsewhere (a slow database query, a render-blocking script, DNS/connection setup), compressing an already-small response won’t move the needle. Measure your own bottleneck and field outcome rather than assuming compression alone fixes speed. The thing that’s scored is the speed outcome, not the compression setting. Getting this right matters because a lot of competitor content conflates “helps SEO” with “ranking factor.” (See Time to First ByteTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good., Largest Contentful PaintLargest Contentful Paint — render time of the largest visible image or text block, relative to when the page started loading. ≤2.5 s (at the 75th percentile) is good., and Core Web VitalsGoogle's three real-user UX metrics — LCP (loading), INP (responsiveness), and CLS (visual stability) — used by Google's ranking systems, with no official weight attached, measured on field data..)
There’s a nice framing from Google’s Martin Splitt on why raw page weight is a misleading way to think about this: what matters is what actually goes over the wire after compression, not the uncompressed size on disk — a page that looks like 10 MB uncompressed might be five or six over the network. That Splitt point is relayed via Search Engine Journal’s reporting rather than a primary transcript, so it’s paraphrased here, not quoted.
Compression and crawl efficiency — the underrated angle
Compression isn’t only about page speed for humans. It also helps you stay under Google’s fetch limit. As of the 2026 “Inside Googlebot” update (Gary Illyes), Googlebot fetches roughly 2 MB per URL for HTML (down from the older 15 MB figure), and content beyond the limit is truncated, not rejected — only the downloaded portion is passed on for indexingStoring a crawled page in the search index so it can appear in results. Crawled is not the same as indexed — Google selects what to keep, and indexing isn't guaranteed.. Compressing your HTML is one of the levers that keeps critical content inside that budget, so this is a crawl-efficiency concern, not just a Core Web VitalsWeb Vitals is Google's initiative (launched May 2020) for unified page-experience quality signals. Core Web Vitals — LCP, INP, and CLS — are the subset used in ranking; the rest (TTFB, FCP, TBT, Speed Index) are diagnostic, not ranking factors. one. (More on the limit in crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor..)
The Lighthouse “Enable text compression” audit — exact thresholds
The PageSpeed InsightsPageSpeed Insights (PSI) is a free Google tool at pagespeed.web.dev that reports two kinds of data for a URL: real-user field data from the Chrome UX Report and a single Lighthouse lab run with the 0–100 Performance score. Only the field Core Web Vitals are what Google uses for ranking. / Lighthouse “Enable text compression” audit has precise
trigger rules worth naming. Lighthouse gathers text-based responses that don’t
already carry a content-encoding of br, gzip, or deflate, compresses each
one with Gzip to estimate savings, and — per Google’s doc — “If the original size
of a response is less than 1.4KiB, or if the potential compression savings is less
than 10% of the original size, then Lighthouse does not flag that response in the
results.” So small files below ~1.4 KiB won’t trip the warning even uncompressed.
(As of Lighthouse 13 this audit was folded into the broader “Document request
latency” insight, but the underlying guidance is unchanged.)
Why a correctly-configured site can still fail the audit
A genuinely useful troubleshooting nugget straight from Google’s PageSpeed
documentation: “Proxy servers and anti-virus software can disable compression when
files are downloaded to a client machine.” That means a compression checker can
report a false negative — “compression isn’t enabled” — even when your server is
configured correctly, because an intermediary stripped the Content-Encoding header
in transit. Before you assume the report is broken or that your server is
misconfigured, test from a clean network path.
The public response is what matters. This shows Content-Encoding and the complete header map for each observed redirect hop, but an intermediary can still make one network path differ from another.
Inspect the transfer facts with my free HTTP Header Checker Free
- Check representative HTML, CSS, JavaScript, SVG, and XML URLs—not just the homepage.
- Confirm the final response reports gzip, br, or another intended Content-Encoding and inspect redirects separately.
- Repeat from a clean path or curl with explicit Accept-Encoding when a proxy or security product may be stripping compression.
Static vs dynamic compression — implementation trade-offs
Compression levels
Both algorithms have adjustable levels trading CPU time against ratio:
- Gzip: levels 1–9.
- Brotli: levels 0 (no compression) to 11 (maximum), per Google’s web.dev codelab.
Higher levels squeeze harder but cost more CPU time.
The CPU cost — and why static vs dynamic matters
Compression isn’t free; it burns CPU, and the cost profile is completely different for pre-built assets versus content generated on the fly. The right setting isn’t a single universal number for either case — it depends on how often the underlying content changes, how your cache/CDN layer handles the compressed variant, your traffic volume, and what you actually measure, not just a rule of thumb copied from a blog post.
- Static (pre-compressed / build-time) assets — content that’s stable enough to
compress once ahead of time (nginx’s
gzip_staticand Apache’smod_deflateboth document selecting a precompressed file instead of recompressing per request) can afford higher compression levels, because the CPU cost is paid once at build time rather than on every request. As the web.dev codelab puts it, when compression happens ahead of time, latency from high compression levels isn’t a concern anymore; the trade-off shifts to longer build times, which you pay once, not per visitor. - Dynamic (generated-per-request) responses — compressing at the highest levels in real time adds latency to every response, so a lower/mid-range level is a common starting point (tooling sometimes defaults Brotli to around quality 4 for dynamic content as a starting ratio-vs-speed balance) — but treat that as a starting point to benchmark against your own CPU headroom and traffic, not a fixed rule. Google flagged this trade-off as far back as the 2008 post: enabling gzip/deflate has a CPU cost, and on a server already heavily CPU-loaded serving dynamic content, it’s worth weighing before turning maximum compression on for everything.
The practical starting point is higher for static, lower/mid-range for dynamic — but confirm it against your own mutability, cache/CDN behavior, traffic, and a measured before/after, not a level number lifted from an article (including this one).
How to enable compression
The mechanism is the same everywhere — configure the server (or CDN/edge) to compress text responses and advertise Brotli with a Gzip fallback. By platform:
- Apache — the
mod_deflatemodule (andmod_brotlifor Brotli). Google’s PageSpeed guidance points here directly. - Nginx — the built-in
ngx_http_gzip_module(gzip on;plusgzip_types), and the Brotli module forbr. - IIS — built-in HTTP Compression (static and dynamic).
- CDN / edge — Cloudflare, Fastly, and similar typically offer Brotli + Gzip as
a toggle; Caddy exposes
encode zstd gzip. - App frameworks — Node/Express
compressionmiddleware; Next.js/Vercel and most modern hosts compress by default.
Then verify it (see the Checklists lens for the exact commands). The fastest check:
curl -I -H "Accept-Encoding: br, gzip" https://example.com/ and look for a
content-encoding header in the response.
What NOT to compress
Don’t re-compress already-compressed formats: JPG, PNG, GIF, most video, WOFF2 fonts, and most PDFs. They’re already compressed internally; running Gzip or Brotli over them again spends CPU for negligible — occasionally negative — size change. Treat that list as examples, not an exhaustive rulebook — GTmetrix’s own troubleshooting guidance frames it the same way: already-compressed or high-entropy formats “may not benefit and can grow.” The safer general rule is to scope compression by MIME type (text-based formats) and, where you can, check the measured output for a given response rather than trusting a blanket file-extension list, since format and content vary.
There’s also no single universal minimum response size below which compression “doesn’t count” — framing and metadata overhead can erase the savings on small or already-dense bodies, but where that break-even point sits depends on the codec, the implementation, the response headers, and the content itself. Lighthouse’s own 1.4 KiB / 10%-savings thresholds (above) are that specific audit’s suppression rule, not a general size floor — a CDN can (and often does) publish its own, different minimum for when it will bother compressing a response at all.
Compression and BREACH — scoped, not a reason to disable compression sitewide
You may see security advice suggesting you turn compression off out of caution. Don’t
generalize from that. BREACH-style risk isn’t a property of “compression” in the
abstract — the original research describes a specific attack shape: an HTTP response
that (1) is compressed, (2) mixes a secret (like a CSRF token) with (3)
attacker-influenced content reflected into the same response, where (4) the attacker
can observe the compressed response’s length across repeated requests to infer the
secret byte by byte. Apache’s mod_deflate documentation flags exactly this
combination as worth a BREACH-specific review. If your responses don’t combine a
secret with attacker-controlled reflected input in a compressed body, the classic
attack doesn’t apply — the fix is to review and mitigate the specific
vulnerable response/context (e.g. don’t reflect user input alongside secrets, add
per-request tokens, or rate-limit), not to disable text compression across the
whole site.
The sitemap myth — compression is explicitly allowed
One correction worth making loudly: the Sitemaps protocol explicitly allows
gzip-compressed sitemaps. Some folks believe you can’t or shouldn’t gzip a sitemap
file — that’s wrong. The
sitemaps.org protocol permits gzipped
sitemap files (.xml.gz) to reduce bandwidth, both engines accept them, and Bing’s
own submission tooling supports gzip too. The only caveat is the usual one: the
uncompressed sitemap must still respect the protocol’s size limits (50,000 URLs /
50 MB uncompressed). Gzipping the file on the wire is fine and encouraged for large
sitemaps.
Where compression sits in web performance
Compression is one lever among the cluster’s siblings. It shrinks the critical bytes the critical rendering pathThe critical rendering path (CRP) is the sequence of steps a browser must complete before it can paint the first pixel: parse HTML into the DOM, parse CSS into the CSSOM, combine them into a render tree, run layout, then paint. Optimizing it shortens time to first render and improves Core Web Vitals. has to download; it lowers TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. and downstream LCPLargest Contentful Paint — render time of the largest visible image or text block, relative to when the page started loading. ≤2.5 s (at the 75th percentile) is good.; it’s a named PageSpeed InsightsPageSpeed Insights (PSI) is a free Google tool at pagespeed.web.dev that reports two kinds of data for a URL: real-user field data from the Chrome UX Report and a single Lighthouse lab run with the 0–100 Performance score. Only the field Core Web Vitals are what Google uses for ranking. / Lighthouse audit; and it pairs with cachingCaching stores a copy of a page or resource — in a browser, a CDN edge node, or a search crawler's own cache — so it can be served again without regenerating or re-downloading it. It isn't a direct ranking factor, but it feeds page speed and crawl efficiency. (encoding vs storage) and render-blocking resourcesRender-blocking resources are CSS and synchronous JavaScript files a browser must download and process before it can paint any visible content. They sit on the critical rendering path and delay First Contentful Paint and Largest Contentful Paint. (smaller blocking files paint sooner). For the broader toolkit, see web performance tools.
AI summary
A condensed take on the Advanced version:
- CompressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. = HTTP content encoding, distinct from HTTP transfer coding
(RFC 9110, §8.4). Shrinks text responses (HTML, CSS, JS, JSON, SVG, XML sitemapsAn XML sitemap is a UTF-8 file listing the canonical URLs on your site (with optional lastmod) so search engines can discover and prioritize them. It's a discovery and diagnostic aid, not a guarantee of indexing — and Google ignores its priority and changefreq tags.)
before they cross the network. Negotiated per request: client sends
Accept-Encoding(can carryqweights andidentity), server replies withContent-Encoding; a cacheable response that varies by coding needsVary: Accept-Encoding(§12.5.5). - Three codecs: Gzip (universal fallback), Brotli (usually smaller on text — often cited ~15–20%, but that varies by content/codec version/level, not a fixed number — and Google’s stated preference when supported), Zstd (registered per IANA and scoped by RFC 9659, but not universally supported; Google’s crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. docs don’t state Zstd support for GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer.). Serve Brotli + Gzip fallback together.
- GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. supports gzip, deflate, and Brotli (
br) — stated plainly in the current crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. docs, informally confirmed by Gary Illyes in 2020, and rooted in Google’s 2008 “First date with the Googlebot” post. Bing has no equivalent crawler-compression doc — assume Gzip from HTTP standards, flag it as a gap. - Not a ranking factor, and not a guaranteed speed win. Smaller transfer size can help TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. → LCPLargest Contentful Paint — render time of the largest visible image or text block, relative to when the page started loading. ≤2.5 s (at the 75th percentile) is good. → Core Web VitalsGoogle's three real-user UX metrics — LCP (loading), INP (responsiveness), and CLS (visual stability) — used by Google's ranking systems, with no official weight attached, measured on field data., but only if that’s the actual bottleneck — Google’s own guidance bounds the direct outcome to fewer network bytes, not a promised Core Web VitalsGoogle's three real-user UX metrics — LCP (loading), INP (responsiveness), and CLS (visual stability) — used by Google's ranking systems, with no official weight attached, measured on field data. or Search change. Compression also helps HTML stay under Googlebot’s ~2 MB fetch limit (2026 “Inside Googlebot”), a crawl-efficiency win; over the limit, fetches are truncated, not rejected.
- LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. “Enable text compression” flags text responses without a
br/gzip/deflateencoding, but skips files under ~1.4 KiB or where savings are under 10% — that’s this specific audit’s suppression rule, not a universal size floor (CDNs publish their own minimums). Proxies/antivirus can stripContent-Encodingand cause a false negative. - Levels & trade-offs: Gzip 1–9, Brotli 0–11. Higher for static/pre-compressed assets (cost paid once at build); lower/mid-range for dynamic content (CPU/latency) — benchmark against your own traffic; no level is universally best.
- Don’t blanket-exclude by extension — decide from MIME type and measured output; already-compressed formats (images, video, WOFF2, most PDFs) rarely benefit.
- BREACH risk is scoped, not a reason to disable compression sitewide: it requires a compressed response mixing a secret with attacker-reflected content and an observable repeated length — review and fix that specific response/context.
- Validate the encoded representation itself:
Vary: Accept-Encodingfor cache variants, watch for double-compression across proxy/CDN/origin hops, and treatETag/Content-Length/range behavior as specific to the selected encoding, not identical across identity/gzip/br. - SitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. myth busted: the SitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. protocol explicitly allows gzip-compressed
sitemaps (
.xml.gz); uncompressed size limits still apply.
Official documentation
Primary-source documentation on compressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. and content encoding.
- Enable text compression (Lighthouse) — the audit’s trigger thresholds, Accept-Encoding negotiation, and the “prefer Brotli, fall back to Gzip” guidance.
- Minify and compress network payloads with brotli (web.dev codelab) — Brotli quality levels (0–11) and static-vs-dynamic compression trade-offs, with a before/after example.
- Enable Compression (PageSpeed Insights) — the “up to 90%” figure, server modules (mod_deflate, gzip module, IIS), and the proxy/antivirus false-negative caveat.
- Google Crawler (User Agent) Overview — the definitive “gzip, deflate, and Brotli (br)” statement and how crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. advertise it via Accept-Encoding.
- First date with the Googlebot: Headers and compression (2008) — Google’s original explanation of gzip vs deflate and which file types don’t benefit from compression.
Standards / protocol
- RFC 9110 — HTTP Semantics — defines content coding as a transformation of representation data (§8.4), distinct from transfer coding;
Accept-Encodingnegotiation withqweights andidentity(§12.5.3); and theVaryrequirement for cache variants that differ by encoding (§12.5.5). - RFC 9659 — Zstandard (zstd) as an HTTP Content Coding — updates RFC 8878, sets the 8 MB interoperable window requirement for
zstd, and distinguishes it from the dictionary codingdcz. - IANA HTTP Content Coding Registry — the authoritative list of registered coding tokensA token is the smallest unit of text (or image/audio/video) an LLM processes — roughly 4 characters, or about ¾ of an English word. A context window is the maximum number of tokens (input plus output) a model can hold at once, like its short-term memory. (
gzip,br,zstd, and others); registration isn’t the same as universal client/crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index./CDN support. - Apache
mod_deflatedocumentation —Vary: Accept-Encodingfor compressed cache variants, avoiding double-compression of precompressed content,DeflateAlterETag, and the BREACH review note. - Cloudflare — content compression — origin/edge transformation behavior, minimum response size for compression, and codec selection by plan/rule.
- nginx
ngx_http_gzip_static_module— serving precompressed static files instead of recompressing per request. - Sitemaps protocol (sitemaps.org) — confirms gzip-compressed sitemap filesA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. are allowed, and the uncompressed size limits.
Security
- BREACH attack — original research — the attack’s actual prerequisites: a compressed response mixing a secret with attacker-influenced reflected content, observed via repeated length measurement.
Bing / Microsoft
- No dedicated Bing document specifies BingbotBingbot is Microsoft Bing's primary web crawler — the bot that discovers, fetches, and renders pages to build the Bing index. That index also powers Yahoo, DuckDuckGo, Ecosia, and Microsoft Copilot, so Bingbot's reach is far wider than Bing's own search-market share.’s content-encoding support for HTML crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor.. Bing’s URL/sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. submission help confirms gzip-compressed sitemap support and Gzip on the Content Submission API — Bing Webmaster Tools help. Treat crawler-side support as an inference from HTTP standards, not a documented statement.
Quotes from the source
On-the-record statements from Google. Each link with a #:~:text= fragment jumps
to the quoted passage on the source page.
Google — LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. “Enable text compressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits.”
- “When a browser requests a resource, it will use the Accept-Encoding HTTP request header to indicate what compression algorithms it supports.” Jump to quote
- “If the browser supports Brotli (br) you should use Brotli because it can reduce the file size of the resources more than the other compression algorithms.” Jump to quote
- “The potential savings that LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. lists are the potential savings when the response is encoded with GZIP. If Brotli is used, even more savings are possible.” Jump to quote
- “If the original size of a response is less than 1.4KiB, or if the potential compression savings is less than 10% of the original size, then Lighthouse does not flag that response in the results.” Jump to quote
- “As of December 2022 Brotli is supported in all major browsers except Safari on iOS.” … “Use GZIP as a fallback to Brotli. GZIP is supported in all major browsers, but is less efficient than Brotli.” Jump to quote
Google — PageSpeed InsightsPageSpeed Insights (PSI) is a free Google tool at pagespeed.web.dev that reports two kinds of data for a URL: real-user field data from the Chrome UX Report and a single Lighthouse lab run with the 0–100 Performance score. Only the field Core Web Vitals are what Google uses for ranking. “Enable Compression”
- “Enabling gzip compression can reduce the size of the transferred response by up to 90%.” Jump to quote
- “Proxy servers and anti-virus software can disable compression when files are downloaded to a client machine.” Jump to quote
Google — CrawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. (User AgentA user agent is the HTTP request header a client (browser, crawler, or bot) sends to identify itself. For crawlers, a short user-agent token — a substring of that string — is what robots.txt rules actually target.) Overview
- “Google’s crawlersA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. and fetchers support the following content encodings (compressions): gzip, deflate, and Brotli (br).” Jump to quote
- “The content encodings supported by each Google user agentA user agent is the HTTP request header a client (browser, crawler, or bot) sends to identify itself. For crawlers, a short user-agent token — a substring of that string — is what robots.txt rules actually target. is advertised in the Accept-Encoding header of each request they make. For example, Accept-Encoding: gzip, deflate, br.” Jump to quote
Gary Illyes, Google — Brotli confirmation (2020)
- Illyes confirmed, after meeting with the GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. team, that GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. supports Brotli compression — posted to X in August 2020 and reported the same day by Barry Schwartz. (Paraphrased from the report, not reproduced as a verbatim pull-quote here.) Read the coverage
#:~:text= deep links weren’t constructed for that legacy template, so they’re
paraphrased rather than quoted. The Martin Splitt “what goes over the wire” point is
relayed via Search Engine Journal’s reporting, not a primary transcript, and is
paraphrased. The Illyes 2020 confirmation is relayed via Search Engine Roundtable and
paraphrased. Confirm any of these against the live/primary source before treating as
final. Which compression setup should I use?
Start from what you’re serving and where, not from “Gzip or Brotli?” — in practice you almost always serve both.
Q1. Can your server/CDN offer Brotli with a Gzip fallback?
- Yes (modern Nginx/Apache module, or a CDN toggle) → enable Brotli + Gzip.
Negotiation picks Brotli for clients that advertise
brand Gzip for everyone else, automatically. This is the default answer. Continue to Q2. - No (older stack, Brotli module unavailable) → enable Gzip alone for now. It’s supported everywhere and captures most of the win. Revisit when you can add Brotli.
Q2. Is the asset static (built ahead of time) or generated per request?
- Static (CSS/JS bundles, prebuilt HTML, sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.) → pre-compress at a higher level (up to Gzip 9 / Brotli 11) at build time, benchmarked for your assets. The CPU cost is paid once; every request gets the smaller file.
- Dynamic (per-request HTML from an app/CMSA content management system (CMS) is software that lets users create, manage, and publish digital content — like blog posts and pages — without writing raw code. WordPress, Drupal, and Joomla are the most common open-source CMS platforms.) → compress at a lower/mid-range level (Brotli ~4 is a common starting point to benchmark from). Maximum levels add real-time latency to every response; the right level depends on your traffic and CPU headroom, not a fixed number.
Q3. Is the file already a compressed binary? (JPG, PNG, GIF, video, WOFF2, most PDFs)
- Yes → don’t compress it. It won’t shrink and you’ll waste CPU (and can even grow the file slightly). Scope your compressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. rules to text MIME types.
- No (HTML, CSS, JS, JSON, SVG, XML) → compress it.
Q4. Your PageSpeed report still says “Enable text compression” after you turned it on.
- Is the flagged file under ~1.4 KiB, or would it save under 10%? → LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. won’t flag those anyway; if it’s not flagged, there’s nothing to fix.
- Test with
curl -I -H "Accept-Encoding: br, gzip"and see acontent-encodingheader? → the server is fine; a proxy or antivirus on the test path likely stripped the header. Retest from a clean network. - No
content-encodingheader at all → the server genuinely isn’t compressing that response — check your MIME-type scoping and module config.
Compression setup & verification — checklist
A pass to confirm text is compressed, Brotli is preferred, and nothing’s double-compressed:
- Brotli enabled with a Gzip fallback — the server advertises
brand serves Gzip to clients that don’t support it. - CompressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. scoped to text MIME types — HTML, CSS, JS, JSON, SVG, XML. Already-compressed formats (images, video, WOFF2, most PDFs) are excluded.
- Static assets pre-compressed at a higher level (up to Gzip 9 / Brotli 11) at build time — benchmarked, not just copied from a blog post.
- Dynamic responses at a lower/mid-range level to balance CPU/latency, confirmed against your own traffic.
-
Vary: Accept-Encodingpresent on cacheable responses that vary by coding, so a shared cache doesn’t serve the wrong variant. - No double-compression across proxy/CDN/origin hops, and
Content-Lengthmatches the actual encoded body at each hop. - Verified from the command line:
curl -I -H "Accept-Encoding: br, gzip" https://example.com/returns acontent-encoding: br(orgzip) header. - Verified in DevTools — Network tab shows transferred size well below resource size for text responses.
- PageSpeed InsightsPageSpeed Insights (PSI) is a free Google tool at pagespeed.web.dev that reports two kinds of data for a URL: real-user field data from the Chrome UX Report and a single Lighthouse lab run with the 0–100 Performance score. Only the field Core Web Vitals are what Google uses for ranking. / LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. — no “Enable text compression” flag on text responses over ~1.4 KiB.
- False negative ruled out — if a checker says compression is off but curl
shows a
content-encodingheader, suspect a proxy/antivirus stripping it in transit, not the server. - SitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. — large sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. served gzipped (
.xml.gz), with the uncompressed file still within the 50,000-URL / 50 MB limits. - MinificationMinification is the process of removing unnecessary characters — whitespace, line breaks, comments, and (for CSS/JS) redundant syntax or long identifiers — from CSS, JavaScript, or HTML source, without changing how the browser parses or executes it. It's distinct from compression (Gzip/Brotli): the two are complementary, applied minify-first, then compress. done too — compression stacks with minificationMinification is the process of removing unnecessary characters — whitespace, line breaks, comments, and (for CSS/JS) redundant syntax or long identifiers — from CSS, JavaScript, or HTML source, without changing how the browser parses or executes it. It's distinct from compression (Gzip/Brotli): the two are complementary, applied minify-first, then compress.; you’re doing both, not one instead of the other.
Compression cheat sheet
The three codecs
| Codec | Header | Ratio vs Gzip | Support | Use it for |
|---|---|---|---|---|
| Gzip | gzip | baseline | Universal | The always-on fallback |
| Brotli | br | Usually smaller (often cited ~15–20%, but varies by content/level) | All major browsers (Safari iOS added it by late 2022) | The preferred codec when supported |
| Zstd | zstd | Comparable, fast decompress | Registered (RFC 9659) but not universal — not confirmed for GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. | Enable where the request supports it; still low adoption |
CompressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. levels
| Codec | Range | Static assets | Dynamic responses |
|---|---|---|---|
| Gzip | 1–9 | higher (benchmark) | lower/mid-range (benchmark) |
| Brotli | 0–11 | higher (benchmark) | commonly ~4 as a starting point (benchmark) |
What to compress vs skip
| Compress | Don’t compress (already compressed) |
|---|---|
| HTML, CSS, JS | JPG, PNG, GIF |
| JSON, SVG | Video (MP4, WebM) |
| XML / sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. | WOFF2 fonts, most PDFs |
Fast facts
- GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. supports gzip, deflate, and Brotli (br) — negotiated via
Accept-Encoding(not confirmed for Zstd). - Gzip’s savings ceiling is cited up to ~90% by Google — a ceiling, not a typical number; actual savings vary by content and codec.
- LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. skips files < ~1.4 KiB or with < 10% potential savings — that audit’s own suppression rule, not a universal minimum size.
- Compression helps HTML stay under Googlebot’s ~2 MB fetch limit (2026) — a crawl-efficiency win; over the limit, fetches are truncated, not rejected.
- Cacheable responses that vary by coding need
Vary: Accept-Encoding, or a shared cache can serve the wrong variant to a client that can’t decode it. - BREACH risk is scoped to responses mixing a secret with attacker-reflected content — not a reason to disable compression sitewide.
- SitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. protocol allows gzip (
.xml.gz) — the “you can’t gzip a sitemap” belief is a myth. - Verify:
curl -I -H "Accept-Encoding: br, gzip" <url>→ look forcontent-encodingandvary.
Compression myths & mistakes
The recurring ones worth un-learning:
- “Enabling compressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. will boost my rankings.” No source, official or rep, says compression is a ranking signal. It’s an input to page speed / Core Web Vitals (page-experience-adjacent), not a scored factor on its own. Enable it for speed, describe it honestly.
- “Brotli isn’t supported by Google, so stick with Gzip.” Outdated. Google’s crawlingCrawling is how search engines use automated bots (like Googlebot and Bingbot) to discover URLs and download pages. A page has to be crawlable to be indexed, but crawling on its own isn't a ranking factor. infrastructure has supported Brotli since at least 2020 (Illyes) and it’s stated plainly in the current crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. docs. The myth survives because the news is old and easy to miss.
- “Compressing my images / fonts / videos will speed up the site.” Counter- productive. JPG, PNG, GIF, most video, and WOFF2 fonts are already compressed; re-compressing wastes CPU for negligible or zero gain, and can occasionally grow the file. Scope compression to text.
- “My PageSpeed report says compression is off, but I turned it on — the report’s
broken.” Not necessarily. Per Google’s own PSIPageSpeed Insights (PSI) is a free Google tool at pagespeed.web.dev that reports two kinds of data for a URL: real-user field data from the Chrome UX Report and a single Lighthouse lab run with the 0–100 Performance score. Only the field Core Web Vitals are what Google uses for ranking. doc, proxies or antivirus can
strip the
Content-Encodingheader before it’s measured, producing a false negative. Test from a clean path with curl before blaming the tool or the server. - “Compression and minificationMinification is the process of removing unnecessary characters — whitespace, line breaks, comments, and (for CSS/JS) redundant syntax or long identifiers — from CSS, JavaScript, or HTML source, without changing how the browser parses or executes it. It's distinct from compression (Gzip/Brotli): the two are complementary, applied minify-first, then compress. are the same thing.” Different stages: minify strips source characters, compression re-encodes bytes for transfer. They stack — do both.
- “Maximum compression level is always best.” Overstated for dynamic content. Max levels (Gzip 9, Brotli 11) cost real CPU per request; use them for static/pre-compressed assets and a mid-range level for on-the-fly responses.
- “You can’t (or shouldn’t) gzip a sitemapA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing..” Wrong — the SitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. protocol
explicitly allows gzip-compressed sitemaps (
.xml.gz); just keep the uncompressed file within the 50,000-URL / 50 MB limits.
Verify negotiated compression from the command line
Run on macOS/Linux. --compressed advertises supported encodings and decompresses the
body for display; the headers still show what traveled over the wire.
curl -sS --compressed -D - -o /dev/null https://example.com/app.js
curl -sS -H 'Accept-Encoding: br' -D - -o /dev/null https://example.com/app.js
curl -sS -H 'Accept-Encoding: gzip' -D - -o /dev/null https://example.com/app.js
curl -sS -H 'Accept-Encoding: identity' -D - -o /dev/null https://example.com/app.jsFor a cacheable resource, the response should include an appropriate
Content-Encoding and a Vary: Accept-Encoding header — check that header
explicitly if the resource sits behind a shared cache or CDN, since its absence is
what lets a cache serve one client’s encoded variant to another client that can’t
decode it.
On Windows PowerShell, request each encoding and inspect headers:
$headers = @{ "Accept-Encoding" = "br, gzip" }
$r = Invoke-WebRequest -Uri "https://example.com/app.js" -Headers $headers
$r.Headers | Select-Object Content-Encoding, Vary, Content-Length Find uncompressed same-origin text resources in DevTools
Paste into the Chrome DevTools Console after the page loads. It uses resource timing sizes as a shortlist; confirm each response in the Network panel because cached and cross-origin entries may omit sizes.
performance.getEntriesByType('resource')
.filter((r) => r.transferSize > 0 && r.encodedBodySize === r.decodedBodySize)
.map((r) => ({ url: r.name, bytes: r.transferSize }));Extract compression headers in a crawler
Use this XPath in a response-header-aware custom extraction workflow to locate the HTML marker only when your crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. has stored headers separately; for ordinary HTML, compressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. is an HTTP header and cannot be extracted from the DOM itself. That distinction prevents a common false test: XPath alone cannot prove transfer encoding.
Patrick's relevant free tools
- Hosting Checker — Find a domain's public IP network, CDN or edge platform, DNS and mail-host evidence, and response transfer facts without pretending a proxy reveals the origin.
- Page Speed Test & Core Web Vitals Checker — One sentence tells you whether a page passes Core Web Vitals and what to fix first — real Chrome user data (CrUX) with a Lighthouse lab fallback, mobile and desktop side by side, loudly-labeled data sources, and prioritized diagnostic fixes. Plus a bulk origin scorecard with CSV export.
- Core Web Vitals History & Competitor Comparison — Chart 40 weeks of real-Chrome-user Core Web Vitals — p75 LCP, INP, CLS, FCP, and TTFB from the Chrome UX Report — and compare up to 5 origins or URLs on one chart. Plus a competitor Leaderboard that ranks curated groups of SEO and page-speed tools on each metric. Pass/fail scorecards, mobile vs desktop, shareable links, CSV export.
Tools for testing HTTP compression
- Chrome DevTools Network panel — inspect
Content-Encoding, transferred size, resource type, and whether a CDN/cache changed the response. curl --compressed— test real content negotiation and compare Brotli, Gzip, and identity responses without relying on a browser UI.- PageSpeed InsightsPageSpeed Insights (PSI) is a free Google tool at pagespeed.web.dev that reports two kinds of data for a URL: real-user field data from the Chrome UX Report and a single Lighthouse lab run with the 0–100 Performance score. Only the field Core Web Vitals are what Google uses for ranking. / LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. — surface text resources that would save enough bytes to trigger the “Enable text compressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits.” audit described in the article.
- WebPageTestWebPageTest is a free, open-source, lab-based website performance testing tool — created by Patrick Meenan in 2008 and acquired by Catchpoint in 2020 — that runs pages on real browsers from distributed locations and produces deep diagnostics (waterfalls, filmstrips, connection views, Core Web Vitals). It's a diagnostic tool, not a Google ranking input. — compare transfer sizes and request waterfalls under a consistent location, browser, and network profile.
- CDN/server configuration and logs — confirm which codec and compression level were selected for static versus dynamic resources.
Prove compression is configured correctly
Encoding-negotiation test
Test to run: request a text resource separately with Accept-Encoding: br,
gzip, and identity. Expected result: supported requests receive the matching
Content-Encoding; identity remains decodable and variants include appropriate
Vary. Failure interpretation: negotiation, cache variation, or origin/CDN
configuration is wrong. Monitoring window: immediate after propagation.
Rollback trigger: corrupted bodies or a variant served with the wrong encoding.
Resource-scope test
Test to run: sample HTML, CSS, JS, JSON/SVG/XML, plus already-compressed images, video, and fonts in the Network panel. Expected result: text formats are compressed while formats that gain nothing are not recompressed. Failure interpretation: the MIME-type allowlist is incomplete or overbroad. Monitoring window: immediate. Rollback trigger: increased transfer size, excessive origin CPU, or broken assets.
Performance-regression test
Test to run: compare repeatable WebPageTestWebPageTest is a free, open-source, lab-based website performance testing tool — created by Patrick Meenan in 2008 and acquired by Catchpoint in 2020 — that runs pages on real browsers from distributed locations and produces deep diagnostics (waterfalls, filmstrips, connection views, Core Web Vitals). It's a diagnostic tool, not a Google ranking input./LighthouseLighthouse is Google's free, open-source tool that audits a page under simulated lab conditions and scores it 0–100 across Performance, Accessibility, Best Practices, and SEO. It's lab data — useful for debugging, not a ranking signal. runs and server CPU before and after enabling dynamic compressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits.. Expected result: text transfer bytes fall without a consistent TTFBTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. or error regression. Failure interpretation: the chosen codec/level costs too much CPU or compression is occurring at the wrong layer. Monitoring window: lab immediately and production during representative load. Rollback trigger: sustained latency, CPU saturation, or elevated errors.
Cache-variant (Vary) test
Test to run: for a cacheable response that’s served with different encodings to
different clients, request it with Accept-Encoding: br, then gzip, then
identity, through the same cache/CDN path. Expected result: each variant comes
back correctly encoded for what was requested, and the response carries
Vary: Accept-Encoding so the cache key includes the negotiated coding (RFC 9110,
§12.5.5). Failure interpretation: a shared cache is storing one encoded variant
and serving it to clients that can’t decode it — the classic symptom is a client that
didn’t advertise br receiving a Brotli body it can’t read. Monitoring window:
immediate after any CDN/cache-layer change. Rollback trigger: any client
receiving a body it didn’t advertise support for, or a cache hit ratio collapse from
an over-widened cache key.
Double-transformation and stale-metadata test
Test to run: trace a response through every proxy/CDN/origin hop (e.g. compare
headers at the origin versus at the edge) to confirm compression is applied exactly
once. Expected result: the final Content-Encoding names one coding, and
Content-Length matches the actual encoded body sent, with no leftover
identity-length header from before a hop transformed the body. Failure
interpretation: an intermediary decompressed and recompressed the response (or
compressed an already-compressed body) without updating length/integrity metadata —
Apache’s mod_deflate and Cloudflare’s compression docs both flag this
recompression/metadata-drift scenario explicitly. Monitoring window: after
introducing or changing any CDN, reverse proxy, or edge transformation rule.
Rollback trigger: corrupted downloads, mismatched Content-Length, or visibly
double-compressed bodies.
Validator and range-request test
Test to run: request the same resource with identity, gzip, and br, and
compare ETag, Content-Length, and behavior under a Range request for each.
Expected result: each encoded representation is treated as distinct — its own
ETag (or an explicitly documented shared-validator policy), correct Content-Length
for that specific encoding, and range semantics applied to the selected
representation, not assumed identical across identity/gzip/br. Failure
interpretation: validators or partial-content handling were built assuming a single
canonical representation, which breaks conditional requestsA conditional request lets a crawler (or browser) ask a server 'has this changed since I last fetched it?' — using If-Modified-Since (checked against your Last-Modified header) and/or If-None-Match (checked against your ETag). If nothing changed, the server replies 304 Not Modified with no body, so the crawler reuses its existing copy instead of re-downloading the page. or byte-range downloads
for compressed variants. Monitoring window: immediate, and again after any change
to compression config or CDN cachingCaching stores a copy of a page or resource — in a browser, a CDN edge node, or a search crawler's own cache — so it can be served again without regenerating or re-downloading it. It isn't a direct ranking factor, but it feeds page speed and crawl efficiency. rules. Rollback trigger: conditional
requests (If-None-Match) or range requests returning wrong or corrupted bodies for
a compressed variant.
Test yourself: Compression
Five quick questions on how HTTP text compressionCompression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits. works and what Google supports. Pick an answer for each, then check.
Compression
Compression (HTTP content encoding) shrinks text-based responses — HTML, CSS, JS, JSON, SVG, XML sitemaps — before they're sent over the network, using an algorithm like Gzip, Brotli, or Zstd, so the browser or crawler downloads fewer bytes. It's not a ranking factor, but it speeds up page loads and helps pages stay under crawler fetch limits.
Related: Caching for SEO, Critical Rendering Path, TTFB
Compression
Compression — more precisely, HTTP content encoding — is the practice of shrinking a text-based response body before it travels over the network, so the client downloads fewer bytes and decompresses them locally. It applies to text assets: HTML, CSS, JavaScript, JSON, SVG, and XML (including sitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing.). It’s negotiated per request: the client advertises what it can decode in the Accept-Encoding request header (e.g. Accept-Encoding: gzip, deflate, br), and the server responds with a Content-Encoding header naming the algorithm it actually used (gzip, br, or deflate).
The three algorithms you’ll meet:
- Gzip — the universal fallback, supported by every modern browser and crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index..
- Brotli (
br) — developed at Google and usually smaller than Gzip on the same content (often cited ~15–20%, though the exact margin varies by content, codec version, and level); Google’s stated preference when the client supports it. - Zstandard (
zstd) — a registered content coding (scoped for HTTP interoperability by RFC 9659) with fast decompression, gaining some browser/CDN support but still low adoption; Google’s crawlerA crawler — also called a spider or bot — is an automated program that fetches web pages, extracts their links, and queues new URLs to visit. Search engines use crawlers to discover and download content for their index. docs don’t state Zstd support for GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer..
Compression is distinct from minificationMinification is the process of removing unnecessary characters — whitespace, line breaks, comments, and (for CSS/JS) redundant syntax or long identifiers — from CSS, JavaScript, or HTML source, without changing how the browser parses or executes it. It's distinct from compression (Gzip/Brotli): the two are complementary, applied minify-first, then compress. (stripping whitespace and comments from source). The two stack: minify first, then compress. It’s also distinct from cachingCaching stores a copy of a page or resource — in a browser, a CDN edge node, or a search crawler's own cache — so it can be served again without regenerating or re-downloading it. It isn't a direct ranking factor, but it feeds page speed and crawl efficiency., which is about storing and reusing a response rather than shrinking its bytes on the wire — and a cacheable response that varies by encoding needs Vary: Accept-Encoding so a shared cache doesn’t serve the wrong variant.
For SEO, compression isn’t a direct ranking factor and doesn’t guarantee a Core Web VitalsGoogle's three real-user UX metrics — LCP (loading), INP (responsiveness), and CLS (visual stability) — used by Google's ranking systems, with no official weight attached, measured on field data. or Search change on its own. It matters because it reduces transfer size, which can help Time to First ByteTime to First Byte — the time from the start of a request to when the first byte of the response arrives. It's a diagnostic metric (not a Core Web Vital) and a major input to FCP and LCP; ≤0.8 s is good. and Largest Contentful PaintLargest Contentful Paint — render time of the largest visible image or text block, relative to when the page started loading. ≤2.5 s (at the 75th percentile) is good. when that’s the actual bottleneck, and because smaller responses help pages stay under a crawler’s fetch limit — GooglebotGooglebot is Google's web crawler — the software that fetches pages so Google can index and rank them. It comes in two variants, Googlebot Smartphone (primary, under mobile-first indexing) and Googlebot Desktop, and runs an evergreen Chromium renderer. supports gzip, deflate, and Brotli, and the SitemapsA sitemap is a file that lists the pages, images, videos, and other files on your site so search engines can discover them. It helps discovery, but submitting a sitemap doesn't guarantee crawling or indexing. protocol explicitly allows gzip-compressed sitemap files.
Related: Caching for SEO, Critical Rendering Path, TTFB
Build-time retrieval analysis plus live signals for this exact article. The automatic chunk report includes a deterministic readiness score and is ready without a model download.
Search Console
sampleGA4 traffic (28d)
sampleCloudflare traffic (7d)
sampledCrUX field data (28d, phone)
sampleGoogle NLP entities
localChangelog
Revision history
Compare the published article with an archived editorial snapshot. Added and removed words are shown only after you open a comparison.
Updated Jul 17, 2026.
Editorial summary and recorded change details.Summary
Qualified universal compression-ratio and CPU-level claims with RFC-backed negotiation detail, and added BREACH scoping and cache/validator checks that were missing.
Change details
-
Added RFC 9110-sourced detail on Accept-Encoding q-values, identity, and the Vary: Accept-Encoding requirement for cacheable responses that vary by encoding.
-
Replaced blanket Gzip/Brotli/Zstd ratio and compression-level claims with content/codec-dependent framing, and scoped Zstd to RFC 9659 (distinct from dcz) with a note that Google's crawler docs don't state Zstd support for Googlebot.
-
Added a BREACH section scoping the risk to responses mixing a secret with attacker-reflected content, rather than treating compression itself as a security risk.
-
Added cache-variant, double-transformation, and validator/range-request checks to the Validation Tests lens, and a Vary check to the verification script.