Why Lighthouse Scores Lie and Still Matter for Search

The Lie

The Truth

Executive TL;DR

Technical Deep Dive

Failure Mechanism

We introduced a split-chunk async import in the client entry that depended on a transient runtime feature flag. The server rendered only the minimum shell HTML at 8.42KB and referenced a chunk that intermittently 404ed under concurrency.

Vite-generated SRI hashes drifted after a deploy pipeline race, so the browser refused the asset with no useful signal in APM. Lighthouse lab runs hit a warmed cache with the chunk present, so the score stayed inflated while field traffic absorbed the failure.

CrUX showed the damage, but aggregation hid the sharp edge of the incident. Meanwhile, the headless crawler fetched the same shell, timed out on the hydration promise, and indexed what was effectively an empty body - reducing crawl value on affected URLs and contributing to the 0.37% drop in organic clicks.

Reproduction Vector

The Code That Caused the Bug

useEffect(() => {
  import(`./widgets/${featureFlag ? 'new' : 'old'}.js`)
    .then(mod => mod.init())
    .catch(() => {});
}, [featureFlag]);

Googlebot & User Perspectives

Metric Regressions

Field Debugging Checklist

Run these checks when you suspect lab/field divergence:

• Compare synthetic Lighthouse lab scores against CrUX and RUM field metrics for the same URL and release window.
• Curl the page and verify Content-Length, body size, and presence of expected article text in the initial HTML.
• Check SRI hashes and chunk availability on the CDN during the deploy window.
• Run a headless render with Puppeteer and record DOM node count plus innerText length for the main article container.
• Audit deploy artifacts and confirm every chunk referenced by index.html exists on the CDN at the exact SHA.

Detection Signals We Saw

Copyable CLI Checks

Run these immediately when investigating:

curl -sI https://www.example.com/article/123 | grep -E '^Content-Length|^ETag|^Cache-Control'

curl -s https://www.example.com/article/123 | wc -c

Quick Test: What Do Bots Actually See?

~30 seconds

Most people guess. Don't.

Run this test and look at the actual response your site returns to bots.

1

Fetch your page as Googlebot

Use your terminal:

curl -A "Googlebot" https://yourdomain.com

Look for:

• Real visible text (not just <div id="root">)
• Meaningful content in the HTML
• Page size (should not be tiny)

2

Compare bot vs browser

Now test what a real browser gets:

curl -A "Mozilla/5.0" https://yourdomain.com

If these responses are different, Google is indexing a different page than your users see.

Stop guessing — measure it.

Real example: 253 words vs 13,547

We see this constantly. Here's a real example from production: Googlebot saw 253 words and 2 KB of HTML. A browser saw 13,547 words and 77.5 KB. Same URL — completely different content.

If your HTML doesn't contain the content, Google doesn't either.

Compare Googlebot vs browser on your site → HTTP Debug Tool

3

Check for common failure signals

We see this all the time in production:

• HTML under ~1KB → usually empty shell
• Visible text under ~200 characters → thin or missing content
• Missing <title> or <h1> → weak or broken page
• Large difference between bot vs browser HTML → rendering issue

Use the DataJelly Visibility Test (Recommended)

You can run this without touching curl. It shows you:

• Raw HTML returned to bots (Googlebot, Bing, GPTBot, etc.)
• Fully rendered browser version
• Side-by-side differences in word count, HTML size, links, and content

Run Visibility Test — Free

What this test tells you (no guessing)

After running this, you'll know:

• Whether your HTML is actually indexable
• Whether bots are seeing partial content
• Whether rendering is breaking in production

This is the difference between "I think SEO is set up" and "I know what Google is indexing."

If you don't understand why this happens, read: Why Google Can't See Your SPA

If this test fails

You have three real options:

SSR

Works if you can keep it stable in production

Prerendering

Breaks with dynamic content and scale

Edge Rendering

Reflects real production output without app changes

If you do nothing, you will not rank consistently. Learn how Edge Rendering works →

This issue doesn't show up in Lighthouse. It shows up in rankings.

Run the TestAsk a Question

Immediate Remediation Steps

What to do in the next 24-72 hours:

Long-term Hardening

FAQ