Visual Regression Testing for Lazy-Loaded Content

To screenshot observer-driven pages reliably, either drive the page so every observer fires (scroll through it, then wait for images, fonts and animations to settle) or put the page in a test mode that loads eagerly and disables motion — and never take a full-page screenshot straight after navigation.

Problem / Scenario Context

A marketing site adds visual regression tests with Playwright's full-page screenshots. Every run differs: below-the-fold images are placeholders in some runs and loaded in others, reveal animations are captured half-faded, and an infinite list has a different number of items depending on how quickly the screenshot scrolled. The team raises the diff threshold until the tests pass — at which point they no longer catch real regressions.

Observer-driven content is non-deterministic in screenshots because a full-page capture does not scroll the page the way a user does. The Testing Observers in JSDOM & Real Browsers topic covers functional tests; this page covers pixels.

Mechanics Explanation

A full-page screenshot is taken by resizing the capture area to the document height (or by stitching viewport-sized captures, depending on the tool and browser). In both cases, the page is not scrolled in the normal way:

• Lazy images whose observers use the implicit root may or may not see themselves as intersecting, depending on whether the tool enlarges the viewport. Native loading="lazy" images may not start loading at all.
• Reveal animations start when observers fire and take hundreds of milliseconds; a capture moments later sees them mid-transition.
• Infinite scroll sentinels may fire once, several times, or not at all depending on the effective viewport size.
• Network timing decides whether an image that started loading has finished.

So the captured state is a race between the tool, the observers and the network. The fix is to remove the race: either make everything happen and wait for it, or stop it from being conditional.

Comparison Table: Sources of Flakiness and Remedies

Minimal Reproducible Example

import { test, expect } from '@playwright/test';

test('home page', async ({ page }) => {
  await page.goto('/');
  await expect(page).toHaveScreenshot({ fullPage: true });   // races lazy loading and reveals
});

Run it five times: the diffs show different images loaded and reveals at different opacities.

Production-Safe Solution

Provide both strategies: a helper that drives the page for tests that must exercise observers, and a test-mode flag the site honours for pure layout screenshots.

// screenshot-helpers.ts
import type { Page } from '@playwright/test';

/** Scroll through the page so every observer fires, then wait for everything to settle. */
export async function settleByScrolling(page: Page): Promise<void> {
  await page.evaluate(async () => {
    const step = innerHeight * 0.8;
    for (let y = 0; y < document.documentElement.scrollHeight; y += step) {
      scrollTo(0, y);
      await new Promise((r) => requestAnimationFrame(() => requestAnimationFrame(r)));
    }
    scrollTo(0, 0);
  });
  await page.waitForLoadState('networkidle');
  await page.evaluate(async () => {
    await document.fonts.ready;
    await Promise.all([...document.images].map((img) => img.decode().catch(() => {})));
    await Promise.all(document.getAnimations().map((a) => a.finished.catch(() => {})));
  });
}

// In the site: honour a test-mode flag before any observer is created.
const testMode = new URLSearchParams(location.search).has('visual-test');
if (testMode) document.documentElement.classList.add('visual-test');

export function lazyLoad(img: HTMLImageElement): void {
  if (testMode) { img.src = img.dataset.src!; return; }       // eager in test mode
  lazyObserver.observe(img);
}

declare const lazyObserver: IntersectionObserver;

.visual-test *, .visual-test *::before, .visual-test *::after {
  transition: none !important;
  animation: none !important;
}
.visual-test .reveal { opacity: 1 !important; transform: none !important; }

// The tests
test('home page layout (test mode)', async ({ page }) => {
  await page.goto('/?visual-test');
  await page.evaluate(() => document.fonts.ready);
  await expect(page).toHaveScreenshot({ fullPage: true });
});

test('home page after real scrolling', async ({ page }) => {
  await page.emulateMedia({ reducedMotion: 'reduce' });
  await page.goto('/');
  await settleByScrolling(page);
  await expect(page).toHaveScreenshot({ fullPage: true, maxDiffPixelRatio: 0.001 });
});

Emulating reduced motion in the driven test is a useful shortcut if the site honours it correctly — reveals then appear without movement, so there is nothing to wait for — and it doubles as a check that the reduced-motion path renders complete content, as covered in respecting prefers-reduced-motion in scroll reveals.

Bounding Infinite Content

Infinite lists are the hardest case, because "scroll to the bottom" never ends. Bound them explicitly:

• Cap pages in test mode. A query flag that limits the feed to one or two pages gives a fixed height.
• Stop at a known item count when driving: scroll until list.children.length >= 40, then stop and capture only the list's container with an element screenshot rather than the full page.
• Stub the data. Serve the list's API from a fixture with Playwright's page.route, so every run has identical items.

await page.route('**/api/feed*', (route) => route.fulfill({ path: 'fixtures/feed-page-1.json' }));

Combining a stubbed API with a page cap makes infinite scroll as deterministic as a static page. The underlying mechanics are in infinite scroll & pagination.

Verification Steps

• Run each visual test ten times locally with --repeat-each=10 and confirm zero diffs.
• Break a lazy image path on purpose and confirm the driven test catches it while test mode does not — then decide which tests need which mode.
• Run under CPU throttling in CI to shake out remaining timing assumptions.
• Check that test mode is inert in production — the flag must only change behaviour when present.
• Review diff thresholds; with deterministic pages they can be near zero.

Common Mistakes to Avoid

• Raising the diff threshold to hide flakiness. It stops catching real regressions.
• Full-page screenshots straight after goto. Observers have not fired yet.
• Waiting with fixed sleeps. They are slow and still race on slow CI machines.
• Letting test mode leak into production behaviour. Gate it on an explicit flag.

FAQ

Related

• Debugging Flaky Observer Tests in CI — timing issues beyond screenshots
• Testing ResizeObserver Callbacks with Playwright — browser tests for size logic
• Building a Lazy Image Loader with IntersectionObserver — the loader under test

↑ Back to Testing Observers in JSDOM & Real Browsers