Preventing Duplicate Fetches in Observer-Driven Pagination

A sentinel is not a button. It reports its position repeatedly, and a fetch started from it will be started again unless something explicitly says otherwise. That something has to be request state, not an assumption about how often the callback runs.

Problem / Scenario Context

An infinite list appends twenty rows per page. Testing on a trackpad it works; on a mouse wheel, or on a phone with a fast flick, the network panel shows page 2 requested three times and page 3 skipped, and the list ends up with duplicated rows and a gap.

The sentinel fired three times in quick succession — once as it entered the rootMargin band, once as it fully entered, once after the previous append briefly moved it — and each call started a request with the same page value, because the counter is only incremented when a response arrives.

The instinct is to debounce the callback. That reduces the frequency without removing the race: two calls 300 ms apart still both start a fetch if the first has not resolved.

Mechanics Explanation

Three separate things must be true before a page request is legitimate, and each corresponds to a distinct piece of state:

• No request is in flight. A boolean, set synchronously before the fetch begins.
• There are more pages. A flag set when a response returns fewer rows than the page size.
• The last attempt did not fail unrecovered. An error state that requires an explicit retry rather than silently re-firing on the next scroll.

The critical word is synchronously. Setting the flag inside a promise callback, or in React state, leaves a window in which a second callback sees the old value. Since observer callbacks for a burst are delivered in the same task, that window is exactly where the duplicates come from.

Incrementing the page counter before the request rather than after removes the second half of the problem: even if two requests do slip through, they ask for different pages.

Comparison Table: Guard Strategies

Minimal Reproducible Example

let page = 1;
let loading = false;

const io = new IntersectionObserver(async ([entry]) => {
  if (!entry.isIntersecting) return;
  if (loading) return;                       // read here…
  const data = await fetch(`/items?page=${page + 1}`).then((r) => r.json());
  loading = true;                            // …set here: three callbacks all passed the check
  page += 1;
  append(data);
  loading = false;
});

Production-Safe Solution

type Status = 'idle' | 'loading' | 'error' | 'exhausted';

const PAGE_SIZE = 20;
let status: Status = 'idle';
let nextPage = 1;
const inFlight = new Set<number>();
let controller: AbortController | null = null;

async function loadNextPage(): Promise<void> {
  if (status !== 'idle') return;             // synchronous guard, first line
  const page = nextPage;
  if (inFlight.has(page)) return;

  status = 'loading';
  inFlight.add(page);
  nextPage += 1;                             // claim the page BEFORE awaiting anything
  controller = new AbortController();

  try {
    const res = await fetch(`/items?page=${page}`, { signal: controller.signal });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const rows: unknown[] = await res.json();

    append(rows);
    status = rows.length < PAGE_SIZE ? 'exhausted' : 'idle';
    if (status === 'exhausted') observer.unobserve(sentinel);   // nothing left to ask for
  } catch (err) {
    if ((err as Error).name === 'AbortError') { status = 'idle'; nextPage = page; return; }
    status = 'error';
    nextPage = page;                         // give the page number back for an explicit retry
    showRetryButton();
  } finally {
    inFlight.delete(page);
    controller = null;
  }
}

const observer = new IntersectionObserver((entries) => {
  for (const entry of entries) if (entry.isIntersecting) void loadNextPage();
}, { rootMargin: '400px 0px' });

observer.observe(sentinel);

// teardown: abort anything in flight so a resolved response cannot append to a dead list
export function dispose(): void {
  controller?.abort();
  observer.disconnect();
}

Four decisions in that code are worth naming.

status !== 'idle' as the first line. It covers loading, error and exhausted in one check, so no code path can start a request from a state that should not.

Claiming nextPage before the await. Even if a second call somehow passes the guard, it asks for a different page — the duplicate becomes a gap-free over-fetch rather than corrupted data.

Giving the page number back on failure. Without it, a failed request permanently skips a page, which shows up as missing rows nobody can explain.

Aborting on teardown. A response that resolves after the component is gone will otherwise append to a detached list, retaining it — the retention path that a WeakMap cannot help with.

Under React Strict Mode the effect runs twice in development, so loadNextPage is called twice. The synchronous guard handles it correctly — the second call returns immediately — which is the same property that makes it correct in production, as described in fixing doubled observer callbacks.

Verification Steps

• Throttle the network to Slow 3G and flick-scroll: exactly one request per page should appear.
• Check page numbers are contiguous in the network panel — a gap means the error path is not returning the page number.
• Force a failure with an offline toggle and confirm the list stops rather than re-firing on every subsequent scroll.
• Unmount mid-request and confirm the request is aborted rather than resolving into a detached list.
• Run in Strict Mode and confirm the doubled call produces one request.

Common Mistakes to Avoid

• Debouncing instead of guarding. Reduces the frequency of the race without removing it.
• Setting the flag after await. Leaves the exact window the burst arrives in.
• Incrementing the page after the response. Means concurrent requests ask for the same page.
• Retrying automatically on error. Turns a failing endpoint into a request loop driven by scrolling.
• Leaving the sentinel observed after exhaustion. Harmless but wasteful; unobserve costs one line.

FAQ

Related

• Infinite Scroll & Pagination — the guard state machine in its wider context
• Creating Smooth Infinite Scroll Without Jank — what to do with the rows once they arrive
• Restoring Scroll Position in Infinite Scroll — why the sentinel must be re-observed last on a restore

↑ Back to Infinite Scroll & Pagination