Fix the Flaky Test

The concept

Most “only fails in CI” bugs are not CI bugs. Your laptop is quietly giving the test something for free: a fast CPU, a warm cache, nothing else running. The shared runner does not, so every step the page takes is a little slower, and by a different amount each run.

In Playwright, actions such as click() and fill() wait on their own: they retry until the element is visible, enabled and stable. Web-first assertions such as expect(locator).toBeVisible() retry too. Methods that return a value straight away do not wait. isVisible(), isEnabled(), textContent() and count() answer once, at the moment they are called.

Wrap one of those snapshot checks in an if and you get the worst kind of flaky test. When the page is quick the check is true and the test passes. When the page is slow the check is false, the if is skipped without an error, and the test fails several lines later, far from the real cause.

A fixed wait such as waitForTimeout(3000) hides the problem rather than fixing it. It is too short on a bad day and wastes time on every good one. In this challenge the hidden validator also checks how soon you moved on after the page was ready.

The fragile version

await page.getByRole('button', { name: 'Place order' }).click();
if (await page.getByText('Payment accepted').isVisible()) {
  await page.getByRole('link', { name: 'View receipt' }).click();
}

Why it breaks: isVisible() checks once, straight after the click. On a slow runner payment has not finished yet, the if is skipped silently, and the test fails later when the receipt is missing.

A robust Playwright version

await page.getByRole('button', { name: 'Place order' }).click();
await expect(page.getByText('Payment accepted')).toBeVisible();
await page.getByRole('link', { name: 'View receipt' }).click();

The assertion retries until payment is accepted, so the test waits exactly as long as the page needs and fails with a clear message if it never happens.

If a later action already depends on the page being ready, that action’s own auto-wait may be all you need.

This is a general example on a made-up page, not the challenge’s reference solution.

Common mistakes

• Raising timeouts or adding waitForTimeout() until the build goes green.
• Using isVisible() or count() as a wait. They are snapshots; use an action or an expect instead.
• Turning on retries to hide the flake instead of finding it. Retries are a safety net, not a fix.
• Reading only the last error. A skipped step usually fails much later than the line that caused it.

FAQ