Shipping visual regression on React Native: Maestro baselines, an RNTL v14 migration, and killing Storybook

I set out to close two gaps in the test suite of a React Native/Expo logistics app (no visual regression, no CI integration), and the path I ended up on deleted the component catalog entirely.

Testing constraints

The app’s testing surfaces were inconsistent. Storybook React Native 10.4 held 65 stories but ran inside the debug dev client as the /storybook Expo route, on the same emulator as the app, not in a browser. That gives up web Storybook’s main advantage (iteration speed in a browser). It was also excluded from tsc and ran in no CI job, so it rotted silently whenever anyone touched a component.

Separately, every test run printed a React 19 deprecation warning: react-test-renderer was being phased out, and it sat in the tree as a transitive peer of React Native Testing Library v13.

Two gaps cut across both Storybook and Maestro at that point:

• No visual regression: Maestro’s takeScreenshot captured PNGs but never diffed them.
• No CI: neither surface ran in a pipeline; everything was a local lane.

Decision and migration

The decision record

The RNTL v14 migration the codemod couldn’t finish

The pre-migration audit called this “dependency-only”. That was incorrect. RNTL v14 makes render, fireEvent, act, unmount, and rerender async by default (they return Promises under React 19’s async rendering model), so every call site needs await. This is the trap I most want on record: an audit can confirm the API surface has no removed methods and still miss that the entire surface moved to async.

The dependency swap itself was mechanical, via the official rntl-v14-update-deps codemod:

pnpm dlx codemod@latest rntl-v14-update-deps --target ./app
pnpm install

The async-API migration used the companion rntl-v14-async-functions codemod:

pnpm dlx codemod@latest rntl-v14-async-functions \
  --target ./app --no-interactive --allow-dirty

That made 44 test callbacks async and awaited direct render / fireEvent / act calls. But it leaves custom render helpers untouched: local wrappers like renderWithSafeArea, renderRoute, renderWithQueryClient that call render() internally and return queries. The codemod can’t know they’re now async, so their call sites never get awaited.

A one-off ts-morph transform closed the gap. Run from an isolated temp project so nothing leaked into repo devDependencies: the transform rewrote every helper to async + await render(...), and gave every call site an async owning callback plus await (parenthesized when chained: (await renderWithSafeArea(<X />)).getByText(...)). The transform also cleaned up four await await artifacts the codemod had produced.

Four cases the codemod cannot handle, each worth documenting because they will bite anyone doing the same migration:

1. .toJSON() precedence. Member access binds tighter than await, so the codemod’s output parsed incorrectly and called .toJSON() on the Promise:

// Before
const tree = render(<X />).toJSON();

// Codemod produced (broken): awaits .toJSON() on a Promise
const tree = await render(<X />).toJSON();

// Fixed: parenthesize so await binds the Promise
const tree = (await render(<X />)).toJSON();

2. act() arrow callbacks. The codemod added await inside the callback but never made the arrow itself async, so the file failed to parse:

// Codemod produced — syntax error: "Unexpected reserved word 'await'"
act(() => {
  await fireEvent.press(button);
});

// Fixed
act(async () => {
  await fireEvent.press(button);
});

3. ReturnType<typeof render> type drift. In v13 this resolves to RenderResult; in v14 it resolves to Promise<RenderResult>. Import RenderResult and annotate directly:

// Before
let view: ReturnType<typeof render>;

// Fixed
import { RenderResult } from "@testing-library/react-native";
let view: RenderResult;

4. await await artifacts, which the ts-morph transform cleaned up where the codemod had double-wrapped an already-awaited expression.

// Codemod produced (redundant): awaits an already-awaited expression
await await fireEvent.press(button);

// Fixed
await fireEvent.press(button);

No application code changed. Only tests, package.json, and the lockfile.

Maestro assertScreenshot visual regression

Two mechanics I had to reverse-engineer from the Maestro docs because they aren’t called out prominently:

Path resolution. For path: X, Maestro searches, in order: (1) <flow-dir>/X, (2) .maestro/results/screenshots/X, (3) <app-root>/X. A plain baselines/<name>.png therefore resolves into either the flow’s directory or the app root, not .maestro/baselines/. The fix is to be flow-relative: path: ../baselines/<name>.png (flows live in .maestro/seeded/) lands the baseline in .maestro/baselines/ where you want it.

cropOn requires a pre-cropped baseline. This is the one I got wrong on the first pass. cropOn narrows the live screenshot to an element, and the docs require the reference baseline to be cropped the same way. A full-screen baseline will never match a cropped assertion. So seed the baseline with a takeScreenshot using the same cropOn:

# Seed step — same cropOn the assertion will use
- takeScreenshot:
    path: home-pending-inbox-cta # no extension; saves as .png
    cropOn:
      id: home-pending-inbox-cta

Then replace it with the assertion once the baseline is committed:

- assertScreenshot:
    path: ../baselines/home-pending-inbox-cta.png
    cropOn:
      id: home-pending-inbox-cta
    thresholdPercentage: 95 # use 98-99 for maps / polylines

The per-screen TDD cycle:

1. RED: drop the assertion in with no baseline. Maestro fails and prints the three directories it searched. There is no auto-generation.

2. Seed: add the takeScreenshot with the matching cropOn, run the flow, copy the cropped PNG from .maestro/captures/latest/ into .maestro/baselines/, git add it.

3. GREEN: replace the seed step with the assertion, rerun, confirm pass.

4. Negative test: perturb the baseline symmetrically with macOS sips so no app rebuild is needed:

   cp home-pending-inbox-cta.png home-pending-inbox-cta.png.real
   sips -f horizontal home-pending-inbox-cta.png
   # rerun the flow → must FAIL with "threshold not met" and emit a _diff.png
   mv home-pending-inbox-cta.png.real home-pending-inbox-cta.png
   # rerun → green

5. .gitignore *_diff.png so failed-run diffs never land in the repo.

Anti-flake rules I learned the hard way:

• cropOn a stable container testID to exclude the status bar, timestamps, GPS coordinates, and live map tiles.
• Keep platform.android.disableAnimations: true in config.yaml.
• Never baseline a live clock or live map tiles without cropping them out.

One screen (a live map showing route geometry) I’d flagged as “inherently flaky, lowest priority”. I measured it before skipping: across consecutive runs it came back ≥99.5% similar, because the emulator GPS is fixed, map tiles are cached, and the route geometry is deterministic. Committed at a 98% threshold; a horizontal flip drops it to ~85.7%, well below.

Full Storybook removal

With edge states now in behavioral RNTL tests, happy paths in Maestro flows, and visual regression on Maestro baselines, the “nothing of value lost” check cleared. I removed Storybook in full: 77 files deleted, 88 packages removed, 3 added. The catalog, the env-gate machinery, the Metro withStorybook wrapper, the tsc include/exclude entries, the npm scripts, the lane guards, all of it. Nothing in the replacement contract is pixel-based; every behavior the stories used to assert has a deterministic test behind it.

The flake RCA: “12/16 in batch, 16/16 individually”

A recurring pattern in the seeded batch run: 12 of 16 flows passed together, but all 16 passed individually. The first diagnosis was “stale local-dev build”. That was a symptom, not the cause.

The actual root cause was a commit that had split login.yaml into two mutually-exclusive branches using when: visible: one for local-dev mode, one for credentials. Maestro evaluates when: visible exactly once with a short default timeout. In a fixed-order batch run, the flow preceding login leaves the emulator in a heavy state (GPS driving mode, deep chat), so when the runner does a cold re-launch after clearState, the sign-in form paints after the probe window closes. Both branches skip, no login happens, and the post-login extendedWaitUntil times out at 30s. Run the same flow individually and the form paints in time. The failure set is deterministic given the fixed order.

The fix branches on an explicit env var instead of probing the screen, forces a true cold start, and waits for the email input before branching:

- killApp # true cold start before launch
- launchApp
- extendedWaitUntil:
    visible: "Sign-in email input"
    timeout: 20000

# Branch on an env var, not on a screen probe
- runFlow:
    when:
      true: ${MAESTRO_AUTH_MODE == "local"}
    commands:
      - tapOn: "Sign in (local dev)"

- runFlow:
    when:
      true: ${MAESTRO_AUTH_MODE == "credentials"}
    commands:
      - tapOn: "Email"
      - inputText: ${AUTH_EMAIL}
      # ...credentials path

The lane scripts export MAESTRO_AUTH_MODE (local for local and seeded lanes, credentials for QA). No screen probing, no race.

Verification

• 117 suites / 534 RNTL tests green.
• 4 visual-regression baselines committed, all negative-checked (one map screen measured ≥99.5% stable across runs, committed at 98% threshold).
• −88 packages from the Storybook removal; +3 from the RNTL v14 swap.
• Login flake: 12/16 → 16/16, verified across 3 consecutive seeded batch runs (one rebuild run + two no-rebuild reruns).
• Zero application code changed across the whole engagement: only tests, baselines, and dependencies.

Tradeoffs

Two things. First, trust the official migration audit less on async-by-default changes. The RNTL v14 audit correctly noted no removed APIs were in use and concluded “dependency-only”, and missed that the entire call surface moved to Promises. The cheap insurance is to run a single test before estimating: if await render(...) is required, the audit is incorrect and the migration is async across the whole suite.

Second, measure flake risk empirically before labeling a screen “inherently flaky.” The live map screen sat at the bottom of my priority list for weeks on the assumption that GPS and map tiles would never be stable. Consecutive-run similarity came back ≥99.5%. I lost time I didn’t need to.

References

• React Native Testing Library v14 migration guide: the async-by-default change and the official codemods.
• rntl-v14-async-functions and rntl-v14-update-deps codemods: what they handle and, just as importantly, what they leave to you.
• Maestro assertScreenshot docs: cropOn and thresholdPercentage.
• test-renderer package: the maintained replacement for react-test-renderer that RNTL v14 ships with.
• jest-expo preset: the official Expo/RN test setup, and the reason vitest wasn’t adopted.
• sips (macOS): for the negative-test baseline perturbation without an app rebuild.