Skip to main content

3 posts tagged with "testing"

View All Tags

Vojtech Miksu

If you use stories to develop your React components, you might be also interested in some sort of test automation. Ladle and Playwright makes it easy to take screenshots of your stories and compare them against the previous version before you changed your code. We call this visual snapshot testing. Let's take a look at how you can automate it with Ladle. This solution is quick, free and self-hosted.

It might surprise you how easy it is - less than 10 lines of code! Ladle exports a static meta.json file that lists all your stories and their parameters. We can use this file to generate our tests. Note: Terms snapshots and screenshots are used interchangeably.

You can jump into the working example right away. Or you can follow the steps below.

The Workflow

  1. Build Ladle using your stories
  2. Start an HTTP server for Ladle
  3. Fetch the meta.json file
  4. Dynamically generate a test for each story
  5. Take a screenshot & compare it with the baseline
  6. Commit changes if there are any
  7. Profit

Playwright

Playwright is a great headless browser testing framework. It works cross-platform and cross-browser and has a nice TypeScript API. It also comes with a test runner. We use it to navigate through Ladle and capture screenshots. It can be installed as:

pnpm install @playwright/test

We also need some other dependencies:

pnpm install start-server-and-test sync-fetch

Stories

Let's create a story file src/abc.stories.tsx so we have something to test. Presumably, you already have stories with your own React components.

export const First = () => {
return <h1>First</h1>;
};

export const Second = () => {
return <h1>Second</h1>;
};
Second.meta = {
skip: true,
};

Test Setup

Now we need to tell Playwright what & how to test. We can create a single test file that dynamically creates subtests for each individual story. The following code in tests/snapshot.spec.ts is the secret sauce of this setup:

import { test, expect } from "@playwright/test";
// we can't create tests asynchronously, thus using the sync-fetch lib
import fetch from "sync-fetch";

// URL where Ladle is served
const url = "http://localhost:61000";

// fetch Ladle's meta file
// https://ladle.dev/docs/meta
const stories = fetch(`${url}/meta.json`).json().stories;

// iterate through stories
Object.keys(stories).forEach((storyKey) => {
// create a test for each story
test(`${storyKey} - compare snapshots`, async ({ page }) => {
// skip stories with `meta.skip` set to true
test.skip(stories[storyKey].meta.skip, "meta.skip is true");
// navigate to the story
await page.goto(`${url}/?story=${storyKey}&mode=preview`);
// stories are code-splitted, wait for them to be loaded
await page.waitForSelector("[data-storyloaded]");
// take a screenshot and compare it with the baseline
await expect(page).toHaveScreenshot(`${storyKey}.png`);
});
});

Run It

Our setup is ready, we just need to run it. Let's add some package.json scripts that we can use as shortcuts:

{
"scripts": {
"serve": "ladle serve",
"build": "ladle build && ladle preview -p 61000",
"test:dev": "start-server-and-test serve 61000 'pnpm playwright test'",
"test": "start-server-and-test build 61000 'pnpm playwright test'",
"test:update": "start-server-and-test build 61000 'pnpm playwright test -u'"
}
}

These should be self-explanatory. If you use yarn or npm you might want to use npx instead. We use the start-server-and-test library to make sure that the Ladle server is running before starting the tests.

The first time you run the test script it will error out since it needs to create the baseline screenshots in tests/snapshot.spec.ts-snapshots folder. The second run will succeed:

Running 2 tests using 1 worker

✓ tests/snapshot.spec.ts:23:3 › abc--first - compare snapshots (259ms)
- tests/snapshot.spec.ts:23:3 › abc--second - compare snapshots


1 skipped
1 passed (952ms)

You can keep adding more stories and they get automatically covered by visual snapshots. If you change an existing story, the test will fail. The playwright also outputs a diff image that highlights the differences between the baseline and actual screenshot. If everything seems ok, you need to run test:update to update existing snapshots. When your code matches the desirable looks you should commit both the code and snapshot changes. This updates the baseline for the future code changes.

Possible Improvements

The first problem you might run into is that your CI (tried Github Actions yet?) might render fonts differently than your local machine. The easiest way to make sure you use the same environment locally and remotely is to use docker.

Our example setup uses a made up meta.skip parameter to opt-out of snapshots for selected stories. You could add more customization like viewport size or browser by adding arbitrary meta parameters to the story and updating the tests/snapshot.spec.ts logic accordingly.

Playwright can do a lot more than just screenshot taking. You can also use it to write all other sorts of assertion/integration tests since you already have the environment ready.

You might reach some point when storing screenshots in your repository is not working anymore due to the size - imagine hundreds of developers using a single monorepo churning thousands of images. At Uber, we've built an internal service that uses S3 to store screenshots, Aurora DB to store test metadata and provides a custom UI to compare & approve changes (GitHub's PR review UI is pretty good for images too!). The goal was to move the snapshot creation and updates into a remote environment so developers are not slowed down by it. It also helps to prevent the issue of local vs remote environment.

Conclusion

We used Playwright and wrote 10 lines of code to automate visual snapshots for our stories and gained great coverage for our React components! Ladle was built for this - to be small, fast and integrate well with other tools. This should give you a good starting point for your own automated and end-to-end testing.

Vojtech Miksu

Ladle has been out for 3 months and the community feedback was overwhelming and amazing. Thank you! Some numbers:

  • 🎯 20,000 unique visitors on this website.
  • ⭐ 1,300+ GitHub stars.
  • 🖊️ 10 contributors.
  • 💬 100+ folks joined our Discord community.

Today, it's time to graduate Ladle to its first major stable version 1. We've fixed many bugs discovered by early adopters and added some big features as well:

V1 brings the most requested feature - the full access to Vite configuration. When Ladle was released, it was intended as a tool that could replace Storybook. The bundler (Vite) was just an implementation detail and completely hidden away. However, for more advanced setups, you need to customize various aspects of compilation (like Storybook's Webpack). It makes sense to fully expose vite.config.js and embrace Vite first applications as well. Ladle's configuration has been rewritten from the ground up.

Check the new config documentation.

Breaking Changes

  • Ladle now loads top-level vite.config.js/ts/mjs and uses all its options.
  • All Vite related options from config.mjs are removed and an error will be thrown, use vite.config.js instead.
  • enableFlow option removed, you can create your own plugin (check our e2e/flow test).
  • Programmatic API imports changed to @ladle/react/serve and @ladle/react/build.
  • --out renamed to --outDir to mimic Vite configuration, added -o alias, outDir moved to the top-level in config.mjs.
  • --port has an alias -p, port moved to the top-level in config.mjs.
  • vite.config.js can be customized through viteConfig and --viteConfig.
  • --base-url removed, use base in vite.config.js.
  • --open removed, use server.open in vite.config.js.
  • --sourcemap removed, use build.sourcemap in vite.config.js.

This makes our internal setup much simpler since we don't need to pass through individual Vite settings - you can now customize all of them without Ladle being in your way.

Next big features?

We are adding MDX support and making it easier to use Ladle for documentation.

Vojtech Miksu

Ladle is a tool for developing and testing your React components in an environment that's isolated and faster than most real-world applications. It supports Component Story Format – a concept widely popular thanks to Storybook. In fact, Ladle has been developed as a drop-in replacement of Storybook – it should already work with your existing stories.

Ladel Demo

Storybook ❤️

At Uber, we are big fans of Storybook. We have over 100 instances of Storybook in our web monorepo. Our teams use it to develop, showcase, document and test React components. We have developed a CI based system that automatically deploys each Storybook with every change and runs automated visual snapshot tests. This happens a thousand times each day. It's a critical tool for our web workflows. The performance is extremely important.

Unfortunately, there are some areas where Storybook is not doing as good as we would like to:

  • production build time - often times it's the slowest part of our CI
  • dev mode start up time - not always faster than the related prod app
  • bundle output - hosting storage + slow initial time to interactive
  • maintenance - we repackaged Storybook, its dependencies and configuration to provide a seamless setup for our developers; however, the addon versioning and monorepo setup makes maintenance difficult
  • testing - for our automated visual testing, we need to crawl and parse stories in a separate process since Storybook doesn't export a static list of stories (and other metadata)

Next Generation Frontend Tooling ⚡

A shift is happening. ES modules are now natively supported by all browsers and Node. We don't need to bundle our components anymore in order to serve them. This is a major slowdown for Storybook and other Webpack based environments.

Ladle is built around Vite, which serves modules directly to the browser and uses fast esbuild when bundling is required for dependencies. It provides the performance leap we were looking for.

Performance Numbers ⏱️

We used Base Web to benchmark Ladle and latest Storybook v6.4.19. Base Web is a complex component library and has about 350 stories. The Storybook uses the default bootstrapped settings. The test is made on a 2018 Macbook Pro, i7 2.7 GHz. The time is measured in seconds and less is better.

BaseWeb stories compilation time

Both Ladle and Storybook utilize cache. The initial dev startup takes 6s vs 58s. Once the cache is built, Ladle starts almost instantly (3s is just the browser tab being opened + time to interactive). Storybook always takes about 25s to start. The production build is about 4x faster with Ladle.

There is another improvement - hot reload takes less than 100ms with Ladle and preserves the state. Storybook takes about 2.5s and doesn't preserve the state.

We also care about the bundle size and the amount of resources that browser has to initially download:

  • Ladle build folder is 4.6 MB vs Storybook's 16.1 MB
  • Ladle downloads 388 kB of resources to open the default story, Storybook 14.3 MB

How is this even possible? Ladle code-splits each story by default, so it doesn't matter how many stories you have. It always keeps the initial bundle reasonable.

Not Just Fast 📝

Ladle is a single package and command. It's easy to install and setup - no configuration required. Some other features:

  • Supports controls, links, dark theme, RTL and preview mode
  • Built-in addons always preserve the state through the URL - useful for testing
  • A11y and keyboard friendly
  • Responsive, no iframes
  • Code-splitting and React Fast Refresh enabled by default
  • Exports meta.json file with list of stories and additional metadata
  • Programmatic API so it can be easily repackaged

Conclusion 🔮

The new set of web tools is coming and brings huge performance wins. Ladle is using them to provide a significantly faster environment for developing, sharing and testing your React components. Are you ready to give it a try? Also, check our GitHub and give us ⭐.

mkdir my-ladle
cd my-ladle
pnpm init
pnpm add @ladle/react react react-dom
mkdir src
echo "export const World = () => <p>Hey</p>;" > src/hello.stories.tsx
pnpm ladle serve

Do you want to work on Ladle and other web tooling? Our team at Uber is hiring!