Snapshot testing

What is Snapshot Testing?

Snapshot testing is a type of “output comparison” or “golden master” testing. These tests prevent regressions by comparing the current characteristics of an application or component with stored “good” values for those characteristics. Snapshot tests are fundamentally different from unit and functional tests. While these types of tests make assertions about the correct behavior of an application, snapshot tests only assert that the output now is the same as the output before; it says nothing about whether the behavior was correct in either case.

The test doesn’t say whether the old or new stylings were correct, it just highlights to the tester that something has changed.

A variety of characteristics can be measured, but front-end tests typically focus on two:

• data (serializable JavaScript values)

• images.

How Snapshot Testing Works

Snapshot tests work by recording some characteristic of a system (e.g., taking a snapshot), and then later comparing that stored snapshot to the current value of the characteristic. For Jest-style tests, that characteristic is typically a serialized render tree. The first time a test is run, the toMatchSnapshot expectation saves the data it receives to a file. During later test runs, the current value of elem is compared to the stored “good” one. The test fails if the two values are different. Updating snapshots is very easy, making them very maintainable, but this can be both a blessing and a curse.

The Benefits of Snapshot Testing

Snapshot testing is the cheapest and fastest way to increase code coverage. It's a great way to check that your application’s behavior isn’t changing unexpectedly during development. (prevents regressions in the app)

Snapshot tests can help out quite a bit in that situation because they’re typically much shorter and easier to write than traditional unit tests.

Snapshot tests are also easy to keep up to date as developers generally just need to run a single command to get the testing system to record new snapshots. This is certainly much easier than needing to edit many test files to bring tests back in sync with reality.

The Drawbacks of Snapshot Testing

A significant disadvantage is that they’re tightly coupled to an application’s output, making them very fragile. Any changes, even to insignificant parts of the output, can cause snapshot tests to fail. Developers then must (or at least should) manually verify that everything is still working properly and update the snapshots.

They are generated files, and developers tend to be undisciplined about scrutinizing generated files before committing them, if not at first then definitely over time. Most developers, upon seeing a snapshot test fail, will sooner just nuke the snapshot and record a fresh passing one instead of agonizing over what broke it.

This leads to another potential problem with snapshot tests: they don’t actually indicate anything about the expected output, just that it shouldn’t change. Unlike unit and functional tests, snapshot tests don’t contain focused, meaningful assertions or expectations. A developer who has to manually verify that the output is still “good” may run into trouble when the failing test is for a part of the app he or she isn’t familiar with, because a snapshot test doesn’t indicate what parts of the output are important.

Snapshot tests aren’t inherently well suited to dynamic content. A “random quote of the day” component will frequently fail snapshot tests since the random quote in the output usually won’t match the stored component data. Tools can deal with this problem by letting users mark areas with dynamic content.

Another potential drawback to snapshot tests is storage requirements. Snapshots must be stored somewhere; typically they’re checked into the project repository. Depending on what characteristic is being recorded, snapshots can be quite large. Render trees are text-based, and generally diff and compress well, but an extensive suite of screenshot-based snapshot tests can easily consume tens of megabytes of space. This is another area where cloud-based services can provide some assistance by storing snapshots for visual tests.