by Jake Worth on September 14, 2017

How do we test a system that seems untestable?

Untested legacy code is the bane of many a developer's existence. In a permissive language like Ruby, continued development in an untested legacy codebase requires adding tests. Users expect certain behavior to survive each release intact.

For some systems, the traditional pyramid of test types (unit, API, and integration) is insufficient. Unit and API tests must presumably cover hundreds or thousands of functions; which matter? Integration tests could cover a wide range of user interactions; which are likely? Familiarity with the project is a luxury we don't all enjoy.

I recently worked on an application that took a complex input (a production database), ran it through a large SQL function, outputting a giant CSV file representing thousands of permutations of data. Each stage of the transformation was complex enough that visual inspection would fail to determine if something was amiss.

None of the traditional test types could handle this challenge. Unit tests came close, but the isolation of the test database could not rival the complexity of a large, messy production database. Unfortunately, not testing was not an option. Like every application, development needed to keep moving forward. So, what did we do?

After much brainstorming, we found a solution: an obscure technique called a Gold Master Test. I wrote about the experience not long ago.

Continued discussion led to a talk that I presented at RailsConf 2017 in Phoenix, AZ. My presentation included finding a clear definition of a Gold Master Test, writing a test, and working with an existing test.

Want to learn more? Check out the recording of my talk at RailsConf, and follow along with the slides.

Test writers of every experience level should leave this talk with a better understanding of this technique, and a broader conception of what a test can be. Thank you to Hashrocket for continuing to sponsor my public speaking endeavors.

Photo Credit: Christoph Gockel