What Makes a Maintainable Test?

If you've been around the block a few times you already know that testing is IMPORTANT. If you've tasted the sweet sweetness of a test-first style of development and continuous delivery, you already know that automated testing is also IMPORTANT.

But how do we write our tests? Do we treat our test harness code with the same 'clean code' standards as our application code? Is it MAINTAINABLE?

Behavior Driven Development

One way to address test maintainability is by using the practice of Behavior Driven Development. BDD is a shift in thinking from building tests around your application to identifying the desired behaviors of the application. Each behavior is described in a human readable fashion along with its acceptance criteria. The desired behaviors of an application could be thought of as the requirements. Although its easy to argue that BDD is just using fancier terms for similar Test Driven Development concepts, the results in practice have far reaching effects, one of which we'll discuss below.

Got to Keep Them Separated

In brief, Separation of Concerns is the keeping of conceptually different aspects separate and loosely coupled within an application. A good example of this would be the Model View Controller pattern. MVC separates the visual representation away from how the information is modeled as well as from the logic that manipulates it.

When used in an area such as web development, a Separation of Technologies (HTML and CSS) is also employed to express the View in the medium that makes the most sense to experts in visual web design. The back end Model and Controller components are implemented in a language like JavaScript because that best represents logic to web developers.

Separating the Testing Concerns

These separations are powerful when applied to testing. Who has the best understanding of the micro and macro requirements of your project? In many cases this is a business analyst or product owner. Is it intuitive for the business analyst or product owner to communicate to the developers via test implementation code? Probably not.

Instead, in the spirit of Separation of Concerns, the requirements should be documented without having to know how their acceptance tests will be implemented. In line with the Separation of Technologies principle the requirements should be written in a broadly understandable domain specific language for the benefit of all stakeholders. The implementation code-behind, loosely coupled to the requirements document, should be built in a coding language most suitable to the developers.

Not only does the concept foster communication among team members, Separation of Testing Concerns through the lens of Behavior Driven Development opens a much larger Can Of Effectiveness:

  1. Clear Intent – The behavior being tested is specific and is reliably communicated. Developers (including the author) often feel that the intent of their test code is self evident. Returning to read the test code later, however, requires time and effort to decipher. When using a textual test description language, test intent is readable and consistently formatted.
  2. Recognizable Grouping - Related requirements/tests are intuitively grouped, making it less likely to miss any. Short, textual descriptions can make it clear which tests are related to each other. During the maintenance of a project adding bug/fix tests in a consistent fashion can largely avoid code rot.
  3. Inherently Up To Date - Requirements documentation remains up to date. If we consider the textual suite of tests as being the embodiment of the requirements rather than a result of them, determining whether or not the software fulfills them is as simple as executing the tests. Perhaps these tests don't embody every requirement (user experience and integration level tests may be done manually or through another tool, for example) but the ones that are included in BDD tests continue to live and grow as the software does. It could be the beginning of a new project or near the life cycle end of one, these requirements will always act as a quantitative measurement at the press of a button.
  4. Version Controlled – Behavioral requirements documents in BDD are most often stored in the same version control repository as the rest of the code base. This means that not only are they growing with the code base but they are also easy to find. At the beginning of many projects, requirements artifacts are created for initial communication with stakeholders but then archived in a business specific location. Later maintainers of that project can often have a difficult time finding these original documents to give context to their development of new features and fixes.
  5. Implementation Agnostic - Test descriptions are not tied to one implementation language. If you have ever been tasked to migrate an entire codebase from an esoteric language into a modern one, having the original requirements decoupled from implementation would have the delicious flavor of a hot cinnamon bun.
  6. One Effort at a Time – Lastly, productive energy can be focused on one frame of reference at a time. First worry about what is required and expected, then worry about how to exercise the test through code. A big win for reducing cognitive load and increasing the quality of effort. A big win, too, for coordinating different skill sets within a team.

Enter Gherkin [stage right]

A large amount of thought work has been put into the Separation of Testing Concerns in BDD. Using the well developed Given/When/Then text structure of the Gherkin behavior language we can generate living test documents that are easily understood by anyone. These documents can also be interpreted and executed in a rigorous manner. Since it’s a descriptive, not a development language, Gherkin is not reliant on any one implementation. Behavior Driven Development tools such as Cucumber, SpecFlow, Behat, and JBehave all employ Gherkin to support human readable testing in their respective coding languages.

Note: This article doesn't cover the specifics of the Gherkin language. That is left up to the many great articles already written such as these from Dan North and Stephan Kurniawan, and Vieira et. Pufal.


Current requirements always change and old requirements are often forgotten. Ensuring that the documentation of these requirements remains as simple and focused as possible means that they must be easily located and understood by future analysts and developers alike. Though the code-behind for test implementation may be difficult to search through, the test specification will not, allowing for improved MAINTAINABILITY as the software matures.

Time to Talk

What do you think? Do you and your team find the above to ring true, to be challenging, or to be completely foreign? If this sparks a response I strongly urge you to bring up the topic of Separated Testing Concerns with a friend or team mate.

Looking for More?

It's rewarding to find out what others in our field have to say as well. Here's a bunch of respected professionals writing about subjects touched on in this article.

  1. B1. Atwood, Jeff. "Understanding Model-View-Controller." Coding Horror. 5 May 2008
  2. "Behavior Driven Development: Extremely Simplified." Centare. 19 April 2013
  3. Greer, Derek. "The Art of Separation of Concerns." Aspiring Craftsman. 3 January 2008
  4. Kurniawan, Stephen. "Cucumber Testing: A Language Everyone Can Understand." Nightlight. March 2015
  5. Martin, Robert C. "Clean code: a handbook of agile software craftsmanship." Prentice Hall. 2009
  6. Mind Tools Editorial Team. "Cognitive Load Theory: Helping People Learn Effectively." Mind Tools. 2016
  7. North, Dan. "Introducing BDD." Dan North & Associates. 20 September 2006
  8. North, Dan. "What's In A Story." Dan North & Associates. 11 February 2007
  9. Osmani, Adi. "Learning JavaScript Design Patterns." Adi Osmani. Volume 1.6.2 2015
  10. "Separation of Concerns, Take pride in your code." DevIQ.
  11. Sweller, John. "Cognitive Load During Problem Solving: Effects on Learning." Cognitive Science. April 1988
  12. "Writing Features – Gherkin Language." Behat. collected 5 February 2017
  13. "Gherkin." Cucumber.io collected 5 February 2017
  14. Vieira, Juraci, and Pufal, Nicholas. "3 Misconceptions About BDD." ThoughtWorks. 17 December 2013

Get your project started today

Get in Touch