Tests should be automated. You know it. I know it. Agile methods insist on it. Yet, all too often we don’t do enough of it, don’t do it soon enough, or worse don’t do it at all. I believe one big reason why we fall short is that we tend to automate at the wrong level. Most teams focus all their energy on unit testing and UI testing, while ignoring service-level testing altogether.
To see why this middle layer is so valuable, let’s look at each layer of the test automation pyramid a bit more closely.
Unit testing forms the foundation of the test automation pyramid. As such, it should be the largest part of a solid test automation strategy. Automated unit tests are wonderful because they give specific data to a programmer—there is a bug and it’s on line 56. Programmers have learned that the bug may really be on line 54 or 62, but it’s much nicer to have an automated unit test narrow it down than it is to have a tester say, “There’s a bug in how you’re retrieving member records from the database,” which might represent 1,000 or more lines of code. Also, because unit tests are usually written in the same language as the system, programmers are often most comfortable writing them.
User Interface Testing
In contrast, we want to do as little automated user interface testing as possible. Why? Because it’s brittle, expensive, and time consuming. Suppose, for example, we wish to test a very simple calculator that allows a user to enter two integers, click either a multiply or divide button, and then see the result of that operation. To test this through the user interface, we would script a series of tests to drive the user interface, type the appropriate values into the fields, press the multiply or divide button, and then compare expected and actual values. It works, but it is not ideal.
Plus, testing an application this way is partially redundant—think about how many times a suite of tests like this will test the user interface. Each test case will invoke the code that connects the multiply or divide button to the code in the guts of the application that does the math. Each test case will also test the code that displays results. And so on. Testing through the user interface like this is expensive and should be minimized.
That doesn’t mean we don’t need to test this kind of feature, though. We just need to find a way to run these test cases outside of the user interface. And this is where the service layer of the test automation pyramid comes in. In the way I’m using it, a service is something the application does in response to some input or set of inputs. Our example calculator involves two services: multiply and divide. Service-level testing is about testing the services of an application separately from its user interface. So instead of running a dozen or so multiplication test cases through the calculator’s user interface, we instead perform those tests at the service level. This is much more effective and much less cumbersome than trying to perform these same tests at the user interface level.
Automated unit testing is wonderful, but it can cover only so much of an application’s testing needs. User interface testing is often necessary, but should be used only in small doses. Service-level testing fills the gap between unit and user interface testing; giving teams the test automation they need, when they need it, with minimal effort and cost.
Do you want one short tip each week
from Mike to help you succeed with agile?