Automated Testing Part 2 - Unit Tests
In the first article in this series I wrote about the case for automated software testing of MultiValue applications. In this article I will be looking at the first stage in automated testing — the unit test.
Unit testing is the first step in any testing strategy and lies at the heart of modern programming practices that encourage a Test First approach to development.
Test Driven Development (TDD) encourages a tight coupling between unit testing and programming in which developers approach any new task by first creating their unit tests and then filling out the code required to pass that test, generally in a very short cycle — a few minutes at most. By doing this, the proponents argue, you not only end up with good test coverage, but also a better understanding of what your code is meant to achieve (you cannot write a test until you know what the code should be doing), and you are forced into adopting better design patterns. You are also more likely to stay focused on the requirements and also, through constant practice, to become as accustomed to writing tests as to developing code.
Unit Tests
So just what makes a good unit test?
Let's start with a standard definition, drawn from "The Art of Unit Testing" by Roy Osherobe:
"A unit test is an automated piece of code that invokes the method or class being tested and checks some assumptions about the logical behaviour of that method or class.
"It can be written easily and runs quickly. It's fully automated, trustworthy, readable and maintainable."
As you can see from the definition above, the concept of unit tests really fits well with modern, object-oriented languages. If you are developing in a language such as C# or Java, the class structure allows you to write your code in small, discreet functions that can be quickly and easily tested. 1
The reaction from most developers on encountering Test Driven Development (TDD) for the first time is that that it is a new demand on an already over-stretched and under-resourced pool. And certainly, just as with anything new, learning to adopt TDD does take time. But with the right frameworks and particularly for new code, actually creating unit tests is not as burdensome as it first appears.
Imagine that you are using C# and wish to write a function very familiar to MultiValue developers — the DCOUNT() function. You want to add this to a new Utils class as a static method and have chosen to adopt a test first approach. Fortunately, you are using Visual Studio which comes with a built in unit testing framework, one that lets you generate a test project to mirror your class library.
Before you even create your Utils class or write your DCount method, you can create a test class and start to code up your test. You know that your DCount method will need to take a string and a delimiter and should return the number of delimited entries as an integer, so you can create a test case (fig. 1).
[TestClass()]
public class UtilsTest{
[TestMethod()]
public void DCountTest()
{
string text = "ONE,TWO,THREE,FOUR";
string delim = ",";
int expected = 4;
int actual = Utils.DCount(text, delim);
Assert.AreEqual(expected, actual);
}
}
Fig. 1
Even if you have never seen C# you should be able to follow the test. Even this simple test encapsulates the main properties of a unit test.
Firstly, it calls one function only and uses predictable data to test expected outcomes. Higher level tests may introduce more complexity and context, but for a unit test we are first and foremost concerned with speed and predictability.
Secondly, the test sits outside the function and examines its inputs and outputs. These are checked through an assertion, and the Visual Studio tests include a range of different assertion types (Assert.AreEqual, Assert.IsTrue, Assert.IsFalse etc.) that make the test easy to read. The test is not part of, and does not alter, the production code.
Finally, creating the test requires only a minute or so of typing before we can build the method to pass it and we can verify the method instantly. Visual Studio has a Run Tests button to automatically run all tests in the solution in a random order so we can make sure it hasn't impacted anything else (fig. 2).
Fig. 2 Unit tests in Visual Studio 2010.
So, is our test complete? It passes, but we have not considered any boundary conditions. What if we pass a null string or the delimiter is empty, or the delimiter is a substring? We need to test each of those conditions to be confident, so you can extend the test as you improve your code: it is a matter of style whether each of these cases should be separate tests (fig. 3).
[TestClass()]
public class UtilsTest{
[TestMethod()]
public void DCountTest()
{
string text = "ONE,TWO,THREE,FOUR";
string delim = ",";
int expected = 4;
int actual = Utils.DCount(text, delim);
Assert.AreEqual(expected, actual);
// empty string
actual = Utils.DCount(string.Empty, delim);
expected = 0;
Assert.AreEqual(expected, actual);
// null string
actual = Utils.DCount(null, delim);
Assert.AreEqual(expected, actual);
// empty delimiter
actual = Utils.DCount(text,string.Empty);
Assert.AreEqual(expected, actual);
// multi-character string
delim = "EE";
expected = 2;
actual = Utils.DCount(text, delim);
Assert.AreEqual(expected, actual);
}
}
Fig. 3
As each case is added, the test is re-run and any improvements are made to the code to ensure a pass. The whole cycle is typically less than a minute, so the unit testing does not disturb the flow.
Unit Testing in the MultiValue World
So far we have looked at unit testing from a modern language perspective: but what about our functionally rich but procedural server code? Let's go back to that definition of a unit test above. Just how much of that is directly relevant to MultiValue code?
Classic MultiValue Basic does not decompose down to the same extent as OO code. A typical routine may contain many internal subroutines, each of which would correspond to an internal method on an OO class. Whilst this makes for well structured code, internal subroutines are by definition enclosed and not surfaced for outside inspection so the only way to test down at that level is to instrument the code, and whilst that is possible it violates the principle that the test itself should not alter the production code.
So instead we have to consider our own definition of a unit test. It is easy to get lost in semantics when discussing the different levels of testing, but the purpose of unit testing is really to quickly check the minimal testable unit, as opposed to full system or process testing (integration and acceptance testing). In our case, that minimal testable unit, maps neatly to an external subroutine or, where supported, an external function.
Functional Decomposition
Unit testing begins with good design and with the desire to make your system testable. The heart of any well designed MultiValue application is the external Basic subroutine — external subroutines form the (hopefully documented) API for your application, and should embed all the validations, business logic and activities of your system. They prevent unnecessary duplication, reduce source code control conflicts and ease development. They can be surfaced through a myriad of APIs so your application can participate in the wider world.
Refactoring an existing legacy system will be covered later in this series: in a perfect state, your application will expose reusable subroutines that perform every action that makes up your system. That then is the recipe for a testable system.
Consider the script in figure 4 written in mvTest as part of a suite used to test a support system. The test calls a subroutine responsible for submitting a new support call and checks the key and any errors returned. I've removed some of the code for brevity and the script is modeled on more familiar MultiValue Basic but in essence it is not so different than the C# example above.
$INCLUDE support.inc api.h * Create the call details InData = '' InData<IN.RAISE_DATE> = Date() InData<IN.RAISE_TIME> = Time() InData<IN.RAISE_USER_ID> = "USER" InData<IN.SHORT> = "AUTO RAISED HOLD CALL" InData<IN.OWN_REFERENCE> = HOLD_REF InData<IN.ACCOUNT_NAME> = "ACCOUNT1" InData<IN.MENU_PATH> = "MENU1" InData<IN.DETAIL> = Text * Add a new support issue Call ssAddIssue( InData, OutData, ErrText ) Key = OutData<1> AssertEmpty "Add issue should not give error", ErrText AssertFull "Add issue OutData should hold key", Key AssertIs "Key should start with an L", Key[1,1], "L" AssertMatch "Rest of Key should be numeric", Key[2,9], "1N0N"
Fig. 4 Calling a subroutine in mvTest.
This kind of test is made possible by the system structure. This whole application is surfaced through a series of subroutines normally called from a web site and has been designed from the outset with testability in mind.
Data Management
Should a unit test touch the database? This is a contentious issue, but one where once again the MultiValue approach may be out of step with the mainstream. Purists may disagree with my verdict, but here are my thoughts on the matter.
When reading any book on unit testing, you will inevitably get to the point where the author decides to mock out a database connection because it is (a) difficult (b) slow and/or (c) not relevant to a client-side developer. The expectation is that any database issues will be caught through subsequent integration or acceptance testing but otherwise get in the way of unit testing.
In the MultiValue world function and data are intimately connected. Our code runs beside our data — a MultiValue system is first and foremost an application platform (I never describe it as a database), but one that offers local persistence and fast data access. Performance and difficulty are no reasons to mock out the database, and for MultiValue developers the storage and management of data is an integral part of their responsibilities. If the action of a subroutine involves writing data, testing that data should lie inside the unit test box.
There is a snippet from another mvTest script in figure 5 checking a subroutine call, this time taken from an agile project management system. The test changes the dates on a sprint causing the application to reschedule the estimated task dates and update the work diary. Notice how the test unashamedly checks the database.
* Update the sprint with the new dates
Call tmUpdateSprint( SprintId, SprintRec, ErrText )
AssertEmpty "ErrText should be empty", ErrText
* Task 1 should now be starting on the first day
TaskId = TASK_LIST<1>
TaskDateRec = Read("TASK_DATES", TaskId)
AssertIs "Task 1 should start on ":FirstDate, FirstDate, TaskDateRec<1,1>
* Task 1 should be in the diary for the assignee
DiaryId = FirstDate : "*brian"
DiaryRec = Read("DIARY", DiaryId )
AssertHas "Diary should have task", DiaryRec<DIARY.EST_TASK_ID>, TaskId
Fig. 5 A Unit Test checking the database, no mocking allowed.
Unit testing today, more than ever, is a key ingredient in delivering software quality and preserving the dubious sanity of developers. If this article has whetted your appetite, there is a huge wealth of material available on unit testing and on Test Driven Development if you wish to learn more.
In the next article I will look at integration and regression testing.
1. The academic name for this is the Single Responsibility Principle (SRP) that dictates that each function or method should have one, and only one, purpose. Whole books have been written on the art of splitting down functions that are doing too much and removing dependencies to encourage isolated testing. Clean Code, by Robert Martin, expends many pages on discussing refactoring for clean unit testing and should be on every programmer's shelf.
