Automated Testing Part 1
Early this year I released a new tool aimed at giving MultiValue developers a practical framework for building automated tests for their applications. The germination of this tool was the realization that I really needed a better way to test my own MultiValue solutions and that none of the regular testing frameworks allowed me to delve easily enough into the database to be effective or provided the latitude I needed to write convincing test cases. At the same time, my New Year's resolution was to improve my working methods, and so better automated testing has been very much on my mind.
So in this series I will be sharing the motivations that led me to build my own testing framework and looking into different types and uses of automated testing. We will turn our attention to what makes for good unit, integration, regression, user interface and volume testing, and how these can be applied to both new and legacy applications. We will also be looking at some of the popular toolsets for unit testing, code coverage and profiling.
Why use Automated Testing?
I'm sure many MultiValue developers regard automated testing as an overhead: when the language is such a joy to use and so easy to craft into business solutions, do you really need it?
Automated testing brings real benefits. The obvious ones are repeatability and consistency: a unit testing framework allows you to run swathes of automated tests to ensure that the application functions as expected and that nothing has broken since the previous run. Overall it can be a huge time saver after the initial investment, and who enjoys manual testing?
But there are other benefits as well, and to appreciate these we need to look at where automated testing stands today and some of the influences that have shaped their popularity.
Toolsets
Over recent years automated testing has moved from an obscure discipline largely confined to professional testers to the forefront of modern development. This rise has been assisted in no small measure by the increasing endorsement of agile development methods and the wealth of testing tools that are now in the reach of mainstream developers. Where Visual Studio, for example, originally confined testing tools to its premium editions, these now form part of their professional versions and are therefore common currency amongst C# developers. The various open source xUnit frameworks have popularized unit testing for other technologies, and have arguably driven down the price of more highly featured professional testing frameworks. I will return to these later in the series.
Agile Development
Agile practices are not new: the first of these, the Dynamic System Design Method (DSDM) appeared as far back as 1994 but was targeted at large enterprises and confined to a small group of cognoscenti. Today the agile landscape has been transformed with simpler methods like Scrum enjoying ever increasing popularity and a merging of agile practices with other software methodologies led by the eXtreme Programming (XP) movement. Scrum in particular has sparked the imaginations not only of developers and project managers, but more importantly the awareness has penetrated those higher echelons of management holding the purse strings. Today it is a lot easier to get approval to run a project along agile lines than ever before.
A common thread amongst agile practices is iterative development with a feedback cycle that requires regular releases. This would not be possible without a supporting infrastructure of automated build, deployment and testing. Scrum and XP both place strong emphasis on unit testing, with proponents of XP going further in extolling the virtues of Test Driven Development: an approach in which failing unit tests are written first, before the code is built to the point where the test passes. This "Test First" approach ensures not only that testing doesn't get pushed out to the last piece (and therefore often forgotten) but that development remains targeted towards specific objectives: there is a clear aim to pass the test and not to go wandering of into realms of technical fancy as developers are wont to do.
More generally, agile development has given rise to other supporting technologies such as storyboarding, prototyping and specification languages that help to improve the workflow and to encourage consistency. Code generation tools are starting to mature in the .NET space. In the world of web development JavaScript frameworks from jQuery and Dojo to Backbone and Knockout are common currency. The mainstream world has woken up to the benefits, always enjoyed by the MultiValue sector with its strong family of 4GLs, of adopting frameworks to assist development.
Design Patterns
Another movement that has played its part is the increasing use of MVC patterns and their derivatives. MVC (Model View Controller) aims to segregate the business layer of an application (the "Model") from its presentation (the "View"), and MVC frameworks are now available for practically all mainstream development technologies. MVC emphasizes a "separation of concerns" that allows automated testing to be much more readily applied to a solution. If you take the typical example of an order form in a web application, it is very difficult to automatically test if you only have the web page itself as your target. Once the working logic has been separated from the page it becomes trivial to test the business rules: adding and removing order lines, checking lookups and validations, examining calculations and boundary conditions. A parallel can be drawn with a well-structured MultiValue application: it is easier to unit test your business logic when it is encapsulated in Basic subroutines.
Testing for MultiValue Developers
Of course it is true to say that the ability for MultiValue developers to write such tests is nothing new. Giving your subroutines a standard calling interface means you can write test programs to call them with standard data and examine the expected results. But even so, I have been surprised at how few MultiValue sites even boast a good test system with adequate test data, let alone a consistent testing framework.
If you're not already using some form of automated testing, here are a number of key reasons (beyond the need to preserve your sanity) why you should consider doing so:
Cost is the first significant factor. Yes, there is a cost to producing unit tests, but the cost of fixing a bug once it is in production, both in financial and reputation terms, is far higher — according to industry estimates between 40 and 100 times greater. The earlier it is possible to catch a bug the cheaper it is to fix, and unit testing is the best way to ensure new bugs are not introduced as a by-product of development before it ships.
Complexity is the next factor. Modern systems have more layers and interdependencies than ever before and with that an increasing need for integration and regression testing and the need to prove that each layer is solid before releasing it into the wild. Even within an application, the fact that MultiValue systems use run-time calls to subroutines means that you need to ensure that you have tested everywhere a subroutine is called when you make a change. Unless you use external Basic functions and INCLUDE your declarations the compiler won't tell you of an argument mismatch on an external subroutine call - and not all MultiValue platforms support that.
Compliance is another powerful driving force, and where the encroachment of requirements such as SOX has increased the cost of performing software releases, so it has affected the cost of releasing patches for bad software.
Comprehension is arguably the most important of these. Test First approaches aim to ensure that by defining unit and acceptance tests from the start a developer proves their full understanding of the problem before coding begins: after all, how can you test for something you don't understand?
Next Article: Unit Testing.
