New Year Resolution

I am always uncomfortable when I hear unwarranted criticism of MultiValue systems by people who are ill-informed or whose exposure has only been to the sort of old, arcane MultiValue systems of which there are far too many still in existence. In my small way I do my best to promote MultiValue as a leading edge technology whose benefits, once understood, have a clear place in the pantheon of technologies available to today's organizations. There are core systems in many key players in major industries that depend entirely on MultiValue and that would simply not perform with other technologies. Since I also work with some of those other technologies, I hope that I can present an open-minded and coherent argument of the benefits and downsides, and can advise on where MultiValue should be deployed and where other models, sitting side by side the MultiValue offerings, can be leveraged to offer the best advantages. This led, for example, to an in-depth meeting just a few days back on the merits of MultiValue, NoSQL, and SQL for a large organization and how the different technologies can mesh to deliver the full scope of what they want to achieve. Horses for courses and no one size fits all.

At the same time, I am always more uncomfortable when I hear entirely warranted criticism of MultiValue practices by people who are well informed and taking a brutally honest look at our industry.

Take as an example a new development manager at one of the major UniVerse sites in the UK. He had come into an organization whose business runs on UniVerse and that depends on UniVerse for its daily operations and for its customer outreach through its web site. This is a large 24 hour operation where system performance is critical and promoting high levels of customer engagement is their main differentiator in their marketplace and a key part of their strategy in winning business.

And yet, he told me, he was amazed to find that the UniVerse developers did not know how to write unit tests, that all testing was performed manually, there was no automated build process, no automated code checking, deployments were sometimes packaged and sometimes manual, and no source code control. Far worse, there did not seem to be any understanding that this unacceptable in today's world. How could he, as a manager, have confidence in the systems under his control if he did not have sight of what was happening at every stage?

Walk into any responsible Java, .NET, or web shop today and what would you expect to find? At the very least you will see:

• Good source control to show a glance what has been committed, when and by whom.
• Automated build and CI processes to show when a commit has broken the build.
• Coverage statistics from the run machines to show how much of the code has been tested.
• Metrics on code complexity to show bad practices and potential code quality issues.
• A fully automated pipeline that links together the commit, build, test, quality assurance and deployment stages.

These things are not luxuries. The core components — source code control systems such as Git and Mercurial, industry standard build managers like Jenkins and Bamboo, BDD, and acceptance platforms like Cucumber and FitNesse — are free and open source (commercial offerings like TeamBuild are also available). Organizations hiring professionals expect them to be experienced in test driven or behavior driven development and to be conversant with these tools. They are the daily bread for development.

How is it that so many MultiValue sites have fallen so far behind?

It's easy to see the progression from outside. MultiValue systems just work, so are frequently underfunded. People get used to the patterns of working that have seen them through in the past. It's all too easy to go from: "This is how we've always done it" to "This is how we do it" to "This is the right way to do it" based on history rather than active research into the best practices around. Just as our systems can become ossified, so can our practices.

There is good news. I've been working for much of this year at Travis Perkins, one of the largest UniVerse users. Travis Perkins is a Builders Merchant that owns a significant number of other brands in the UK, most of which have been assimilated over time into their UniVerse system. For those who like numbers: around 40,000 concurrent UniVerse processes spread over 3 machines cooperating over uv/Net; over 12,000 programs and 4 million lines of code making up a 20 year old system; and running a business with an annual turnover of over 5 billion pounds. That's a lot of concrete, and a lot of code to manage. It's a busy system — for most of the day each system is handling a million reads a second. That's a lot of work going on.

At this years' XP Days conference I had the pleasure of co-presenting an experience report with Nik Silver, an Agile Coach working at Travis Perkins, on the transformation we have seen there over the last year. This had been a traditional UniVerse site, with developers working away at a code face in an atmosphere of silence as part of small hard working and under-resourced functional teams. There was a code review process with applications support but no direct engagement with the business. There were experts in different parts of the system, so critical knowledge was siloed. There were no current coding standards, no source control, and only green screen editors. Despite this, they still managed to keep the business running, but through heroic efforts, and parts were in danger of becoming unmaintainable.

Today, that Travis Perkins is history. With new leadership they have restructured their IT department into Agile teams following the Scrum model. Teams now have control over their workload and collaborate directly with their product owners. Teams are cross functional, so there is true knowledge sharing. There is a buzz about the place that I have never seen there before. Out of this has come a pride and ownership — passing one of the Scrum boards someone has just pegged up a sheet, "We do not ship FINE." Just how far this has gone is proven by one of the teams wining the Scrum Team of the Year at the Agile Awards for 2013.

And it is not just the organization that has changed, but the UniVerse practices. Travis Perkins has embraced test driven development: even the "old hands" who — frankly — took some convincing have bought in and would now not go back to the old ways of doing things. We have introduced a CI pipeline that begins with individual developers creating automated tests and profiles for creating and tearing down assets before committing to git repositories and pushing their changes. This in turn kicks off an automatic git pull to a CI account in which the code base is rebuilt and all the tests are run. Failing and passing tests are reported via web and email, and only on passing runs is a separate and segregated release software account populated so no software can be released without integration testing. The next phase is to link this to a new packaging system currently being approved so developers can see where their releases have been installed and trace them through UAT, Pre-Production, and Live. We have started to investigate how to gauge cyclomatic complexity for UniVerse code so we can improve and measure code quality and have plans to automate acceptance testing.

All this has been delivered against a 20 year old UniVerse system and for a business that is still rapidly expanding. And TP is not alone in embracing this. Where I've been teaching Test Driven Development, I've found developers — even old hands like me — getting enthused and recognizing the advantages. They just hadn't realized that this can be done with MultiValue.

It can, and it should.

Perhaps your organization does an end of year retrospective, and perhaps not. But take a step back and review your year and realize that with a New Year comes a new challenge for all of us who work in MultiValue.

Do better.

Featured:

Jan/Feb 2014