MultiValue Communications: The Persnickety Persistence Problem
Last issue we took a look at the good, the bad, and the ugly of the Telnet protocol in the context of our MultiValue applications. In brief, the Telnet spec is nearly 30 years old, the terminal types emulated over Telnet were antiques years ago, the protocol is implicitly insecure, and if that's not enough, it drops connections far too frequently.
We also briefly mentioned SSH and Telnet over SSL, and while these can provide improvements in security, they do little to nothing to improve the reliability of the connection. With miles of cable and countless devices in the network that are out of our control, improving reliability of persistent connections is problematic. But consider this: Might we improve reliability simply by decreasing our dependence upon persistent network connections?
On behalf of an industry that has been dependent upon Telnet from the beginning, the heresy of such a radical recommendation is well appreciated. However, as we look another 30 years into the future, perhaps radical thinking is exactly what we need, and now is exactly when we need it.
If we're perfectly honest with ourselves, we have to admit that all of our best planning will not prevent someone with a backhoe inadvertently digging a hole through the wires that are carrying all of our precious communications. It happens. Moreover, networking equipment fails and with all these devices between us and our information, something is definitely going to fail at some time, and it'll likely happen at the least convenient moment.
So if we can't stop these kinds of failures, perhaps we should instead be focused on minimizing their impact. To achieve that, let's start by taking a brief look back at our history of persistent connections.
Though it may seem obvious, we remain tethered to persistent connections simply because of how we communicate with our MultiValue applications using the Telnet protocol. Telnet is a persistent protocol; as long as there's a channel of communication between the client and server, Telnet works very well. Break that channel, however, and things can get ugly. Connections are dropped, sessions are terminated, and whatever was in progress at the time ceases to be in progress (barring some form of pre-established intervention).
Equally interesting, Telnet is a single channel protocol. In other words, when you establish a Telnet connection to your MultiValue system, you establish each connection one at a time. If that connection drops for any reason, you can certainly start another connection but there is no provision in the Telnet protocol for redundant or re-connectable connections to protect against interruptions and disconnects. Therefore, and at the risk of sounding alarmist, with Telnet we are all just one blip away from getting disconnected at the worst possible moment. For some, this may rarely happen. For others of us who seem to always be hundreds of miles away from our systems, these kinds of disconnects happen far too frequently.
Persistence has become so common throughout the MultiValue world, I would venture to say that most don't think much about it until the network hiccups and a connection is lost. THEN we think about it as we establish a new connection, clear any rogue record locks, check to be sure nothing was damaged or left hanging by the disconnect, and … possibly mutter a few questionable words throughout the process!
With this in mind, do we REALLY need persistence? Is it an absolute necessity that we keep a live connection to our MultiValue servers all the time in order to maintain a dialogue with our applications? Or do we simply continue to rely on persistent communications because this is just a fact of Telnet life, and that's the way things have always been done?
The greatest advantage of persistence is, of course, state management. With a persistent connection to a back-end server there is a block of memory available where we can store all sorts of information. Records can be read from disk into memory, manipulated in memory for the entire length of the connection, and written out as the application sees fit. As long as nothing bad happens to the network, it's all a beautiful thing.
There is of course a cost to all of these connections. Each persistent connection requires some long term commitment of memory, and depending on the number of simultaneous connections and data needed by the application, that can work out to be a pretty big chunk. There's also the ongoing network traffic for maintaining all of these connections, and while Telnet itself is really quite lightweight, anything multiplied by hundreds or thousands of clients can introduce a concern.
But back to the question: Do we really need persistence? If the greatest advantage that persistent connections offer is state management, and especially where that state is always just one network blip away from oblivion, then there must be a better way. Fortunately there is a better way, and the PHP language offers a stellar example of one possible solution.
In PHP, state management is handled much like MultiValue common variables. Simply set a "session variable" now and when you come back later that variable will still have its assigned value. In a web context where each request to the back-end server requires a separate and independent request, this actually looks to the web application like a block of persistent memory. But alas, it is a cleverly crafted illusion.
What's really happening is that the browser is announcing itself to the server on each request and the server is using that announcement to load a specific block of memory for that user from a disk file or database query. When the PHP runtime finishes processing a request, it updates that memory to disk (or database) until the information is needed again. The PHP script may never make another connection to the application. But if it does, all that information is readily available until the script intentionally discards it.
So in contrast to holding a big block of variables in memory indefinitely, PHP holds its information on disk and loads it into memory only when needed, manipulates it as necessary, and then puts it back out to disk before terminating. This process then repeats until the session is closed by the application, at which point the disk storage is eliminated. (If you think about it, this isn't all that different from what happens to memory when you logoff a Telnet session.)
The disadvantage of extra I/O aside, the advantages to storing state information on disk are numerous. Without a persistent connection there's less chance of a network hiccup causing problems. The actual state information will persist until the application says it's no longer necessary, even between reboots of the server itself! And the memory footprint of the application can be dramatically reduced as memory is committed only when it's needed.
So to answer the question of the necessity of persistent network connections, allow me to offer the oft-quoted words of motivational speaker Anthony Robbins: "If you do what you've always done, you'll get what you've always gotten".
As we look ahead at the next 30 years of MultiValue excellence, are you okay with continuing to get what we've always gotten?
