Evolving code and adding new technologies - Part 4

In this part we will focus more on the process of changing a system architecture and the design at hand  than on code. After trying different approaches in an effort to not change the architecture entirely, with no success, there is only one thing to do. Change the architecture entirely. TL;DR Brace yourselves, major changes are coming. And they are good!

For a recap here is Part 1, Part 2, Part 3 and Part 5

What this change is not

The system at hand is an on premise system. It does not run in the cloud. I dare say that it does not even run on the same network as the customer's other services. It is an island with a closed network with no access to the world outside including the client's kitchen. These things exist to the extent that remote debugging is not possible, not even plugging a USB key after installation (if there is a USB port on the machine). I plan to address debugging such systems in a post after the series is over so stay tuned ;)

About the process of changing a system's architecture

Suppose that after long months of trying to fix the system, you reach the conclusion that there is no real option than to break the monolith into separate services. Not necessarily micro-services, and I have nothing against them, but Rome was not built in one day and re-architecting a monolith will be not be done in one go either.

You must remember that every time you want to make radical changes to a piece of software, you have to involve the stake holders. And they come in two flavors. Those who understand the problem and see the need for change and those who understand the problem but do not want to accept the fact that for the past few months they poured money down the drain and you produced no definitive results as far as they are concerned.

In fact you are asking them to trust you and pour more money down the drain, the way they see it. You have to convince them that what you suggest is the only way. By the way if you suggested that, before the hopeless effort to fix the system and they didn't hear you now is not the time to remind them. Always remember (sadly enough) that the higher the managerial level, the more ego gets on the way and the less self responsibility is exhibited. In fact as the one suggesting the change, you have great opportunities before you, which come with great responsibility as well.

Changes in architecture

Observe the diagram below. On the left is where we are and on the right where we want to get to.

From a monolithic application holding display and processing logic mixed together we want to separate our processing blocks into separate processes. We want to separate the concerns of those building blocks and we also want to clean their code; remove dead code for example, refactor and make code leaner.

What we will also achieve is the ability to individually test each building block and even run it on a different machine in order to load balance the compute resources. After all as we have seen our system is growing. Messages are pouring into our system by the thousands or hundreds of thousands. We need solid infrastructure to cache messages and allow for the individual processes to deal with the them the best possible way. We also need a queuing mechanism and maybe even a messaging system. Now I know that there are a few 3rd party products out there for the job and you can choose to use anyone you like to accomplish the task at hand. For the purpose of this series I chose Redis. In this case a one stop shop for all our needs.

One other thing to remember is that all these changes in a real system will be made little by little so you can always have a working product for a demo or a client delivery. Change small parts and write unit tests, have QA check for regressions and conduct internal acceptance tests. Control the development and the rate of bugs and be proactive.

Data Flows

One other important aspect when refactoring a system is to map the flow of data and the different use cases to be moved to other processes. This would be an excellent time to group together relevant flows and APIs and forego any circular dependencies and God Objects that most certainly have been formed throughout the lifetime of your product. It does not matter how you record these flows. It may be a spreadsheet, a document, a drawing as long as you do it. The reasons to do this are:

• Create and agree on a common language with your team mates
• Be able to measure progress by writing off the developed/refactored parts
• Always know which changes are going to break things and for how long. Assess risks and plan your work accordingly.
• Be able to communicate said progress to the system stakeholders and your peers in a concise way.

Time Estimates

I may not need to say that but plan for surprises and feature creeps while you are refactoring. Add buffers in your time estimates. If you want to put on a good show and make them lean and mean you may end up death-marching to failure or gaining the distrust of the stakeholders. Give sufficient, but not too large,  buffers to make them feel comfortable, but make sure the buffers are not too comforable. In these time estimates you have to be prepared for change, unit tests, code reviews and life in general. It may sound impossible but remember, you are supposed to know  the strengths and weaknesses of your team. 

Working with Redis - A Prelude

As a prelude for the next part, from the Redis website:

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius queries. Redis has built-in replication, Lua scripting, LRU eviction, transactions and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster.

The natural environment for Redis is anything but Windows OSs. It is however most certainly Linux. There is a port for Windows but my favorite setup on my own rig is a Linux virtual machine running Redis and Windows processes connecting to it. When I need to see the contents of Redis I am using Redis Desktop Manager and lately FastoRedis.

As for a library to use in code I am using the Nuget package StackExchange.Redis. You can also clone it from GitHub if you wish to see the way it works from the inside. In the code of this post I will not be using this library in an abstraction, however for your product you should.

After these words of wisdom lets pretend we persuaded our stake holders and move on.