Posts

Showing posts with the label Redis

Evolving code and adding new technologies - Part 4

Image
In this part we will focus more on the process of changing a system architecture and the design at hand  than on code. After trying different approaches in an effort to not change the architecture entirely, with no success, there is only one thing to do. Change the architecture entirely. TL;DR Brace yourselves, major changes are coming. And they are good!

For a recap here is Part 1Part 2Part 3 and Part 5
What this change is not The system at hand is an on premise system. It does not run in the cloud. I dare say that it does not even run on the same network as the customer's other services. It is an island with a closed network with no access to the world outside including the client's kitchen. These things exist to the extent that remote debugging is not possible, not even plugging a USB key after installation (if there is a USB port on the machine). I plan to address debugging such systems in a post after the series is over so stay tuned ;) About the process of changing a…

Evolving code and adding new technologies - Part 5

Image
The implementation of the proposed architecture is at hand. We dive into the change and see how Redis (or any other message queue or pub/sub broker) makes our life easier. TL;DR Time to make the change. Read it all!

For a recap here is Part 1Part 2Part 3 and Part 4
Components of our exampleGitHub branch
Our monolithic example will be divided to three processes: ProducerRedis (3rd party broker)Consuming ProcessorThin client After we are done with the change, the server will look like this:

On to the code The Dataflow library from the previous part will be removed in the consumer. Now that the processes are separated the consumer is super fast up until the actual processing which takes around 1 second.

The different components will work directly with Redis which will in fact be the backbone of our pipeline.

Ten producers are being used to send messages into 10 Redis lists. Each list has a key named {Producer_i} where i is the number of the producer. The consumer is aware that there a…