Evolving code and adding new technologies - Part 5

The implementation of the proposed architecture is at hand. We dive into the change and see how Redis (or any other message queue or pub/sub broker) makes our life easier. TL;DR Time to make the change. Read it all!

For a recap here is Part 1Part 2Part 3 and Part 4
Components of our exampleGitHub branch
Our monolithic example will be divided to three processes: ProducerRedis (3rd party broker)Consuming ProcessorThin client After we are done with the change, the server will look like this:

On to the code The Dataflow library from the previous part will be removed in the consumer. Now that the processes are separated the consumer is super fast up until the actual processing which takes around 1 second.

The different components will work directly with Redis which will in fact be the backbone of our pipeline.

Ten producers are being used to send messages into 10 Redis lists. Each list has a key named {Producer_i} where i is the number of the producer. The consumer is aware that there a…

Evolving code and adding new technologies - Part 3

In this part of the series we will see how our code, after having evolved from a simple event call to a concurrent mess, will first and foremost get a bit into shape and then being throttled to produce best results. TL;DR The light at the end of the tunnel is not oncoming traffic.

More parts are here:  Part 1 and Part 2Part 4 and Part 5

Before we start I urge you to go read through the documentation. These are the links that hlped me write this post and learn how to use the Dataflow Library
Dataflow (Task Parallel Library)Walkthrough: Creating a Dataflow pipelineWalkthrough: Using Dataflow in a Windows Forms applicationHow to: Write messages and read messages from a Dataflow blockUpdate: On Post and SendAsync On Post and SendAsync in TPL Dataflow Let the data flowGitHub branch

This may be "old" tech for some or all of you but it was new to me when I decided to pop my head out of the "company" man role and into the consultancy role in my career. So lets evolve and …

Evolving code and adding new technologies - Part 2

In Part 1 of this series we saw a basic, naive implementation of a monolithic desktop client that draws input from some input producer and displays data. Now imagine that this is your system. You built and deployed it a while ago and time went by. On a quiet morning while you sit on your desk sipping coffee and reading your favorite news outlet, your PO and hands you over the new requirements for the next version. TL;DR it will get messy before it will get sweet

Parts 3-5 are here: Part 3Part 4 and Part 5
Version 2GitHub branch

According to the requirements now you have more than one producer. And they produce data faster. However your processing of data just got slower. You have to fetch some older data from the DB, do some correlations and then save data back to the DB. No matter what the processing is your data processing takes anywhere from 600ms to 1000ms. Yes you read right; 1 sec by the time data is correlated, stored in the database and displayed. Database optimization and m…

Evolving code and adding new technologies - Part 1

This is a start of a series starting on some simple old fashion code and evolving such code to something more modern. I will not use buzzwords like microservices or containers. This is beyond the scope of this article. What I will try to show is how to approach the past and future decisions step by step by way of refactoring and introducing new technologies. TL;DR There is none, read it all
Disclaimer The idea for this series is taken from a real life "war story". I am however not the least bit inclined to disclose the source. Nor does any part of the code reflect the full and actual system, the idea for the post has stemmed from.

Parts 2-5 are here: Part 2,  Part 3, Part 4 and Part 5
Part 1 - Old School The code for the old school code is pretty straight forward and resides on the master brunch of this GitHub repository. Initial architecture The initial architecture of the system looks as such:

Consider an application with two major threads; One thread reads data from some …

Code Review Boards - How to stay professional

I have recently come across the concept of a code review board. At first it was an awkward moment; My change set was out there for all the world to see and comment. I admit I was stressed about it and with good reason. What I will do in this post is put the record straight, as far as I am concerned, on the good, the bad and the ugly of public/social code reviews. And since in the near future my code will be standing there naked for all to see on some public repository, this should make for a very good exercise on what to expect and what to reject when it comes to review boards. TL;DR Code reviews are good. Review boards meh...

Past (nearly) perfect At some point during my searching for topics to write I recalled some articles I wrote back in the day on CodeProject. And I also do remember that, unless your article was pure garbage, review of the article and comments where polite, leveled and cool headed. Social media was not that proliferated and people did not openly enjoy putting oth…

How to approach a new technology

Someone asked me to write a post about how I approach new technologies. There isn't really one way to do that and what works for me may not work for someone else. Having said all that let's explore the different options standing before any developer who wants to explore a new technology. TL;DR Put time in your calendar and go for it.
Τα πάντα ρει, μηδέποτε κατά τ' αυτό μένειν - Everything flows, nothing remains the same - Heraclitus We are living in a world where you see a new framework or a new language pop all the time. It was less than a year ago that Angular took the web by storm and went head to head with its predecessor AngularJS and React and then VueJS popped it's head to say Hi.
Who would have thought Python would become the go to language for cyber security, or that Go would become the next hot thing.
So let's stop beating around the bush and get to it.
Go Online When I started to plow my way through technology all we had was BBS message boards and noisy m…

Time Travel Debugging!

Exciting news! We can now go back and forth in time without a Delorian with WinDBG Preview for TTD!

Hopefully a post with this will come soon!

Update 25/10/2017: Unfortunately Time Travel debugging works well only with C++ currently. Since there are quite a few blog posts on the subject of C++ I chose not to pursue the matter any further until a version for .NET comes out.