Posts

Showing posts with the label queue

Evolving code and adding new technologies - Part 5

Image
The implementation of the proposed architecture is at hand. We dive into the change and see how Redis (or any other message queue or pub/sub broker) makes our life easier.  TL;DR Time to make the change. Read it all! For a recap here is  Part 1 ,  Part 2 ,  Part 3  and Part 4 Components of our example GitHub branch Our monolithic example will be divided to three processes: Producer Redis (3rd party broker) Consuming Processor Thin client After we are done with the change, the server will look like this: On to the code The Dataflow library from the previous part will be removed in the consumer. Now that the processes are separated the consumer is super fast up until the actual processing which takes around 1 second. The different components will work directly with Redis which will in fact be the backbone of our pipeline. Ten producers are being used to send messages into 10 Redis lists. Each list has a key named {Producer_i} where i is the number of