Struggling with the engineering part of the legacy brokers and queues planted the idea that it needs to be disrupted.
We dagged up more and realized a very rhetorical fact. Still, one that we usually forget – messaging queues and brokers are a means to an end, not the goal itself, and that understanding open a whole new variety of solutions.
The chosen solution was the most challenging one, but we believe, also the right one – a) It has to be open-sourced. b) It can’t be just an intelligent message broker on steroids. c) It has to offer what we call the “Day 2” operations on top to help build queue-based applications in minutes. From the more common ones, which are async communication between microservices, task scheduling, to event-driven applications, event sourcing, data ingestion, system integration, log collecting, and forming a data lake
With that understanding in mind, we formed the vision of Memphis – an intelligent and frictionless message broker that enables an ultra-fast development of queue-based applications for developers and data engineers.
I mentioned 1st part, so there is a 2nd part.
Memphis’ 1st part is the storage layer, the message broker with all its benefits as we know it today, and will continue to evolve dramatically over the coming releases. We will also push hard on GitOps, automation enablement, and reconstructing some of the APIs so they can be modular and open for the community to self-implement new ones. Last but not least – multi-tenancy, partitions, read-replicas, and more.
Memphis’ 2nd part is all about helping developers and data engineers build valuable use cases and queue-based applications on top of Memphis. More on that in the coming weeks.
Memphis cloud is right around the corner,
but if you prefer to self-host Memphis now – head here .