What is the difference between SQS vs RabbitMQ vs Memphis?
Avatar Idan Asulin
Apr 16, 2023
13 14 15
What is the difference between RabbitMQ vs ZeroMQ vs Memphis?
Avatar Idan Asulin
Apr 16, 2023
10 11 12
What is the difference between Kafka vs ZeroMQ vs Memphis?
Avatar Yaniv Ben Hemo
Apr 16, 2023
2 3 4
What is the difference between Azure Event Hub vs Kafka vs Memphis?
Avatar Idan Asulin
Apr 16, 2023
5 6 7
How does ActiveMQ vs RabbitMQ vs Kafka vs Memphis work?
Avatar Idan Asulin
Apr 16, 2023
7 8 9
What is ActiveMQ vs IBM MQ vs Memphis?
Avatar Yaniv Ben Hemo
Apr 16, 2023
3 4 5
What is IBM MQ?
Avatar Yaniv Ben Hemo
Apr 16, 2023
4 5 6
What is an mq server?
Avatar Idan Asulin
Apr 16, 2023
3 4 5
How does mq messaging take place?
Avatar Idan Asulin
Apr 16, 2023
2 3 4
Understanding Kafka Topics and Partitions
1 2 3
Memphis Approach

An event stream that enters Kafka persists as a topic, which is what Kafka considers to be a materialized event stream. Simply put, a topic is any event stream at rest. A topic groups related events and stores them durably. For the sake of understanding, you can think of a topic in Kafka as a folder in a file system or a table in a database.

In Kafka, topics are used to decouple consumers and producers. A consumer pulls messages from the topic, while producers push them into the topic. A topic can have numerous consumers and producers. 

Topics in Kafka are divided into numerous partitions. You can think of partition as the smallest unit of storage that stores a subset of all the records owned by a particular topic. A partition is a single log file and records are written to it (following the append-only property, where the existing data is immutable).

Memphis Approach

Just like Kafka has topics and RabbitMQ has queues, Memphis has stations. These are essentially distributed units used for storing messages and provide a powerful and easy-to-use messaging queue for applications. Think of them as virtual entities residing on a kind of file known as “stream” that stores the data. The stream files are stored either on non-volatile storage or in the broker’s memory, depending on your configuration per station. 

All stations have a retention policy (by total size of the stored messages, store time, and stored messages) that defines how and when the messages will be removed from them. They are also distributed across Memphis brokers (one or more) depending on the number of station replicas that have been configured, and data is poured in a RAID-1 fashion.