Redis
Redis is an acronym for the Remote Dictionary Server. It is a key-value store, this key-value store can be used as a repository for reading and writing data. Redis is a No-SQL database, which means that unlike MySQL, or Oracle database, it does not have structures such as table/row/column/functions/ procedures, etc. and also it does not support commands like SELECT, INSERT, DELETE and UPDATE. Strings, hashes, lists, sets and sorted sets are the different data structures provided by Redis.
Redis clients and servers can be present on the same computer or on two different computers. Use a client/server interface to communicate with Redis.
Redis Server: Redis server is responsible for storing data in memory. This manages all manner of operations and forms the architecture’s main part.
Redis Client: Redis client may be the Redis console interface or the Redis API of any other programming language which sends data to the Redis server.
Kafka
Kafka is a publish-subscribe messaging system. With streaming technologies such as Kafka, you can actually process new data as it is generated in your cluster, you might save it to HDFS, or you can save it to HBase or some other database, so you can actually process it in real-time as it comes in, you can do all that with streaming.
Producers: Producer publishes a message to a topic.
Broker: Broker is a cluster made up of one or more servers in Kafka. The broker receives messages from the producer, assigns them to offset and commits the message.
Topics: Messages are divided into topics. Topics are broken down into a number of partitions where they index and store messages that receive an incremental Id named offset.
Consumers: Consumers subscribe to various topics and read data from brokers. They read data in consumer groups. The consumer always keeps track of which messages it has consumed by keeping track of the offset of messages.
Difference between Redis vs Kafka
Redis Kafka
It was developed by Salvatore Sanfilippo Kafka is an open-source software platform developed
for developing a real-time weblog by LinkedIn.
Redis supports push-based delivery of Kafka supports pull-based delivery of messages
messages that means messages published meaning that messages published in Kafka are never
to Redis will be delivered automatically distributed directly to consumers, consumers subscribe
to subscribers immediately. to topics and ask for messages when consumers are
ready to deal with them.
Redis does not support the concept Kafka supports parallelism due to the log partitioning
of parallelism. of data where multiple consumers consume in
consumer groups at the same.
Redis sends the messages to the Because Kafka is a log, there are always messages;
consumer all at once and later the you can monitor this by setting a retention policy for
message is removed. Thus, no one messages. E.g. 7 days retention.
knows where the data is held.
If speed is of concern than use Redis Even after it delivers the message. Kafka Persists the
as messages are not persisted by Redis, messages, so it is quite slow compared to Redis.
so it delivers them more quickly.
Redis is an in-memory store. It ensures Kafka is meant to handle large amounts of data. It
that it uses its primary memory for allows as many servers as required to be used. It uses a
storage and processing which makes disk to for its storage, so it may be slow to load.
it much faster than the disk-based Kafka. Nevertheless, it can hold a large amount of data (i.e. in
The only issue with Redis ‘ in-memory terabytes) for a longer retention period thanks to the
storage is that we cannot store large disk storage ability
amounts of data for a long time.
Because the primary in-memory is
smaller than a disk
It is used for various use cases such Kafka has various use cases such as Messaging,
as Session Cache, Full Page Cache Website Activity Tracking, Log Aggregation,
(FPC), Leader Boards/Counting, Stream Processing, Metrics, Event Sourcing, Commit
Pub s/ Sub, Queues log.
The Tech Platform