In this article, we will explore how to use StackExchange.Redis in an ASP.NET Core application to access a Redis server running in Docker. Redis is a powerful tool that can be used for various purposes such as session or full page caching, queues, pub/sub, leaderboards, and counting.
The example application we will be working with involves users writing posts in different categories. To optimize performance, we will use Redis to cache aggregated data for top categories and users, while the actual item/line data will be stored in a database as the "source of truth".
The article will guide you through two different use-cases, starting with a simple one and gradually progressing to a more complex scenario involving Lua scripting and the inbox pattern.
First, we will set up a Redis server using Docker. Docker provides an easy and convenient way to create and manage containers, allowing us to run Redis locally for development purposes. We will utilize StackExchange.Redis, a popular Redis client library for .NET, to connect to the Redis server from our ASP.NET Core application.
Next, we will implement the caching functionality in our application using StackExchange.Redis. We will demonstrate how to cache aggregated data for top categories and users, leveraging Redis' powerful data structures and features.
As we progress, we will dive deeper into more advanced topics. We will explore the use of Lua scripting in Redis to perform complex operations efficiently. Lua scripting allows us to execute a set of Redis commands atomically, reducing network overhead and improving performance.
Additionally, we will explore the inbox pattern, a technique used for real-time notification systems. We will leverage Redis' pub/sub feature to implement a simple inbox system where users can receive notifications for new posts or updates.
STEP 1: Setup Redis in Docker and Prepare the .NET Core Project
To set up Redis in Docker and prepare the .NET Core project, follow the steps below:
STEP 1: Install Docker Desktop: Download and install Docker Desktop from the official Docker website based on your operating system.
STEP 2: Create the Redis container: Open a command prompt or terminal and run the following command:
C:\dev>docker run --name redis -d redisThis command creates a Redis container named "redis" using the official Redis image.
STEP 3: Set up the .NET Core project: Open Visual Studio Community, which is a free version of Visual Studio, and make sure you have the ASP.NET and web development workload installed. Create a new ASP.NET Core 5 Web API project with Entity Framework.
STEP 4: Install the required NuGet packages: In the NuGet Package Manager Console, run the following commands to install the necessary packages:
Install-PackageMicrosoft.EntityFrameworkCore.Tools
Install-PackageStackExchange.RedisThese packages provide tools for Entity Framework and the Redis client library, respectively.
STEP 5: Create the entity classes: In your project, create a folder named "Entities" and add a new class file. Add the following code to the class file:
using Microsoft.EntityFrameworkCore;
using System;
using System.ComponentModel.DataAnnotations;
namespace PostService.Entities
{
[Index(nameof(PostId), nameof(CategoryId))]
public class Post
{
public int PostId { get; set; }
public string Title { get; set; }
public string Content { get; set; }
public Int64 TimeStamp { get; set; }
public int UserId { get; set; }
public User User { get; set; }
[Required]
public string CategoryId { get; set; }
public CategoryCategory { get; set; }
}
public class Category
{
public string CategoryId { get; set; }
public long PostCount { get; set; }
}
public class User
{
public int ID { get; set; }
public string Name { get; set; }
public int Version { get; set; }
} This code defines three entity classes: Post, Category, and User. Each class represents a table/entity in the database and includes the necessary properties.
STEP 2: Implement the Top Categories
To implement the top categories functionality using StackExchange.Redis in a C# application, follow the steps below:
STEP 1: Connect to Redis:
Declare a ConnectionMultiplexer object and connect it to the Redis server. In this example, the Redis server is running on the local machine at port 6379.
private readonly ConnectionMultiplexer _muxer = ConnectionMultiplexer.Connect("localhost:6379");STEP 2: Add Top Categories:
Implement the logic to increment the category count when inserting a new post. Use a transaction to ensure atomicity.
public void CreatePost(Post post)
{
using var dbContext = new PostServiceContext
(GetConnectionString(post.CategoryId));
var transaction = dbContext.Database.BeginTransaction();
dbContext.Post.Add(new Post()
{
CategoryId = post.CategoryId,
Content = post.Content,
Title = post.Title,
UserId = post.UserId });
dbContext.SaveChanges();
dbContext.Database.ExecuteSqlRaw(
"Update Category set PostCount = PostCount + 1 where CategoryId
= {0}",
post.CategoryId);
transaction.Commit();
var category = dbContext.Category.Single
(c=>c.CategoryId==post.CategoryId);
var cache = _muxer.GetDatabase();
var res = cache.Execute("ZADD", new object[] {
"CategoriesByPostCount",
"GT",
category.PostCount,
category.CategoryId },
CommandFlags.FireAndForget);
}STEP 3: Read Top Categories:
Implement the logic to read the top categories from the Redis sorted set. Use the SortedSetRangeByRankWithScores method to retrieve the category IDs and their corresponding post counts. Perform pagination using the offset and countCategories parameters.
public IEnumerable<Category> ReadTopCategories(
int offset,
int countCategories)
{
var cache = _muxer.GetDatabase();
var res = cache.SortedSetRangeByRankWithScores(
"CategoriesByPostCount",
offset,
offset+countCategories-1,
Order.Descending);
var ret = new List<Category>();
foreach(var e in res)
{
ret.Add(new Category {
CategoryId = e.Element,
PostCount= (long)e.Score });
}
return ret;
}In the above code, make sure you have the appropriate namespaces imported and replace any placeholders such as PostServiceContext and GetConnectionString with the relevant code for your application.
STEP 3: Top Users, the Inbox Pattern, and Lua Scripting for Atomicity
To implement the top users functionality, inbox pattern, and Lua scripting for atomicity using StackExchange.Redis in a C# application, follow the steps below:
STEP 1: Atomically Add Posts with Lua Scripting:
Implement the PusblishOutstandingPosts method to send new posts to Redis using the inbox pattern and a Lua script for atomicity.
public void PusblishOutstandingPosts(CancellationToken stoppingToken)
{
const string script = @"if tonumber(redis.call
('ZADD', @key1, @timestamp, @value)) == 0 then
return redis.call('GET', @key2)
else
return redis.call('INCR', @key2)
end";
var prepared = LuaScript.Prepare(script);
var cache =_muxer.GetDatabase();
for (int i=0; i<_connectionStrings.Count; i++)
{
cache.StringGet("Shard:" + i + ":LastPostId").TryParse(out long
lastPostId);
using var dbContext = new PostServiceContext
(_connectionStrings[i]);
foreach (var post in dbContext.Post.Where(p => p.PostId>
(int)lastPostId).OrderBy(p=>p.PostId))
{
var res = prepared.Evaluate(cache, new {
key1= (RedisKey)"{User:"+post.UserId+"}:PostsByTimestamp",
key2= (RedisKey)"{User:"+post.UserId+"}:PostCount",
timestamp=post.TimeStamp.ToString(),
value=post.PostId.ToString()});
cache.Execute("ZADD", new object[] {
"UsersByPostCount",
"GT",
int.Parse(res.ToString()),
post.UserId }, CommandFlags.FireAndForget);
lastPostId=post.PostId;
}
cache.StringSet("Shard:"+i+":LastPostId", lastPostId);
}
}The Lua script tries to add the post ID and timestamp to the PostsByTimestamp sorted set of the user. If it can add the key, it also increments the PostCount of the user. The script makes the ZADD and ZINCRBY commands atomic. If the post ID already exists, it returns the existing PostCount.
An extra counter per user is needed so that all key parameters map to the same Redis hash tag. And therefore would be placed on the same Redis shard. The curly braces like in the key “{User:5}:PostsByTimestamp” are signifiers for a Redis hash tag.
STEP 2: Read the Top Users:
Implement the ReadTopUsers method to read the IDs and names of the top users from the Redis sorted set.
public class UserWithPostCount
{
public int ID { get; set; }
public string Name { get; set; }
public int PostCount { get; set; }
}
public IEnumerable<UserWithPostCount> ReadTopUsers
(int offset, int countCategories)
{
var cache =_muxer.GetDatabase();
var topUsers = cache.SortedSetRangeByRankWithScores(
"UsersByPostCount",
offset,
offset+countCategories-1,
Order.Descending);
var ret = new List<Dto.UserWithPostCount>();
var keys = new List<RedisKey>();
foreach (var e in topUsers)
{
string redisId = e.Element.ToString();
keys.Add(new RedisKey(redisId+":Name"));
}
var names = cache.StringGet(keys.ToArray());
for (int i = 0; i<names.Length; i++)
{
int.TryParse(topUsers[i].Element.ToString().Substring(5), out
int id);
ret.Add(new Dto.UserWithPostCount {
ID=id,
PostCount= (int)topUsers[i].Score,
Name=names[i] });
}
return ret;
}In the above code, make sure you have the appropriate namespaces imported and replace any placeholders such as PostServiceContext and _connectionStrings with the relevant code for your application.
Summary
You used StackExchange.Redis to access Redis from C#. See my previous article if you want to manually access Redis via redis-cli or want more information about the example data model and use-cases.