Secure your API with Rate Limiting using Redis

Fixed window rate limiting - Edition #20

Hey, I'm Marco and welcome to my newsletter!

As a software engineer, I created this newsletter to share my first-hand knowledge of the development world. Each topic we will explore will provide valuable insights, with the goal of inspiring and helping all of you on your journey.

In this episode I want to show you how to create a rate limiter using Redis with its atomic and transactional methods.

You can download all the code shown directly from my Github repository: https://github.com/marcomoauro/redis-rate-limiter

---

👋 Introduction

Rate limiting is a technique employed to restrict the number of requests a client can make to a server, ensuring the security and stability of the application. By implementing rate limiting, we can prevent overload scenarios that may impact essential resources such as the database.

This technique is crucial in countering various cyber attacks, including:

• Brute force attacks: By limiting the number of requests, we prevent attackers from using server resources to systematically attempt to guess user passwords and gain unauthorised access to the application.

• Denial of Service attacks (DoS): Rate limiting prevents attackers from flooding the server with a barrage of requests, thus limiting attempts to overload and disrupt the server's operations.

• Scraping: Limiting the number of requests per IP address prevents attackers from scraping sensitive data from our application by restricting their ability to make excessive requests.

Possible solutions

We can use various strategies to implement a rate limiting mechanism in our application, one does not exclude the other, in fact mixed systems are usually employed:

• Fixed-window: this consists of checking how many calls are made by the client in a fixed window such as 1 minute, if more are made then all subsequent calls are rejected until the start of the next window.

• Sliding-window: differs from fixed-window in that the window runs over time, e.g. we could take into account the calls made in the last 5 minutes, this approach allows us to better adapt to traffic spikes.

• Token bucket: a bucket is used which contains tokens, each of which represents a call that can be made and with each invocation the tokens are decremented. Once finished, the client cannot call again until the tokens in the bucket are restored, so the server can manage the rate of calls. This approach is efficient in memory but may be susceptible to race conditions in multi-threaded environments or when processes share the same resources.

In our case, we are going to implement the fixed-window mechanism using Redis via the atomic function INCR and the use of TTL.

---

✅ Fixed-window strategy

Sequence diagram illustrating the interaction among different components and the flow of execution

1. The client calls the server API identifying itself by the API key.

2. The server queries the database to verify the validity of the API key.

3. The database returns the result of the key validity check to the server.

4. The server verifies the use of the API key in the cache.

5. The cache increments the API key usage counter within the time window.

6. The cache checks whether the API key usage is within the limits allowed by the time window.

7. If the API key usage is within limits, cache informs the server that the usage is within limits.

8. If the API key usage is within limits, server tells the client that it can continue with processing.

9. If the API key usage exceeds the allowed limits, cache informs the server that the usage has exceeded the limits.

10. If the API key usage exceeds the allowed limits, server stops API execution by responding to the client that API key usage has exceeded limits.

    ---

⚡️ Redis: MULTI, INCR and EXPIRE commands

I used Redis in this way to limit requests within a fixed window of one minute:

First, I created a key in this format: "rate_limit/{api-key-id}/{time-format}/{time}", where:

• api-key-id: is the ID of the API key in our database.

• time-format: is the format used, in our case is minutes ("m").

• time: represents the specific minute when a client makes a call.

if a call is made at minute 3 of the api key with ID 1 we would have the following Redis key: