Have you considered restricting requests or blocking web scrapers, bots, and spammers?
How do you handle that? Well, there is a technique for restricting network traffic in a system known as rate-limiting. It imposes a cap on how many times an operation can be repeated in a system in a given period of time.
This article will teach you all of the techniques involved.
Table of Content
- Introduction.
- Prerequisites.
- What is rate-limiting.
- Why do you need to limit network request rates.
- Techniques for rate limiting.
- How to implement rate-limiting techniques in NodeJS APIs.
- Conclusion
Introduction
Rate limiting regulates the number of times an operation is repeated and guarantees that the operation's fixed restriction is not surpassed.
In simple terms, the method of rate limiting can be summarized as follows:
- The rate limit rules for particular operations are specified by the system.
- In response to the user's query, the duration of the request increases. 3.The device keeps track of each process or request initiated by the user.
- When the frequency exceeds the system's fixed threshold (rate limit), no further requests are processed until the limit is lifted or changed.
Prerequisites
It is recommended that the following be in order to have a smooth understanding of this tutorial:
- Node version 10 or newer.
- Basic understanding of JavaScript and NodeJS.
- Integrated Development Environment (eg. Visual studio code)
- API testing tool (eg. postman)
Benefits of Rate Limiting
- Improved defense performance by preventing cyber attacks such as DDoS.
- Prevents web scraping, bots, and spammers from accessing your site.
- Prevents the server from running out of resources.
- Oversees the administration of internal and external programs, regulations, and quotas.
- Manages the transfer of data and device processes.
- Reduces the need for costly repairs.
Techniques for rate limiting
A rate is a count of the number of times an operation is performed in a system in general. Rates can be measured and limited using a variety of methods.
They include:
Sliding Window
This strategy is similar to the Fixed Window, which makes a set number of requests in a set period of time. It does, however, use a Rolling Time Window to account for large request spikes that would have decreased the Fixed Window technique's effectiveness.
Fixed Window
The technique specifies a maximum number of requests that can be handled in a given period of time. Setting a window size of 100 requests per hour, for example, ensures that only the first 100 requests made by the user within that hour will be handled, with all subsequent requests being rejected until the next hour.
Token Bucket
In this process, the system keeps track of how many tokens a user has to make requests in its memory (bucket). When a request is made, the tokens in the bucket are depleted, and the request is fulfilled until the token runs out. This strategy has the advantage of being able to delegate various amounts of tokens to different processes depending on the process's operating capacity.
Leaky Bucket
The Token Bucket strategy is very similar to this one. The number of requests that leak out of memory, however, limits the rates (bucket). Until handling an incoming request, this method tests to see whether the device has enough processing power to accommodate it, and if it doesn't, it's discarded.
How to implement rate-limiting techniques in NodeJS APIs
Implementing rate-limiting in a new or existing API project is simple and requires little configuration. I'll launch a new project in which we'll create the rate-limiter from the ground up.
Step 1
Kindly create a new folder in any directory of your choice, then initialize npm with the command below:
npm init
Step 2
we'll install some packages like express, express-rate-limiter, and dotenv with the command below:
npm i express express-rate-limit dotenv
Step 3
After that, you should have something close to what is shown below if you set up a simple NodeJS server inside index.js
const express = require("express");
const limiter = require("express-rate-limit");
const dotenv = require("dotenv");
// Enable access to environment variables
dotenv.config();
// Initialize express
const app = express();
const PORT = process.env.PORT || 5000;
app.listen(PORT, () => {
// eslint-disable-next-line no-console
console.log(`App running on port ${PORT}`);
});
Now that we have our server up and running let's proceed to the step of implementing the rate limiter.
Step 4
Since we have installed the express-rate-limit package, let's create a simple route to demonstrate how it works. Add the below snippet to index.js
.
// A simple api route
app.get("/api/welcome", (req, res) => {
res.send("Welcome to the API rate Limiter project ๐");
});
Step 5
We are going to implement the rate limiter in two different ways:
- Implementing it on all routes.
- Implementing it on a single route.
- Add the snippet below to implement it on all routes.
// Initialize Limiter
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // limit each IP to 100 requests per windowMs
});
// apply to all requests
app.use(limiter);
make sure it's added right before any route similar to this:
const express = require("express");
const rateLimit = require("express-rate-limit");
const dotenv = require("dotenv");
dotenv.config();
const app = express();
// Initialize Limiter
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // limit each IP to 100 requests per windowMs
});
// apply to all requests
app.use(limiter);
// A simple api route
app.get("/api/welcome", (req, res) => {
res.send("Welcome to the API rate Limiter project ๐");
});
const PORT = process.env.PORT || 5000;
app.listen(PORT, () => {
// eslint-disable-next-line no-console
console.log(`App running on port ${PORT}`);
});
- Add the snippet below to implement it on a single route by including it to our newly created endpoint
/api/welcome
:
// Initialize Limiter
const limiter = rateLimit({
windowMs: 60 * 60 * 1000, // 1 hour window
max: 5, // start blocking after 5 requests
message:
"Too many requests created from this IP, please try again after an hour",
});
// A simple api route
app.get("/api/welcome", limiter, (req, res) => {
res.send("Welcome to the API rate Limiter project ๐");
});
Step 6
Now let's test what we've implemented so far in this tutorial, I will be using postman but you can use any other test tools of your choice.
Now, Testing with the second rate limiter approach we configured earlier i.e the system should limit a request from the same IP after 5 requests.
As shown above, the system returns a 429 status code that indicates too many requests and also our error message Too many requests created from this IP, please try again after an hour.
Conclusion
In this post, we saw how the NodeJS API project handles rate-limiting implementation seamlessly.
If you have any additional questions, please post them in the comments section or reach out via Twitter ๐.