How to Make the Best use of API Pagination with Node.js

Offset-based vs cursor-based - Edition #23

Hey, I'm Marco and welcome to my newsletter!

As a software engineer, I created this newsletter to share my first-hand knowledge of the development world. Each topic we will explore will provide valuable insights, with the goal of inspiring and helping all of you on your journey.

In this episode, I want to show you the differences between the offset-based and cursor-based approach to pagination and a possible implementation in Node.js.

You can download all the code shown directly from my Github repository: https://github.com/marcomoauro/pagination-offset-cursor

---

📃 Pagination

Imagine having millions of records stored in a database and having to retrieve them via an API. This can become problematic as the huge volume of data causes delays and performance problems.

With such a large volume of data, the recovery process can quickly become resource-intensive. The size of the data set introduces challenges such as latency issues, where the time required to retrieve and transmit data increases as the volume increases. In addition, the pressure on server resources increases, potentially leading to slowdowns or even system crashes.

Pagination is a simple but effective solution to this problem, consists of dividing the data into manageable parts or “pages”, facilitating retrieval and processing for the server and management on the client, as displaying a lot of data on the interface can cause performance problems due to the size of the data to be managed.

In detail, we will analyse two techniques:

• Offset-based pagination

• Cursor-based pagination

---

🔢 Offset-based pagination

select *
from customers
order by id asc
limit 3
offset 0

Pagination based on limit and offset is a common technique used in databases to retrieve a subset of data at a time.

• limit: The limit refers to the maximum number of records that you want to retrieve in a single query. For example, if you set the limit to 10, the query will return a maximum of 10 elements.

• offset: The offset refers to the starting point from which you wish to retrieve data. It indicates the number of records you wish to skip before starting to retrieve data. For example, if you set the offset to 3, the query will start retrieving data from the 4 record onwards, skipping the first 3 records.

To retrieve subsequent data, simply increase the offset as shown below: