A guide to offset pagination

To help you fetch data from a 3rd-party resource quickly and without overwhelming your server, you’ll need to leverage API pagination.

And while there are many pagination methods to consider, offset pagination, or offset-based pagination, is likely at the top of your list.

We’ll break down how this method works so that you can decide whether it’s the right fit for any of your integration scenarios.

What is offset pagination?

It’s an approach to pagination that, for a given request, uses an offset to determine the starting point of data to fetch and a limit to cap the volume of data that's returned.

To prevent any confusion, we’ll distinguish offset pagination from other methods.

Offset pagination vs cursor pagination

Cursor pagination lets you define the starting point of each request via a unique identifier (e.g., a timestamp), while offset pagination simply skips a predefined set of items with each request.

Offset pagination vs keyset (i.e., seek) pagination

Keyset pagination uses a key to determine the starting point of a request while offset pagination just skips a predetermined set of data.

Offset pagination vs token-based pagination‍

Similar to the methods above, token-based pagination uses a token in a given request to signify the starting point for fetching data; offset pagination doesn’t use a token—or any unique identifier—to fetch data across requests.

Offset pagination examples

To help bring our definition to life, let’s walk through a few examples of offset pagination.

Employees

Say you want to get a list of employees from an HRIS solution and each page includes 30 employees. 

Your first request can be structured as follows:
<code class="blog_inline-code">GET /v1/employees/?offset=0&limit=30</code>

Product reviews

Say you want to fetch reviews from a specific product (e.g., product ID=12345) and from a certain site. Let’s also assume that the site shows 10 reviews per page and you only want to get reviews that are on the 3rd page.

To retrieve these reviews, you can make the following request:
<code class="blog_inline-code">GET /v1/products/12345/reviews?offset=20&limit=10</code>

Related: Everything you need to know about REST API pagination

Invoices

Imagine that you want to retrieve invoices from your ERP system. You also only want to retrieve invoices from a certain vendor (e.g., vendor ID=12345) and you decide to only retrieve 5 invoices at a time.

You can structure your first request to something as follows:
<code class="blog_inline-code">GET /v1/invoices/12345/?offset=0&limit=5</code>

Offset pagination pros and cons

Now that you know how offset pagination works and how it differs from other methods, you may be wondering what its benefits and drawbacks are. 

We’ve broken them down below.

Pros

• It’s relatively easy to implement and understand, as the offset is usually determined by the page size
• It’s easy to access a specific set of data via the offset
• It’s widely support by API providers, so understanding how it works and how to leverage it can prove useful for many, if not all, of your integrations

Cons

• If a dataset grows or shrinks, you can end up skipping or double counting data
• Even with a large offset, every data point needs to be processed. As a result, requests with a large offset can take a while to execute and consume significant computational resources
• If a dataset isn’t paginated or sequentially ordered in another intuitive way, offset pagination can end up yielding confusing and inconsistent responses

Related: Tips for implementing API pagination effectively

Manage pagination with ease by using Merge

Merge, the leading unified API solution, lets you connect your product with hundreds of 3rd-party SaaS applications through the platform’s unified API.

This means you only have to consider how Merge’s Unified API handles pagination (among other items, like rate limits and authentication), which helps you scale your integration builds more quickly and maintain them more easily.

You can learn more about Merge’s Unified API, among the platform’s other features, by scheduling a demo with one of our integration experts.

Offset pagination FAQ

In case you have any more questions on offset pagination or API pagination more broadly, we’ve addressed several more commonly-asked questions below.

What is API pagination?

It refers to any pagination method that lets you fetch a subset of data with each request. 

And while the methods differ, they serve the same purpose: to help you retrieve data from a large data set in a way that won’t overwhelm the server and that'll allow you to receive responses relatively quickly.

What is the best API pagination method?

The “best” pagination method largely depends on the type of data you’re pulling from. 

For example, if the data set changes frequently, the best method can be one that uses a unique identifier, like cursor-based pagination. But if the data set changes infrequently, offset pagination can be the best option since it’s relatively easy to implement and would fetch data reliably.

That said, in some cases, you won’t have the ability to choose a method, so your “best” pagination option would be whatever method the API provider allows.

What are some best practices for using offset pagination?

Here are just a few to keep in mind:

• Sort the data with immutable fields. By using fields that won’t change the position of data over time (e.g., <code class="blog_inline-code">created_at</code> instead of <code class="blog_inline-code">updated_at</code>), you can avoid dealing with duplicate and/or missing data 

• Avoid setting a high limit. This ensures that you’re able to reap the benefits of pagination—minimal server strain and fast response times 

• Review the API provider’s documentation. Their docs may provide useful insights, like the maximum limit that can be used in a request or the default volume of data that’s returned when a limit isn’t provided  

• Log the offset and limit used in each request. These logs can make it easier for your team to debug issues quickly over time