A wide range of API providers use pagination to help preserve their server load, deliver responses faster, and only provide clients the data they need, when they need it.

That said, using API pagination effectively—both as a consumer and as a provider—isn’t necessarily easy.  

We’ll help both parties leverage API pagination successfully by breaking down several best practices. To start, let’s cover a few best practices for providers.

Adopt the pagination type that’s best suited for each endpoint  

Some of your endpoints will naturally return a high volume of data, such as employees if you’re a large company; while other types of data will return a smaller volume, like candidates if your company isn't hiring aggressively. In addition, many of your endpoints will return data types that frequently change (e.g., marketing qualified leads) and others that won’t (e.g., files on employees). 

Based on how frequently data changes and the volume of data that’s stored, you’ll want to adopt a specific type of pagination. 

For example, when a data set changes frequently and is large, it might be best to use cursor-based pagination. Reason being, if data in the data set is modified, added, or removed, your future requests can capture these changes. Moreover, it can use database indexes to quickly find the starting point for the next set of results, which naturally lowers the load on the database and improves response times. 

Related: Best practices for building API integrations

Default to page sizes that provide ample information but don’t compromise on performance

In many, if not most, cases, your API consumers won’t provide explicit guidance on how they want to paginate the responses, putting the burden on you instead.

With that in mind, you should opt for page sizes that won’t strain your resources and delay your response time but still provide enough information per page that clients aren’t forced to make immediate follow-up requests. 

It’s a tricky balance to strike and might require some level of optimization over time, but the long-term benefits for both clients and your business make the effort worthwhile.

Don’t use pagination for certain endpoints

In cases where an API endpoint typically returns a small set of data (and that’s unlikely to change) and/or when the API endpoint’s data is constantly evolving, it might not make sense to use pagination. There’s countless examples here, from getting the status of an IoT device to receiving a report from a security scan.

Share details on how to use pagination in your documentation 

Your API consumers might struggle with incorporating pagination in their requests.

To provide some guidance, you can outline the specific query parameters they should use and how each works. You can go a step further by showing a specific example of using these parameters and how the response can look.

For example, Sendoso’s documentation clearly outlines all of these details.  

They explain that you can use page and page_size query parameters to retrieve specific results. They then go on to use the following example:

GET https://app.sendoso.com/campaigns?page=2&per_page=50

They explain that the consumer is looking to get campaigns run through Sendoso, but to bucket the campaigns by 50 per page and to only share the 2nd page of campaigns.

They then provide the following response (which uses an ellipses in place of where the campaign data would appear):

{
  "campaigns": [...],
  "current_page": 2,
  "per_page": 50,
  "total_campaigns": 234
}

Research the different pagination options to pinpoint the best one for your needs

For some endpoints, you have the choice of which API pagination approach to use; while other endpoints don’t provide pagination at all. 

In cases where your situation is the former, you should carefully consider your options and see which one best meets your needs. As an example, NetSuite, an ERP system, offers offset-based pagination and page-based pagination.

Related: Best practices for managing API rate limits

Use parameters to only retrieve the data you need, quickly

Use certain parameters, like limit or offset, to minimize the data you get in a given response. This gives you more control over how much data you collect, which in turn can let you balance receiving the data you need while not straining your resources.

Avoid dealing with different API providers’ approaches to pagination with Merge

As you look to build and maintain customer-facing integrations, the process of understanding how different API endpoints use pagination—and then structuring your responses accordingly—can prove overwhelming.

To help you avoid this complexity, you can use Merge, the leading unified API solution.

Through Merge, you can simply build to a single, aggregated API to add hundreds of integrations to your product. This means that you only have to consider Merge’s approach to pagination—along with rate limits and authentication.

You can learn more about how Merge handles pagination—and how Merge works more broadly—by scheduling a demo with one of our integration experts.