When it comes to obtaining complete lists of entities exposed by the API, a distinction needs to be made.
Some entities (i.e. models) can be retrieved all at once with a single API call, while others (i.e. records), are returned by the API in the form of pages. In the latter case, the response will contain the total number of resources in the meta.total_count
property:
The pagination that the API offers is offset-based: this means that you can control the results that are returned with the parameters page[limit]
and page[offset]
:
page[limit]
is the maximum number of entities to be returned
page[offset]
is the (zero-based) offset of the first entity returned in the collection (always defaults to 0)
Both the default value of page[limit]
and its maximum (that is, the maximum number of items that can be asked per page) vary depending on the specific endpoint. To obtain this information, refer to the specific documentation for the endpoint's page
query parameter.
Setting a page[limit]=5
and page[offset]=5
will return entities 6 through 10:
curl \ -H 'Accept: application/json' \ -H 'Authentication: Bearer <YOUR-API-TOKEN>' \ https://site-api.datocms.com/items?page[limit]=5&page[offset]=5
Our JavaScript client makes the list()
method available for fetching collections of entities. As we have seen, depending on the endpoint, the result may contain all the entities in the collection, or just one page.
// Returns all the modelsconst itemTypes = await client.itemTypes.list();
// Returns a single page of recordsconst items = await client.items.list();
In the case of a paginated endpoint, you can configure the pagination with page.limit
and page.offset
:
// Returns the first 10 recordsawait client.items.list({ page: { limit: 10 } });
// Returns records 6 through 10await client.items.list({ page: { limit: 5, offset: 5 } });
In the case of paginated entities, the client also provides the listPagedIterator()
method, which allows for fetching all the pages of the collection in a simplified manner, without manually handling offset-based pagination.
You can use this method in an async iteration statement:
1// We'll be building up an array of all records using an AsyncIterator2const allRecords = [];3
4for await (5 const record of client.items.listPagedIterator(6 // You can define any query parameter that the endpoint permits,7 // except for page (refer to the following example for clarification)8 { filter: { type: "article" } }9 )10) {11 allRecords.push(record);12}13
14console.log(allRecords);
The method listPagedIterator()
offers a few options to configure its behavior:
The concurrency
option determines how many API calls can be performed in parallel (up to a maximum of 10). The default setting is 1, implying that the calls are made sequentially, not in parallel.
The perPage
option specifies the size of the pages in the sub-requests that it will carry out in the background.
1for await (2 const record of client.items.listPagedIterator(3 // You can define any query parameter that the endpoint permits,4 // except for page5 { filter: { type: "article" } },6 // Pagination options7 { concurrency: 5, perPage: 100 },8 )9) {10 // ...11}
Normally, our listPagedIterator
handles pagination for you, but if you need to retrieve the total count of a query, you can use the rawList()
command to access the response's meta.total_count
property.
In this example, we query records (items
) of a certain model type (page
) using rawList()
with a filter, and then access meta.total_count
for the total.
const records = await client.items.rawList({ filter: { type: 'page' // API key (that you gave it) or ID (from its URL) }, page: { limit: 0 // We don't need any actual records, just the meta } })
console.log(records.meta.total_count) // Returns `11`