Most of the endpoints returning a paginable collection (user timeline, tweet search, home timeline, ...) returns a sub class of TwitterPaginator.
By default, instance is built with the initial response data, and you have access to method to fetch automatically next page(s).
const homeTimeline = await client.v2.homeTimeline();You can iterate with native Symbol.iterator.
for (const fetchedTweet of homeTimeline) {
// do something with fetchedTweet
}In paginators that contains
- tweets:
.tweets - lists:
.lists - users:
.users - Only IDs:
.ids - Welcome messages:
.welcomeMessages - Events (like DM events):
.events
You can know if a paginator is over by looking up the .done property.
while (!homeTimeline.done) {
await homeTimeline.fetchNext();
}You can fetch the next page of the collection with the method .fetchNext().
await homeTimeline.fetchNext();Items will be added to current instance and next calls to .tweets or Symbol.iterator will include previously fetched tweets and new tweets.
Once the paginator is .done, .fetchNext() and .fetchLast() do nothing and send no request.
Search paginators (.search() and .searchAll()) still send one, to ask for older tweets with until_id.
Alternatively, you can use .next() to fetch a next page and get it as a separator instance.
const timelinePage2 = await homeTimeline.next();Next page items will be only available on timelinePage2.tweets or through timelinePage2[Symbol.iterator]. New instance will not have items of previous page.
If there is no next page, .next() gives an empty instance, without a request.
If you want to fetch the maximum of tweets or iterate though the whole possible collection, you can use async iterator or .fetchLast().
// Will fetch 1000 more tweets (automatically split into separate requests),
// except if the rate limit is hit or if no more results are available
await homeTimeline.fetchLast(1000);Alternatively, you can achieve this behaviour with async iterators:
// Note presence of 'await' here
for await (const tweet of homeTimeline) {
// Iterate until rate limit is hit
// or API calls returns no more results
}Page size: each request returns one page, and max_results sets its size. X's default is small (10 tweets for search), so pass it if you want more.
.fetchLast() and async iteration ask for 100 items per page.
A page can have fewer items than max_results even when more results exist: use .done to know if the collection is over.
const nodeTweets = await client.v2.search('nodeJS', { max_results: 100 });Cost: the X API is pay-per-use, and reads are billed per item returned (see X pricing).
Async iteration and .fetchLast() without a count keep fetching until there are no more results or your rate limit is hit, which can be a lot of items.
Give .fetchLast() a count, break out of your for await loop when you have enough, and set a spending limit in the Developer Console.
Resuming later: v2 paginators expose the token of the next page in .meta.next_token.
Store it, and give it back to the same method to continue from there: as pagination_token for most endpoints, and as next_token for search.
const timeline = await client.v2.userTimeline('12');
const savedToken = timeline.meta.next_token;
// Later
const nextTimelinePage = await client.v2.userTimeline('12', { pagination_token: savedToken });
const search = await client.v2.search('nodeJS');
const nextSearchPage = await client.v2.search('nodeJS', { next_token: search.meta.next_token });For tweets endpoints that returns metas and includes in their payload, v2 paginators supports them (and merge them into a unique container :D),
just use Paginator.meta or Paginator.includes.
.includes is an accessor to a TwitterV2IncludesHelper instance. See how to use it here.
const mySearch = await client.v2.search('nodeJS');
for await (const tweet of mySearch) {
const availableMeta = mySearch.meta;
const availableIncludes = mySearch.includes;
// availableMeta and availableIncludes are filled with .meta and .includes
// fetched at the time you were reading this tweet
// Once the next page is automatically fetched, they can be updated!
}In some cases, paginators in v2 API can contains errors. You can access then with .errors getter.
On paginators that supports it, you can get previous pages with .previous() and .fetchPrevious().
Their behaviour is comparable as .next() and .fetchNext().