---
title: Cursor Pagination
description: Deep dive into cursor-based pagination for consistent, scalable navigation through large datasets in TraceRTM
---

# Cursor Pagination

Cursor-based pagination provides consistent, reliable navigation through large datasets by using opaque cursor tokens instead of numeric offsets. This approach ensures stable results even when data changes during pagination.

## Why Cursor Pagination?

### Advantages Over Offset Pagination

| Aspect | Cursor | Offset |
|--------|--------|--------|
| Consistency | Stable during concurrent changes | May skip/duplicate items |
| Performance | O(1) lookup | O(n) for large offsets |
| Scalability | Excellent for large datasets | Degrades with size |
| Real-time data | Handles inserts/deletes | Affected by changes |
| Random access | Sequential only | Jump to any page |

### When to Use Cursor Pagination

- **Large datasets** (>10,000 items)
- **Real-time data** that changes frequently
- **Infinite scroll** interfaces
- **API integrations** requiring consistency
- **Background sync** operations

---

## How Cursors Work

### Cursor Structure

Cursors are opaque, base64-encoded tokens containing position information:

```
eyJpZCI6Iml0ZW1fcmVxXzA1MCIsInMiOiJjcmVhdGVkX2F0IiwiZCI6ImRlc2MiLCJ2IjoiMjAyNC0wMS0yMFQxMDowMDowMFoifQ
```

When decoded (for illustration only - treat as opaque):

```json
{
  "id": "item_req_050",
  "s": "created_at",
  "d": "desc",
  "v": "2024-01-20T10:00:00Z"
}
```

### Cursor Components

| Component | Description |
|-----------|-------------|
| `id` | Last item's unique identifier |
| `s` | Sort field(s) |
| `d` | Sort direction |
| `v` | Sort field value(s) for tie-breaking |

---

## Basic Usage

### First Page Request

```bash
curl "https://api.tracertm.dev/v1/projects/proj_abc123/items?limit=20" \
  -H "Authorization: Bearer rtm_live_sk_..."
```

### Response with Cursors

```json
{
  "data": [
    {"id": "item_req_001", "title": "User Authentication", "created_at": "2024-01-20T10:00:00Z"},
    {"id": "item_req_002", "title": "Password Reset", "created_at": "2024-01-20T09:30:00Z"},
    {"id": "item_req_003", "title": "Session Management", "created_at": "2024-01-20T09:00:00Z"}
  ],
  "pagination": {
    "has_next": true,
    "has_prev": false,
    "next_cursor": "eyJpZCI6Iml0ZW1fcmVxXzAyMCIsInMiOiJjcmVhdGVkX2F0IiwiZCI6ImRlc2MifQ",
    "prev_cursor": null,
    "count": 20
  }
}
```

### Next Page Request

```bash
curl "https://api.tracertm.dev/v1/projects/proj_abc123/items?limit=20&cursor=eyJpZCI6Iml0ZW1fcmVxXzAyMCIsInMiOiJjcmVhdGVkX2F0IiwiZCI6ImRlc2MifQ" \
  -H "Authorization: Bearer rtm_live_sk_..."
```

### Previous Page Request

```bash
curl "https://api.tracertm.dev/v1/projects/proj_abc123/items?limit=20&cursor=eyJpZCI6Iml0ZW1fcmVxXzAyMSIsInMiOiJjcmVhdGVkX2F0IiwiZCI6ImRlc2MifQ&direction=prev" \
  -H "Authorization: Bearer rtm_live_sk_..."
```

---

## Pagination Parameters

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `limit` | integer | 20 | Items per page (1-100) |
| `cursor` | string | null | Cursor from previous response |
| `direction` | string | `next` | `next` or `prev` |

## Response Fields

| Field | Type | Description |
|-------|------|-------------|
| `has_next` | boolean | More items available after current page |
| `has_prev` | boolean | More items available before current page |
| `next_cursor` | string | Cursor for next page (null if no more) |
| `prev_cursor` | string | Cursor for previous page (null if at start) |
| `count` | integer | Number of items in current response |

---

## Cursor Pagination with Sorting

### Single Sort Field

```bash
curl "https://api.tracertm.dev/v1/projects/proj_abc123/items?sort=priority&order=desc&limit=20" \
  -H "Authorization: Bearer rtm_live_sk_..."
```

### Multiple Sort Fields

```bash
curl "https://api.tracertm.dev/v1/projects/proj_abc123/items?sort=status,priority,created_at&order=asc,desc,desc&limit=20" \
  -H "Authorization: Bearer rtm_live_sk_..."
```

### Response with Multi-Sort Cursor

```json
{
  "data": [...],
  "pagination": {
    "has_next": true,
    "has_prev": false,
    "next_cursor": "eyJpZCI6Iml0ZW1fcmVxXzAyMCIsInMiOlsic3RhdHVzIiwicHJpb3JpdHkiLCJjcmVhdGVkX2F0Il0sImQiOlsiYXNjIiwiZGVzYyIsImRlc2MiXSwidmFsdWVzIjp7InN0YXR1cyI6ImFwcHJvdmVkIiwicHJpb3JpdHkiOiJoaWdoIiwiY3JlYXRlZF9hdCI6IjIwMjQtMDEtMjBUMTA6MDA6MDBaIn19",
    "prev_cursor": null
  }
}
```

---

## Cursor Pagination with Filtering

Cursors remain valid when used with the same filters:

```bash
# First page with filters
curl "https://api.tracertm.dev/v1/projects/proj_abc123/items?type=requirement&status=approved&limit=20" \
  -H "Authorization: Bearer rtm_live_sk_..."

# Next page with same filters
curl "https://api.tracertm.dev/v1/projects/proj_abc123/items?type=requirement&status=approved&limit=20&cursor=eyJ..." \
  -H "Authorization: Bearer rtm_live_sk_..."
```

### Filter Consistency Rule

Cursors are **only valid with the same filters** used to create them. Changing filters invalidates the cursor:

```bash
# This will return an error - filter changed
curl "https://api.tracertm.dev/v1/projects/proj_abc123/items?type=test_case&limit=20&cursor=eyJ..." \
  -H "Authorization: Bearer rtm_live_sk_..."
```

```json
{
  "error": {
    "code": "INVALID_CURSOR",
    "message": "Cursor is invalid for current filter parameters",
    "details": {
      "cursor_filters": {"type": "requirement", "status": "approved"},
      "request_filters": {"type": "test_case"}
    }
  }
}
```

---

## SDK Examples

### Python

```python
from tracertm import TraceRTMClient

client = TraceRTMClient(api_key="rtm_live_sk_...")

# Basic cursor pagination
def fetch_all_items(project_id, page_size=50):
    """Fetch all items using cursor pagination"""
    all_items = []
    cursor = None

    while True:
        response = client.items.list(
            project_id=project_id,
            limit=page_size,
            cursor=cursor
        )

        all_items.extend(response.data)

        if not response.pagination.has_next:
            break

        cursor = response.pagination.next_cursor

    return all_items

# Generator for memory efficiency
def iterate_items(project_id, page_size=50, filters=None):
    """Generator that yields items one at a time"""
    cursor = None

    while True:
        response = client.items.list(
            project_id=project_id,
            limit=page_size,
            cursor=cursor,
            **(filters or {})
        )

        for item in response.data:
            yield item

        if not response.pagination.has_next:
            break

        cursor = response.pagination.next_cursor

# Usage with filters
for item in iterate_items("proj_abc123", filters={"type": "requirement", "status": "approved"}):
    process_item(item)

# Bidirectional navigation
class ItemNavigator:
    def __init__(self, client, project_id, page_size=20):
        self.client = client
        self.project_id = project_id
        self.page_size = page_size
        self.current_page = []
        self.next_cursor = None
        self.prev_cursor = None
        self.has_next = False
        self.has_prev = False

    def load_first_page(self):
        response = self.client.items.list(
            project_id=self.project_id,
            limit=self.page_size
        )
        self._update_state(response)
        return self.current_page

    def next_page(self):
        if not self.has_next:
            return None

        response = self.client.items.list(
            project_id=self.project_id,
            limit=self.page_size,
            cursor=self.next_cursor,
            direction="next"
        )
        self._update_state(response)
        return self.current_page

    def prev_page(self):
        if not self.has_prev:
            return None

        response = self.client.items.list(
            project_id=self.project_id,
            limit=self.page_size,
            cursor=self.prev_cursor,
            direction="prev"
        )
        self._update_state(response)
        return self.current_page

    def _update_state(self, response):
        self.current_page = response.data
        self.next_cursor = response.pagination.next_cursor
        self.prev_cursor = response.pagination.prev_cursor
        self.has_next = response.pagination.has_next
        self.has_prev = response.pagination.has_prev

# Usage
nav = ItemNavigator(client, "proj_abc123")
page1 = nav.load_first_page()
page2 = nav.next_page()
page1_again = nav.prev_page()
```

### TypeScript

```typescript
import { TraceRTMClient, Item, PaginatedResponse } from '@tracertm/sdk';

const client = new TraceRTMClient({ apiKey: 'rtm_live_sk_...' });

// Async generator for cursor pagination
async function* iterateItems(
  projectId: string,
  pageSize = 50,
  filters?: Record<string, unknown>
): AsyncGenerator<Item> {
  let cursor: string | undefined;

  while (true) {
    const response = await client.items.list({
      projectId,
      limit: pageSize,
      cursor,
      ...filters
    });

    for (const item of response.data) {
      yield item;
    }

    if (!response.pagination.hasNext) {
      break;
    }

    cursor = response.pagination.nextCursor;
  }
}

// Usage
async function processAllItems(projectId: string) {
  for await (const item of iterateItems(projectId, 50, { type: 'requirement' })) {
    await processItem(item);
  }
}

// Bidirectional pagination class
class CursorPaginator<T> {
  private currentData: T[] = [];
  private nextCursor?: string;
  private prevCursor?: string;
  private _hasNext = false;
  private _hasPrev = false;

  constructor(
    private fetchFn: (cursor?: string, direction?: 'next' | 'prev') => Promise<PaginatedResponse<T>>,
    private pageSize: number = 20
  ) {}

  get data(): T[] {
    return this.currentData;
  }

  get hasNext(): boolean {
    return this._hasNext;
  }

  get hasPrev(): boolean {
    return this._hasPrev;
  }

  async loadFirst(): Promise<T[]> {
    const response = await this.fetchFn(undefined, 'next');
    this.updateState(response);
    return this.currentData;
  }

  async next(): Promise<T[] | null> {
    if (!this._hasNext || !this.nextCursor) return null;

    const response = await this.fetchFn(this.nextCursor, 'next');
    this.updateState(response);
    return this.currentData;
  }

  async prev(): Promise<T[] | null> {
    if (!this._hasPrev || !this.prevCursor) return null;

    const response = await this.fetchFn(this.prevCursor, 'prev');
    this.updateState(response);
    return this.currentData;
  }

  private updateState(response: PaginatedResponse<T>) {
    this.currentData = response.data;
    this.nextCursor = response.pagination.nextCursor;
    this.prevCursor = response.pagination.prevCursor;
    this._hasNext = response.pagination.hasNext;
    this._hasPrev = response.pagination.hasPrev;
  }
}

// Usage
const paginator = new CursorPaginator(
  (cursor, direction) => client.items.list({
    projectId: 'proj_abc123',
    limit: 20,
    cursor,
    direction
  })
);

await paginator.loadFirst();
console.log(paginator.data);  // First page
await paginator.next();
console.log(paginator.data);  // Second page
```

### Go

```go
package main

import (
    "context"
    "fmt"
    "github.com/tracertm/tracertm-go"
)

// ItemIterator provides cursor-based iteration
type ItemIterator struct {
    client    *tracertm.Client
    projectID string
    pageSize  int
    filters   map[string]interface{}

    items     []tracertm.Item
    index     int
    cursor    string
    hasMore   bool
    err       error
}

func NewItemIterator(client *tracertm.Client, projectID string, pageSize int, filters map[string]interface{}) *ItemIterator {
    return &ItemIterator{
        client:    client,
        projectID: projectID,
        pageSize:  pageSize,
        filters:   filters,
        hasMore:   true,
    }
}

func (it *ItemIterator) Next(ctx context.Context) bool {
    // Return next item from current page
    if it.index < len(it.items) {
        it.index++
        return true
    }

    // Fetch next page if available
    if !it.hasMore {
        return false
    }

    response, err := it.client.Items.List(ctx, &tracertm.ListItemsRequest{
        ProjectID: it.projectID,
        Limit:     it.pageSize,
        Cursor:    it.cursor,
        Filters:   it.filters,
    })
    if err != nil {
        it.err = err
        return false
    }

    it.items = response.Data
    it.index = 0
    it.cursor = response.Pagination.NextCursor
    it.hasMore = response.Pagination.HasNext

    if len(it.items) == 0 {
        return false
    }

    it.index++
    return true
}

func (it *ItemIterator) Item() tracertm.Item {
    if it.index > 0 && it.index <= len(it.items) {
        return it.items[it.index-1]
    }
    return tracertm.Item{}
}

func (it *ItemIterator) Err() error {
    return it.err
}

func main() {
    client := tracertm.NewClient("rtm_live_sk_...")
    ctx := context.Background()

    // Use iterator
    iter := NewItemIterator(client, "proj_abc123", 50, map[string]interface{}{
        "type": "requirement",
    })

    for iter.Next(ctx) {
        item := iter.Item()
        fmt.Printf("Processing: %s\n", item.Title)
    }

    if err := iter.Err(); err != nil {
        panic(err)
    }

    // Bidirectional navigation
    nav := &CursorNavigator{
        client:    client,
        projectID: "proj_abc123",
        pageSize:  20,
    }

    page1, _ := nav.LoadFirst(ctx)
    page2, _ := nav.NextPage(ctx)
    page1Again, _ := nav.PrevPage(ctx)
}

type CursorNavigator struct {
    client     *tracertm.Client
    projectID  string
    pageSize   int

    Current    []tracertm.Item
    nextCursor string
    prevCursor string
    HasNext    bool
    HasPrev    bool
}

func (n *CursorNavigator) LoadFirst(ctx context.Context) ([]tracertm.Item, error) {
    response, err := n.client.Items.List(ctx, &tracertm.ListItemsRequest{
        ProjectID: n.projectID,
        Limit:     n.pageSize,
    })
    if err != nil {
        return nil, err
    }

    n.updateState(response)
    return n.Current, nil
}

func (n *CursorNavigator) NextPage(ctx context.Context) ([]tracertm.Item, error) {
    if !n.HasNext {
        return nil, nil
    }

    response, err := n.client.Items.List(ctx, &tracertm.ListItemsRequest{
        ProjectID: n.projectID,
        Limit:     n.pageSize,
        Cursor:    n.nextCursor,
        Direction: "next",
    })
    if err != nil {
        return nil, err
    }

    n.updateState(response)
    return n.Current, nil
}

func (n *CursorNavigator) PrevPage(ctx context.Context) ([]tracertm.Item, error) {
    if !n.HasPrev {
        return nil, nil
    }

    response, err := n.client.Items.List(ctx, &tracertm.ListItemsRequest{
        ProjectID: n.projectID,
        Limit:     n.pageSize,
        Cursor:    n.prevCursor,
        Direction: "prev",
    })
    if err != nil {
        return nil, err
    }

    n.updateState(response)
    return n.Current, nil
}

func (n *CursorNavigator) updateState(response *tracertm.ListItemsResponse) {
    n.Current = response.Data
    n.nextCursor = response.Pagination.NextCursor
    n.prevCursor = response.Pagination.PrevCursor
    n.HasNext = response.Pagination.HasNext
    n.HasPrev = response.Pagination.HasPrev
}
```

---

## Cursor Expiration

### Cursor Lifetime

| Cursor Type | Lifetime | Description |
|-------------|----------|-------------|
| Standard | 24 hours | Default cursor validity |
| Long-lived | 7 days | For background sync jobs |
| Session | Until logout | Tied to user session |

### Handling Expired Cursors

```json
{
  "error": {
    "code": "CURSOR_EXPIRED",
    "message": "The provided cursor has expired",
    "details": {
      "cursor_created_at": "2024-01-19T10:00:00Z",
      "expired_at": "2024-01-20T10:00:00Z"
    }
  }
}
```

### Best Practice: Cursor Recovery

```python
def paginate_with_recovery(project_id, page_size=50):
    """Paginate with automatic recovery from expired cursors"""
    cursor = None
    last_seen_id = None

    while True:
        try:
            response = client.items.list(
                project_id=project_id,
                limit=page_size,
                cursor=cursor
            )

            for item in response.data:
                yield item
                last_seen_id = item.id

            if not response.pagination.has_next:
                break

            cursor = response.pagination.next_cursor

        except CursorExpiredError:
            # Recover by starting after last seen item
            if last_seen_id:
                response = client.items.list(
                    project_id=project_id,
                    limit=page_size,
                    after_id=last_seen_id  # Resume point
                )
                cursor = response.pagination.next_cursor
            else:
                # No recovery point, restart from beginning
                cursor = None
```

---

## Best Practices

### 1. Always Use Cursors for Large Datasets

```python
# Good: Cursor pagination for large data
for item in client.items.iterate(project_id, method="cursor"):
    process(item)

# Bad: Offset pagination for large data
for offset in range(0, 100000, 50):
    items = client.items.list(project_id, offset=offset)  # Slow!
```

### 2. Don't Store Cursors Long-Term

```python
# Bad: Storing cursor in database
db.save_sync_state(cursor=response.pagination.next_cursor)

# Good: Store item ID for recovery
db.save_sync_state(last_item_id=items[-1].id if items else None)
```

### 3. Handle Cursor Invalidation

```python
try:
    response = client.items.list(project_id, cursor=saved_cursor)
except (CursorExpiredError, InvalidCursorError):
    # Restart pagination
    response = client.items.list(project_id)
```

### 4. Use Consistent Sort/Filter Parameters

```python
# Good: Same parameters with cursor
params = {"type": "requirement", "sort": "created_at", "order": "desc"}

page1 = client.items.list(project_id, **params)
page2 = client.items.list(project_id, cursor=page1.pagination.next_cursor, **params)

# Bad: Changing parameters mid-pagination
page1 = client.items.list(project_id, type="requirement")
page2 = client.items.list(project_id, type="test_case", cursor=page1.pagination.next_cursor)  # Error!
```

---

## Related Documentation

- [Pagination](/docs/api-reference/rest-api/pagination) - Overview of pagination methods
- [Filtering](/docs/api-reference/rest-api/filtering) - Filter results
- [Sorting](/docs/api-reference/rest-api/sorting) - Sort results
