Skip to main content

Incremental Property Search: Cursor Pagination & Search Sessions

Learn how to page through large Property Search result sets reliably with opaque cursors, and how to layer Search Sessions on top to deliver unique properties incrementally — without paying for duplicates across requests.

Written by Charles Parra

The BatchData Property Search API now supports two new ways to retrieve large result sets reliably: cursor pagination and search sessions. Cursor pagination replaces skip/take for stable paging through tens of thousands of properties. Search sessions wrap a named, persistent context around that cursor so a single delivery stream can extend across days, weeks, or months without re-delivering properties you've already received.

When to use each option

Use this decision table to pick the right paging mode for your integration:

Your situation

Use this

Pulling fewer than ~500 properties in a single call

skip / take (the existing pattern works fine)

Paging through tens of thousands of results in one session, same script run

useCursorPagination: true with pageCursor

Delivering properties incrementally across multiple days, weeks, or months — and you must NOT re-deliver a property you've already received

searchSession

Monthly marketing lists, daily CRM deltas, multi-day backfills with resume-on-failure

searchSession

If you are paginating today with skip and take deeper than ~1,000 results, cursor pagination is strictly better and we recommend switching.

Why skip / take falls short for large result sets

The traditional skip/take pattern works against a moving index. As BatchData ingests property updates, the document at skip: 1000 shifts position — sometimes off the end of one page, sometimes onto the front of the next. The deeper your skip, the worse it gets:

  • Duplicate billing. A property that drifts across a page boundary mid-pagination gets returned (and billed) on two pages.

  • Missed rows. A property that shifts the other way slips between two pages entirely.

  • Slower large pages. The engine has to count past every skipped row, so a skip: 50000 request is meaningfully slower than skip: 0.

Cursor pagination fixes all three problems by giving you a token that says "next time, start exactly after this row" rather than "next time, skip 50,000 rows." The position is anchored to immutable sort values, not row indexes.

Cursor Pagination

A cursor is an opaque, signed string that represents a specific position in your result set. You round-trip it verbatim to the API to retrieve the next page.

First page

Set options.useCursorPagination: true on your first request:

POST /api/v1/property/search
Authorization: Bearer <YOUR_API_TOKEN>{
  "searchCriteria": {
    "address": { "zip": { "equals": "85020" } }
  },
  "options": {
    "take": 100,
    "useCursorPagination": true
  }
}

The response includes a nextPageCursor (and no previousPageCursor, because there is no page before the first):

{
  "status": { "code": 200, "text": "OK" },
  "results": {
    "properties": [ ... 100 items ... ],
    "nextPageCursor": "RPmK5mZrhfAXdh...dfe7c5c9eea27049",
    "meta": {
      "results": {
        "resultCount": 100,
        "resultsFound": 14001
      }
    }
  }
}

Subsequent pages

Pass the previous nextPageCursor value as options.pageCursor:

POST /api/v1/property/search{
  "searchCriteria": {
    "address": { "zip": { "equals": "85020" } }
  },
  "options": {
    "take": 100,
    "useCursorPagination": true,
    "pageCursor": "RPmK5mZrhfAXdh...dfe7c5c9eea27049"
  }
}

Page 2 and onward return BOTH nextPageCursor (advance forward) and previousPageCursor (return to the prior page). Keep paging until nextPageCursor is absent — that means you've reached the end of the result set.

Rules and gotchas

  • Treat cursors as opaque. Do not parse, modify, or generate cursor values yourself. They are signed; any byte-level tampering returns a 400.

  • Cursors are bound to your searchCriteria. Reusing a cursor against a different searchCriteria block returns a 400. If you change criteria, start a new search from page 1.

  • resultsFound is computed once. The total result count is captured on page 1 and carried forward inside the cursor. Subsequent pages return that same value — the index may have grown or shrunk in the interim, but resultsFound will not move during a paging run.

  • Random sort is not supported. useCursorPagination: true cannot be combined with sort.sortOrder: "random" — random sort has no stable position to anchor to.

  • Available on async too. Cursor pagination is fully supported on POST /api/v1/property/search/async. The webhook payload carries results.nextPageCursor and results.previousPageCursor in exactly the same shape as the sync response.

Cursor error messages

All cursor errors return 400 with an actionable message so you can recover:

Trigger

Message

Cursor bytes tampered or unparseable

pageCursor is invalid or has been tampered with. Omit pageCursor to start a new search.

Cursor reused against different searchCriteria

pageCursor was generated for different search criteria. Omit pageCursor to start a new search.

Cursor minted under an older sort configuration

pageCursor is no longer valid; the sort configuration has changed. Omit pageCursor to start a new search.

Combined with random sort

Invalid request. useCursorPagination cannot be used with random sort order.

In all four cases, the recovery is to omit pageCursor and start a fresh search.

Search Sessions

A search session is a named, persistent delivery context layered on top of cursor pagination. When you provide an options.searchSession, BatchData remembers which properties have already been delivered to you for that query and excludes them from subsequent responses — even if those subsequent requests come days or months later.

This is the right tool for any integration that needs to deliver fresh properties incrementally and must NOT pay for the same property twice.

Common use cases

  • Monthly marketing lists. Pull "all properties in Phoenix matching X criteria" once a month and only receive new matches each time.

  • Daily delta delivery into a CRM. A nightly job that adds newly-matching properties to a downstream system without checking for duplicates client-side.

  • Multi-day backfills with resumption. A long-running export job that can stop and resume after a failure without losing position or re-delivering rows.

Token ability requirement

Sessions and the session-management endpoints both require the property-search-sessions ability on your API token. You can grant this in the BatchData Token Management UI. Without this ability, calls to the session-management endpoints return 403 Forbidden.

First request — create a new session

Pick a session id that is meaningful to you — it must match ^[a-zA-Z0-9_-]{1,128}$ and is scoped to your team. Two different teams can use the same session id without interference.

POST /api/v1/property/search{
  "searchCriteria": {
    "address": { "zip": { "equals": "85020" } }
  },
  "options": {
    "take": 500,
    "useCursorPagination": true,
    "searchSession": {
      "id": "monthly-leads-az-85020"
    }
  }
}

The response includes a searchSessionMeta block with isNewSession: true and a running totalDelivered count. Notice what is NOT in the response: nextPageCursor and previousPageCursor are deliberately absent. The session owns the cursor, so exposing one would invite you to bypass the session and risk duplicate delivery.

{
  "status": { "code": 200, "text": "OK" },
  "results": {
    "properties": [ ... 500 items ... ],
    "searchSessionMeta": {
      "isNewSession": true,
      "totalDelivered": 500
    },
    "meta": {
      "results": {
        "resultCount": 500,
        "resultsFound": 14001
      }
    }
  }
}

Subsequent requests — advance the session

Send the same request body again. BatchData looks up the session by id, advances past the properties you've already received, and returns the next batch:

{
  "status": { "code": 200, "text": "OK" },
  "results": {
    "properties": [ ... 500 new items, zero overlap with the first batch ... ],
    "searchSessionMeta": {
      "isNewSession": false,
      "totalDelivered": 1000
    },
    "meta": {
      "results": {
        "resultCount": 500,
        "resultsFound": 14001
      }
    }
  }
}

isNewSession is false from the second request onward. totalDelivered is cumulative.

What stays the same regardless of how you call

  • resultsFound is frozen at the value captured on your first request. It does not move as the underlying index changes during the session's lifetime. If you need a current total, start a fresh session.

  • An empty response page means "caught up — retry later." If no new properties match beyond your session's position, you get 0 properties and totalDelivered unchanged. The session is still valid; pull again tomorrow (or next month) and any newly-indexed matches flow through.

  • take can change mid-session. Pull 500 on day 1, 50 on a retry, and 200 next month — no errors, no duplicates, no lost results. The session stores cursor position, not page size.

  • Forward-only. There is no "previous page" within a session. Going backward would re-deliver properties, which contradicts the entire feature.

  • A property deleted and re-added with the same id is not re-delivered. If you need a current snapshot of a specific property, call the Property Lookup endpoint — that is the canonical "I want the current data" path.

Idempotency keys (optional)

If an upstream system (queue worker, webhook retry path, CI job) might retry the same logical request, supply an idempotencyKey on the session block:

"options": {
  "take": 500,
  "useCursorPagination": true,
  "searchSession": {
    "id": "monthly-leads-az-85020",
    "idempotencyKey": "batch-2026-04-req-001"
  }
}

A retry carrying the same idempotencyKey against a session that has already advanced returns 409 Conflict — the session is not advanced again, and you are not double-charged. For one-shot calls from a script, you can omit the key safely.

Forbidden combinations

Search sessions are intentionally strict about what they pair with. Each of these returns 400 Bad Request:

Combination

Message

searchSession without useCursorPagination: true

Invalid request. searchSession requires useCursorPagination to be true.

searchSession + pageCursor

Invalid request. searchSession cannot be combined with pageCursor.

searchSession + skip > 0

Invalid request. skip and cursor cannot be used together.

searchSession on POST /api/v1/property/search/async

Invalid request. searchSession is not supported on async endpoints.

searchSession with sort.sortOrder: "random"

Invalid request. useCursorPagination cannot be used with random sort order.

The async restriction matters: sessions need a synchronous response to confirm delivery, which async cannot provide.

Per-team limits

  • 100 sessions per team. Creating a 101st returns 403 Forbidden with the message Maximum of 100 search sessions per client-account-id reached. Delete unused sessions via DELETE /v2/property/search-sessions/{id}. The search is not run.

  • Sessions are scoped to your team — they are invisible to other BatchData customers, even if another team happens to use the same session id.

Managing your sessions

Two endpoints let you list and clean up your team's sessions. Both require the property-search-sessions token ability.

List your sessions

GET /api/v2/property/search-sessions?skip=0&take=50
Authorization: Bearer <YOUR_API_TOKEN>

Returns your team's sessions, oldest-first by default. Use skip and take to page through if you have more than 50.

{
  "status": { "code": 200, "text": "OK" },
  "results": {
    "sessions": [
      {
        "id": "monthly-leads-az-85020",
        "totalDelivered": 5000,
        "lastRequestAt": "2026-05-10T14:23:11Z",
        "createdAt": "2026-04-01T09:15:00Z"
      },
      {
        "id": "daily-crm-delta",
        "totalDelivered": 12450,
        "lastRequestAt": "2026-05-12T01:00:00Z",
        "createdAt": "2026-03-15T08:00:00Z"
      }
    ],
    "total": 2,
    "meta": { "requestId": "01KRCV36NPGHBE6FQER77S0XZR" }
  }
}

The total field is your team's overall session count — use it to know whether you're approaching the 100-session cap.

Delete a session

When you're done with a session — or want to free up a slot under the 100-session cap — delete it:

DELETE /api/v2/property/search-sessions/monthly-leads-az-85020
Authorization: Bearer <YOUR_API_TOKEN>

A successful delete returns 200 with deleted: true:

{
  "status": { "code": 200, "text": "OK" },
  "results": {
    "deleted": true,
    "id": "monthly-leads-az-85020",
    "meta": { "requestId": "01KRCV49383QRW53VBJX6CVDCT" }
  }
}

Deleting a session frees its slot immediately and discards all its delivered-id state. A subsequent call to a session with the same id would create a fresh session that starts over from page 1.

If you try to delete a session id that does not exist for your team, you get 404 Not Found. (This is also the response if a different team happens to have a session with that id — by design, so cross-tenant enumeration is not possible.)

Quick reference

Cursor pagination, first page (sync):

{
  "searchCriteria": { ... },
  "options": { "take": 100, "useCursorPagination": true }
}

Cursor pagination, resume (sync):

{
  "searchCriteria": { ... },
  "options": { "take": 100, "useCursorPagination": true, "pageCursor": "<from previous response>" }
}

Cursor pagination on async:

{
  "searchCriteria": { ... },
  "options": {
    "take": 500,
    "useCursorPagination": true,
    "webhookUrl": "<your hook>",
    "errorWebhookUrl": "<your error hook>"
  }
}

Search session, first or subsequent request (sync):

{
  "searchCriteria": { ... },
  "options": {
    "take": 500,
    "useCursorPagination": true,
    "searchSession": { "id": "<your label>", "idempotencyKey": "<optional>" }
  }
}

List your team's sessions:

GET /api/v2/property/search-sessions?skip=0&take=50

Delete one session:

DELETE /api/v2/property/search-sessions/<your label>

Frequently asked questions

Can I switch a long-running pull from skip/take to cursor pagination mid-way through?

No. The cursor encodes a specific position in the result set, so you have to start a fresh search with useCursorPagination: true from page 1. The two pagination modes are not interoperable.

What's the difference between an idempotency key and a session id?

The session id names the long-lived delivery context (e.g. monthly-leads-az-85020). It is reused across many requests. The idempotency key is per-request — it lets a queue worker safely retry a single request without advancing the session twice.

My session has been idle for a few weeks. Is it still valid?

Yes. Sessions do not expire on their own. They occupy one of your 100 session slots until you explicitly delete them.

What happens if a property is deleted from the BatchData index and later re-added?

A session will not re-deliver the property — the delivered-id state survives the deletion. If you specifically need the property's current data, call the Property Lookup endpoint instead.

Do cursor pagination and sessions cost more than skip/take?

No. Each returned property is one billable record, same as today. The features are about avoiding the duplicate-billing and missed-row problems of skip/take, not changing the per-record price.

How do I know which token I'm using has the property-search-sessions ability?

Open the Token Management UI in the BatchData app. Each token shows its granted abilities. If property-search-sessions is unchecked, tick it and save — the change takes effect immediately for new requests.

Related guides

Related Articles
How to perform geo-coordinate searches
How to search comparable properties
Introduction to the Property Search API
Retrieving Paginated Results
Property Monitoring - Getting Started Guide
Did this answer your question?