List Jobs & Pages

Two endpoints for listing your crawl history and paginating through stored page results.

GET/api/crawl

Returns a paginated list of your crawl jobs, newest first.

Query parameters

limitnumberMax results to return. Default: 20. Max: 100.

statusstringFilter by job status (e.g. "completed", "running").

Response200 OK

{
  "success": true,
  "jobs": [{
    "id": "…",
    "url": "https://…",
    "status": "completed",
    "pagesTotal": 42,
    "pagesFinished": 42,
    "creditsUsed": 42
  }]
}

GET/api/crawl/:id/pages

Paginate through all stored pages for a completed job. Heavy content fields (html, markdown, extractedJson) are excluded from list responses — fetch a single page by ID to get full content.

Query parameters

limitMax pages per page. Default: 20. Max: 100.

cursorPagination offset (integer). Use the cursor from the previous response.

statusFilter by page status.

Tip: Use GET /api/crawl/:id/pages/:pageId to fetch a single page with full Markdown, HTML, and extracted JSON content.

Get Job Status Export Data