> ## Documentation Index
> Fetch the complete documentation index at: https://docs.surfsky.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Scraping API

> Use a Surfsky browser with plain HTTP requests instead of an automation framework.

The Scraping API accepts HTTP requests to load pages in a Surfsky browser and return rendered HTML. It supports waiting for content, screenshots, CAPTCHA solving, and human input emulation without a framework connection.

Use it when you need HTML or a screenshot and do not want to hold a WebSocket connection. For forms, multiple tabs, or click sequences, use a [framework](/quickstart#choose-an-integration) or an [SDK](/sdk).

<span id="using-scraping-api" />

## Prerequisites

Set your [API token and base URL](/quickstart#before-you-start). The examples start a browser with an empty request body, which uses your account's default proxy pool. Proxy, fingerprint, and other start options are in the [API reference](/api-reference/profiles/start-one-time-session).

<span id="code-example" />

## Run an example

The example starts a one-time browser, scrapes a page, prints the result, and stops the browser. To keep cookies and login state between runs, start a [persistent profile](/sessions#persistent-profiles) instead; the scrape call is the same.

<Tabs>
  <Tab title="curl">
    ```bash theme={null}
    INTERNAL_UUID=$(curl -s -X POST "$SURFSKY_API_BASE_URL/profiles/one_time" \
      -H "X-Cloud-Api-Token: $SURFSKY_API_TOKEN" \
      -H "Content-Type: application/json" -d '{}' | jq -r .internal_uuid)

    curl -s -X POST "$SURFSKY_API_BASE_URL/profiles/$INTERNAL_UUID/scrape" \
      -H "X-Cloud-Api-Token: $SURFSKY_API_TOKEN" \
      -H "Content-Type: application/json" \
      -d '{"url": "https://example.com", "wait_for": "h1"}'

    curl -s -X POST "$SURFSKY_API_BASE_URL/profiles/$INTERNAL_UUID/stop" \
      -H "X-Cloud-Api-Token: $SURFSKY_API_TOKEN"
    ```
  </Tab>

  <Tab title="Python">
    ```bash theme={null}
    pip install requests
    ```

    Save as `scrape.py`:

    ```python theme={null}
    import os
    import requests

    base_url = os.environ["SURFSKY_API_BASE_URL"].rstrip("/")
    headers = {"X-Cloud-Api-Token": os.environ["SURFSKY_API_TOKEN"]}

    response = requests.post(f"{base_url}/profiles/one_time", headers=headers, json={}, timeout=120)
    response.raise_for_status()
    session = response.json()

    result = requests.post(
        f"{base_url}/profiles/{session['internal_uuid']}/scrape",
        headers=headers,
        json={"url": "https://example.com", "wait_for": "h1"},
        timeout=130,
    ).json()
    print(result["data"]["status"])
    print(result["data"]["content"])

    requests.post(f"{base_url}/profiles/{session['internal_uuid']}/stop", headers=headers, timeout=120)
    ```

    ```bash theme={null}
    python scrape.py
    ```
  </Tab>
</Tabs>

Expected output: `200` followed by HTML containing `Example Domain`.

<span id="scraping-parameters" />

## Request parameters

`POST /profiles/{internal_uuid}/scrape` accepts:

| Field                | Default              | Meaning                                                                       |
| -------------------- | -------------------- | ----------------------------------------------------------------------------- |
| `url`                | Required             | Page to visit.                                                                |
| `wait_until`         | `"domcontentloaded"` | Navigation condition: `commit`, `domcontentloaded`, `load`, or `networkidle`. |
| `wait_for`           | Omitted              | CSS or XPath selector to wait for after navigation.                           |
| `timeout`            | `30000`              | Navigation and selector timeout, in milliseconds.                             |
| `wait`               | `0`                  | Extra delay after loading, in seconds, up to `60`.                            |
| `screenshot`         | `false`              | Include a Base64 PNG screenshot.                                              |
| `auto_captcha_solve` | `false`              | Solve CAPTCHAs during the scrape. Requires `anti_captcha` at browser start.   |
| `human_actions`      | `0`                  | Random browsing actions before returning, up to `3`.                          |

`wait_for` with a selector for the content you need is more reliable than `networkidle`, which can time out on pages with background traffic. The whole request must finish within 120 seconds.

## Read the response

```json theme={null}
{
  "success": true,
  "msg": "",
  "data": {
    "url": "https://example.com/",
    "status": 200,
    "status_text": "OK",
    "content": "<!doctype html><html>...</html>",
    "cookies": [],
    "screenshot": "iVBORw0KGgo..."
  }
}
```

`success` is about the Surfsky request. `data.status` is the target site's HTTP status, so a `200` from Surfsky can carry a `403` or a challenge page from the site. Check both. `screenshot` is present only when requested; decode it with Base64 to get a PNG.

<span id="batch-scraping" />

<span id="request-queue-and-rate-limits" />

## Scrape several pages

Send an array of request objects to the same endpoint:

```json theme={null}
[
  {"url": "https://example.com", "wait_for": "h1"},
  {"url": "https://example.org", "screenshot": true}
]
```

Pages run one after another in the same browser and share its cookies. `data` comes back as an array in request order; failed items carry `error` and `status_code` instead of a page. Keep a batch within the 120-second limit. For parallel work, use several browsers.

<span id="important-notes" />

## Stop the session

Stop the browser through the API with the `internal_uuid` from the start response, as the examples do:

```bash theme={null}
curl --fail-with-body -X POST \
  "$SURFSKY_API_BASE_URL/profiles/INTERNAL_UUID/stop" \
  -H "X-Cloud-Api-Token: $SURFSKY_API_TOKEN"
```

Otherwise the browser stops after the inactivity timeout, 30 seconds by default. Do not mix the Scraping API with CDP automation in the same browser; the scraper takes over the page and closes extra tabs.