Scrape API.

01Overview

The Scrape API is the workhorse endpoint. Give it a URL, optionally describe how you want it fetched, and it returns:

• The final HTML, after redirects and JS execution if requested.
• A structured data object if you supplied an extract schema.
• An optional screenshot (PNG, JPEG, or PDF).
• An optional markdown rendering for LLM ingestion.
• Telemetry — latency, proxy used, credits charged, retries, and a trace_id.

02Endpoint

Use GET when all your parameters fit comfortably in a URL (under 2 KB total) — it's marginally faster and trivially idempotent. Use POST for everything else, especially when you're passing an extract schema, custom headers, a session blob, or a JS pre-script.

03Minimal request

curl https://api.hypedata.io/v1/scrape \
  -H "Authorization: Bearer $HYPEDATA_API_KEY" \
  -G --data-urlencode "url=https://example.com"

const page = await hd.scrape({
  url: "https://example.com"
});

page = hd.scrape(url="https://example.com")

page, err := hd.Scrape(ctx, &hypedata.ScrapeReq{URL: "https://example.com"})

04Request parameters

Core

Rendering

Proxy & geo

Output controls

Operational

05Rendering

When render: true, Hypedata routes the request to a Chromium pool, loads the page like a real browser, executes JavaScript, and returns the post-render HTML — exactly what a human would see.

Wait strategies

Picking the right wait_until is the difference between a fast scrape and a flaky one.

For SPAs that render after a known element appears, use wait_for instead. It's much more reliable than a fixed delay.

Pre-execution script

The script parameter accepts arbitrary JS that runs after the page loads. Useful for clicking "Show more", scrolling, or filling forms.

{
  "url": "https://news.example.com/articles",
  "render": true,
  "script": "for (let i=0;i<5;i++){window.scrollTo(0,document.body.scrollHeight); await new Promise(r=>setTimeout(r,500));}",
  "wait_until": "networkidle2"
}

06Proxy & geo

By default Hypedata routes through datacenter IPs — fast, cheap, and good enough for most sites. Switch to residential for sites that fingerprint at the network layer (most major retailers, ticketing, classifieds), and mobile for the strictest targets (some banks, telco self-service portals).

Targeting examples

// US-only residential, NYC if available
{ "url": "…", "proxy_type": "residential", "country": "us", "city": "new-york" }

// France via Free Mobile (AS12322)
{ "url": "…", "proxy_type": "mobile", "country": "fr", "asn": 12322 }

See the full Proxies guide for credit cost per tier, success-rate benchmarks, and decision flowchart.

07Sessions

Many flows require multiple requests from the same identity — login, then dashboard, then a detail page. Pass the same session string to all of them and Hypedata will:

• Pin the proxy to the same exit IP for up to 10 minutes.
• Persist cookies, localStorage, and sessionStorage.
• Reuse the same browser fingerprint across requests.

// 1. Log in
hd.scrape({
  url: "https://target.example/login",
  method: "POST",
  body: "email=…&password=…",
  session: "order-history-42"
});

// 2. Use the authenticated session
hd.scrape({
  url: "https://target.example/orders",
  session: "order-history-42"
});

Session IDs are opaque to Hypedata — use any string, but keep it secret-grade for keys you don't want others on your team to "borrow".

08Screenshots

Available only when render: true. Pass an object:

{
  "url": "https://example.com",
  "render": true,
  "screenshot": {
    "format": "png",         // png | jpeg | pdf
    "full_page": true,        // scroll & stitch
    "width": 1440,
    "height": 900,
    "omit_background": false,
    "clip": { "x":0, "y":0, "w":1440, "h":600 }
  }
}

The response gains a screenshot_url pointing to a signed CDN URL valid for 24 hours (configurable in workspace settings, up to 30 days).

09Inline extraction

Pass an extract object and the response will include a typed data field. Two formats are accepted: plain-English ("schema sketch") and JSON Schema. The plain-English form is more concise; JSON Schema gives you strict validation.

{
  "extract": {
    "title": "string · article headline",
    "author": "string · byline name only",
    "published_at": "string · ISO 8601 date",
    "tags": "array of strings",
    "word_count": "integer"
  }
}

{
  "extract": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "required": ["title", "published_at"],
    "properties": {
      "title":        { "type": "string" },
      "author":       { "type": "string" },
      "published_at": { "type": "string", "format": "date-time" },
      "tags":         { "type": "array", "items": {"type":"string"} },
      "word_count":   { "type": "integer", "minimum": 0 }
    }
  }
}

Full mechanics — confidence scores, citations, multi-pass extraction — live in AI Parser.

10Response shape

Example — extracted product

{
  "status": "ok",
  "url": "https://shop.example.com/p/alpha-coat",
  "final_url": "https://shop.example.com/p/alpha-coat",
  "http_status": 200,
  "data": {
    "name": "Alpha Coat — Tobacco",
    "price": 219.00,
    "currency": "EUR",
    "in_stock": true,
    "images": ["https://cdn.example.com/.../1.jpg", "…"]
  },
  "meta": {
    "latency_ms": 1812,
    "proxy": "residential.us-east",
    "render": true,
    "credits_used": 6,
    "trace_id": "trc_3F2D1A77B0E1"
  }
}

11Credit cost

Every Scrape API request is billed by adding up modifiers:

So a residential + render + extract call costs 1 + 1 + 4 + 5 = 11 credits. You can preview the cost without making the request: POST /v1/scrape/estimate returns the same envelope shape but the data field is null.

12Errors

Common Scrape-specific errors. The full catalog lives in Errors & status codes.