Production-Grade Reliability and Token Economy for Visual AI Agents

Aliax is the production-grade reliability and token-economy layer between your VLM and the live browser. The Compactor strips 90% of DOM noise so your model sees ~1,500 tokens instead of 100,000. The Steady Hand stamps every interactable node with a persistent data-aliax-id so execution is CLS-immune and deterministic. The Airbag ships a visual crash trace to your dashboard the moment an agent stalls — Sentry-style, for agents.

Five steps. You'll have a VLM agent clicking the right element in under five minutes. Code blocks below follow your language pick above — change it any time and every snippet updates in place.

pip install aliax
playwright install chromium

from aliax import Aliax

aliax = Aliax(
    api_key="sk_live_...",
    # Default endpoint — points at the api.aliax.xyz edge.
    # The SDK transparently fails over to a secondary origin
    # if the apex is unreachable, so most users never set this.
    # endpoint="https://api.aliax.xyz/v1",
)

from playwright.async_api import async_playwright

async with async_playwright() as p:
    page = await (await p.chromium.launch()).new_page()
    await page.goto("https://shop.example.com/cart")

    # 1. Translate the live DOM → Set-of-Mark image + JSON node map.
    ctx = await aliax.parse_ui(page, render_config={"format": "jpeg", "quality": 70})
    #    ctx.image_bytes  — JPEG (default) with numbered Set-of-Mark boxes.
    #    ctx.elements     — [{element_id, tag, bounds, text, ...}, ...]

    # 2. Hand BOTH to your VLM. It replies with a structured action.
    decision = await ask_llm(image=ctx.image_bytes, map=ctx.elements)
    #   → {"action": "CLICK", "element_id": "el_41"}

    # 3. Aliax executes natively — scroll-into-view, React onChange,
    #    iframe coord math, retina DPR, all handled.
    result = await aliax.execute(page, decision)
    print(result)  # {'ok': True, 'action': 'CLICK', 'coords': [905, 150], ...}

# When the agent loops past recovery, dump the state to the queue:
await aliax.capture_failure(
    page,
    goal="Close newsletter popup",
    thoughts=agent.current_reasoning,
    last_attempted_action={"action": "CLICK", "element_id": "el_41"},
    failure_reason="modal_blocked",
    step=agent.loop_step,
)
# Then open /captures to see it land and tap the right element.

The DOM mapper's semantic pruning reduces a typical 100,000-token raw HTML payload to ~1,500 tokens before it reaches your VLM — a 90%+ cost reduction with zero semantic loss on interactable elements. Topology-wise, Aliax is three components: an SDK (Python or Node) that runs inside your agent process, a Cloudflare Worker at the edge, and a Postgres + R2 backend.

The three product pillars

data-aliax-id — the DNA stamp

When parse_ui() runs, the DOM mapper writes a data-aliax-id attribute on every interactable element before the screenshot is taken:

<button data-aliax-id="el_41" aria-label="Add to cart">Add to cart</button>

execute() resolves the target via document.querySelector('[data-aliax-id="el_41"]') — never by coordinate. This makes every click CLS-immune: React re-renders, lazy hydration, ad reflow, and viewport resize cannot shift the target out from under the executor. The stamp is rewritten on every parse_ui() call so it always reflects the live DOM.

Deployment topology

The interceptor flow (parse_ui + execute) is fully local — the Set-of-Mark overlay is painted by the bundled JS layer (position:fixed; pointer-events:none at the top z-stack) inside Chromium, and the screenshot is encoded by Playwright's native C++ CDP path. No Python pixel processing, no network round-trip on the agent's critical path. A tiny non-blocking ping to /v1/telemetry records the call for billing. Only capture_failure() actually uploads a screenshot, and even then via a single multipart POST — if it fails, your agent's flow continues uninterrupted. Screenshots are served directly from the public R2 CDN; the Worker is never on the read path.

The pure, zero-dependency local loop. Import Aliax, call parse_ui(), send the compact payload to your VLM, and call execute(). No frameworks, no orchestrator — just you, Playwright, and your model of choice.

The complete loop

import os
from playwright.async_api import async_playwright
from aliax import Aliax, SYSTEM_INSTRUCTIONS

aliax = Aliax(api_key=os.environ["ALIAX_API_KEY"])
goal = "Add the cheapest in-stock item to the cart and check out."

async with async_playwright() as p:
    page = await (await p.chromium.launch()).new_page()
    await page.goto("https://shop.example.com/cart")

    while True:
        # 1. SEE — compact the live DOM (100k → ~1.5k tokens) + Set-of-Mark image
        ctx = await aliax.parse_ui(page)

        # 2. THINK — your VLM picks an element_id, not a pixel
        decision = await ask_llm(
            image=ctx.image_bytes,
            elements=ctx.elements,
            system=SYSTEM_INSTRUCTIONS,
        )
        if decision["action"] == "DONE":
            break

        # 3. ACT — CLS-immune execution via data-aliax-id
        result = await aliax.execute(page, decision)
        if not result["ok"]:
            # reason is required; Gatekeeper decides whether to escalate.
            await aliax.report_issue(
                page,
                reason="execute_failed",
                goal=goal,
                last_attempted_action=decision,
            )
            break

Aliax connects over any remote CDP WebSocket and runs the DOM mapper inside the cloud sandbox — Browserbase, Steel, Browserless, or your own Chromium-in-a-Worker. Because The Compactor runs in the remote browser, the only payload crossing the wire on each step is the compacted JSON node map (~1.5k tokens) plus the JPEG screenshot — never the raw 100k-token HTML DOM.

1. Mint a Browserbase session (or Steel)

Both providers expose a REST endpoint that returns a CDP URL. Browserbase: POST https://www.browserbase.com/v1/sessions. Steel: POST https://api.steel.dev/v1/sessions. Same shape, swap the host and the auth header.

import os, httpx

async def mint_browserbase_cdp_url() -> str:
    async with httpx.AsyncClient(timeout=10) as client:
        r = await client.post(
            "https://www.browserbase.com/v1/sessions",
            headers={"x-bb-api-key": os.environ["BROWSERBASE_API_KEY"]},
            json={"projectId": os.environ["BROWSERBASE_PROJECT_ID"]},
        )
        r.raise_for_status()
        return r.json()["connectUrl"]

# Steel variant — identical shape, different host:
#   POST https://api.steel.dev/v1/sessions
#   Header: Steel-Api-Key: <key>
#   Response: { "websocketUrl": "wss://..." }

2. Connect Playwright, drive Aliax

from playwright.async_api import async_playwright
from aliax import Aliax

aliax = Aliax(api_key=os.environ["ALIAX_API_KEY"])
cdp_url = await mint_browserbase_cdp_url()

async with async_playwright() as p:
    browser = await p.chromium.connect_over_cdp(cdp_url)
    try:
        page = await browser.new_page()
        await page.goto("https://target.example.com")

        # The Compactor runs INSIDE the remote browser — no extra round-trip.
        ctx = await aliax.parse_ui(page)
        decision = await ask_llm(image=ctx.image_bytes, elements=ctx.elements)
        await aliax.execute(page, decision)
    finally:
        await browser.close()  # release the Browserbase session

Already happy with Browser Use, Stagehand, or LangChain? You don't have to rewrite your agent's brain — but Aliax is not a passive middleware you can pass as a one-line config either. It is the perception + execution loop. The honest pattern: wire parse_ui() into your framework's per-step hook so Aliax's compact node map (not the raw DOM) is what your LLM actually sees, then drop capture_failure() into your error handler for visual telemetry.

Per-step hook (Browser Use)

Browser Use exposes lifecycle hooks. Call aliax.parse_ui() before each model decision and stash the compact context on the agent state — your LLM step then reads ctx.elements and ctx.image_bytes instead of Browser Use's raw DOM dump.

import os
from browser_use import Agent, AgentHooks
from aliax import Aliax

aliax = Aliax(api_key=os.environ["ALIAX_API_KEY"])

class AliaxHooks(AgentHooks):
    async def on_step_start(self, agent_state, browser_context):
        page = await browser_context.get_current_page()
        ctx = await aliax.parse_ui(page)
        # Compact node map (~1.5k tokens) + Set-of-Mark image.
        # Pass these to your VLM step in place of Browser Use's raw DOM.
        agent_state.aliax_ctx = ctx

agent = Agent(
    task="Find the cheapest non-stop flight from JFK → LHR next Friday.",
    llm=my_llm,
    hooks=AliaxHooks(),
)

await agent.run()

Visual telemetry on error (Stagehand / LangChain / custom)

If your framework throws on a failed action, capture_failure() snapshots the live page and ships it to your annotation queue unconditionally — no Gatekeeper gate, fires on every caught error. Use this for the "something blew up in production, give me the screenshot" path.

try:
    await stagehand.act(instruction)
except Exception as e:
    # Unconditional capture: snapshots the page, posts to your annotation queue.
    await aliax.capture_failure(
        page,
        goal=current_goal,
        thoughts=agent.last_reasoning,
        last_attempted_action={"action": "CLICK", "element_id": e.target},
        failure_reason="stagehand_error",
    )

Gatekeeper-gated stagnation escalation

For loops where the agent thinks it's making progress but is cycling, use report_issue(). The Gatekeeper refuses to escalate until 3 identical DOM hashes or a period-2/3/4 action cycle is proven, so it's safe to call on every step. reason is required.

await aliax.report_issue(
    page,
    reason="suspected_stall",
    goal=current_goal,
    thoughts=agent.last_reasoning,
    last_attempted_action=last_decision,
)

Every request to POST /v1/capture must carry a valid API key. Keys are scoped to a single Aliax workspace (your authenticated user) and resolve to a user_id server-side.

Key format

Keys are minted from 32 bytes of crypto.getRandomValues and rendered into a 32-character lowercase Crockford-style alphabet (digits 2-9, letters a-z excluding l and o) — shown as sk_live_<32 chars> (40 total characters including the prefix). We store only the SHA-256 hash; the plaintext is shown exactly once.

How the SDK sends it

The client adds the key as an Authorization: Bearer <key> header on every request (telemetry and capture). The plaintext token never appears in the request body, never lands in multipart parts, and is never logged at proxies, CDNs, or in Worker request logs. The Worker reads it exclusively from this header — a request that puts the key in a form field will be rejected with 401 missing_api_key.

Rotation & revocation

Revoke from API keys. A revoked key returns 401 revoked_api_key on the next call. Rotate quarterly and on any suspected leak.

A single endpoint. Multipart in, JSON out.

cURL

curl -X POST https://api.aliax.xyz/v1/capture \
  -H "Authorization: Bearer sk_live_..." \
  -H "X-Aliax-SDK-Version: 1.0.2" \
  -H "X-Idempotency-Key: $(uuidgen)" \
  -F "goal=Add the Medium size to cart" \
  -F "ai_thoughts=Targeting the green dismiss button at top-right of modal." \
  -F 'last_attempted_action={"action":"CLICK","target_x":905,"target_y":150,"selector":".modal-close","element_id":"el_88"}' \
  -F "failure_reason=element_not_found" \
  -F "step=3" \
  -F 'viewport={"width":1920,"height":1080,"dpr":2}' \
  -F 'spatial_map=[{"element_id":"el_1","role":"button","label":"Close","bbox":[1820,40,80,40]}]' \
  -F "sdk_version=1.0.2" \
  -F "screenshot=@./snag.jpg;type=image/jpeg"

Limits

Every field is sent as a multipart/form-data part. JSON fields are serialised client-side; the Worker parses them with strict JSON.

v0.2 promotes the agent's chain-of-thought and its last attempted action to first-class fields. This is the data shape that turns a snag into a DPO training pair — the agent's failed move becomes the negative example; the human annotator's tap becomes the positive.

Why it matters

A screenshot tells you what the page looked like. The attempted action tells you what the model tried to do about it. Storing both in the same row lets the dashboard render a pulsing red-X exactly where the agent clicked, cross-reference that pixel against the spatial map, and surface "AI clicked <div> when the real target was <button>Submit</button>" — at a glance.

The attempted-action shape

from dataclasses import dataclass
from typing import Optional

@dataclass
class AttemptedAction:
    action: str                          # "CLICK" | "TYPE" | "SCROLL" | "HOVER" | "NAVIGATE" | ...
    target_x: Optional[int] = None       # CSS pixels, viewport-relative
    target_y: Optional[int] = None
    value: Optional[str] = None          # text typed (TYPE actions)
    selector: Optional[str] = None       # css/xpath the agent THOUGHT it was hitting
    element_id: Optional[str] = None     # the el_<N> id from parse_ui, if any

    def to_dict(self) -> dict: ...       # asdict() with None values stripped

Either pass an AttemptedAction instance or a plain dict/object — the SDK accepts both. Coordinates are CSS pixels in the same coordinate space as spatial_map[].bbox, so the dashboard can do bounding-box lookups directly.

End-to-end example

import os
from aliax import Aliax, AttemptedAction

aliax = Aliax(api_key=os.environ["ALIAX_API_KEY"])

await aliax.capture_failure(
    page=page,
    goal="Close the newsletter modal and continue to checkout.",
    thoughts=agent.current_reasoning,
    last_attempted_action=AttemptedAction(
        action="CLICK",
        target_x=905,
        target_y=150,
        selector=".modal-close",
    ),
    failure_reason="element_not_found",
    step=agent.loop_step,
)

How the dashboard uses it

1. Reads last_attempted_action.target_x / target_y and pins a pulsing red-X marker on the screenshot at the scaled pixel.
2. Runs a point-in-bbox lookup against the spatial map to find which element the agent actually hit (often a wrapper, a backdrop, or empty space).
3. Renders ai_thoughts in the side panel so the annotator sees the model's reasoning, not just the outcome.
4. On annotation, writes back corrected_action alongside the original attempt — a complete (prompt, rejected, chosen) tuple for DPO.

Retina / HiDPI

The SDK reads window.devicePixelRatio inside the same __AliaxCore.mapDOM() call that returns the spatial map, then ships it as viewport.dpr. The dashboard scales the overlay accordingly, so a screenshot taken at 2× on a Retina display still lands the red-X on the right pixel.

201 — success

{
  "status": "success",
  "capture_id": "8f3c2a64-9c2e-4a3b-9d20-1f4e1c6e2b51",
  "screenshot_url": "https://cdn.aliax.xyz/<user_id>/8f3c2a64-….jpg",
  "msg": "Captured."
}

4xx / 5xx — error envelope

{
  "status": "error",
  "code": "invalid_api_key",
  "msg": "Unknown API key"
}

The SDK normalises both shapes into {status, capture_id, msg} so call sites don't need to branch on HTTP status.

The spatial map is the bridge that turns a human tap into a clean training tuple. Each node describes one visible, interactable element. v0.2 wraps the node list in an envelope so DPR travels with the same round-trip.

Envelope shape (v0.2)

{
  "elements": [ /* node[] */ ],
  "viewport": { "dpr": 2, "width": 1920, "height": 1080 }
}

The SDK extracts elements for the spatial_map field and copies viewport.dpr into the top-level viewport object. v0.1 clients that read a bare array still work — the SDK's extractor falls back to result.elements ?? result.

Node shape

{
  "element_id": "el_14",
  "role": "button",
  "label": "Close",
  "bbox": [1820, 40, 80, 40],
  "z": 999,
  "in_viewport": true,
  "frame": "main"
}

Errors are never silent. Every non-2xx response carries a stable code string — log against this, not against the HTTP status.

One API surface, two languages. Every method is identical in behavior — Python uses snake_case, Node/TypeScript uses camelCase. The wire format and the action verbs inside decision objects stay snake_case in both because they are JSON keys, not language identifiers.

pip install aliax
playwright install chromium

from dataclasses import dataclass
from typing import Optional, Union, Mapping, Any, List

@dataclass
class AttemptedAction:
    action: str
    target_x: Optional[int] = None
    target_y: Optional[int] = None
    value: Optional[str] = None
    selector: Optional[str] = None
    element_id: Optional[str] = None

@dataclass
class ParseContext:
    image_bytes: bytes          # JPEG (default) screenshot with numbered SoM boxes
    image_mime: str             # "image/jpeg" or "image/png"
    image_size: tuple           # (width, height)
    elements: List[dict]        # [{element_id, tag, bounds, text, ...}]
    viewport: dict              # {width, height, dpr, scroll_x, scroll_y}
    url: str
    path: str
    title: str
    truncated: bool

class Aliax:
    def __init__(
        self,
        api_key: Optional[str] = None,         # falls back to os.environ["ALIAX_API_KEY"]
        endpoint: str = "https://api.aliax.xyz/v1",
        debug_mode: bool = False,
        redact_selectors: Optional[list] = None,
        check_for_updates: bool = True,
        max_image_dim: int = 0,                # DEPRECATED — no-op since v1.0
        fallback_endpoint: Optional[str] = None,  # auto-set on the public default
    ): ...

    # Async context manager — deterministic httpx pool teardown.
    async def __aenter__(self) -> "Aliax": ...
    async def __aexit__(self, *exc) -> None: ...
    async def aclose(self) -> None: ...

async def parse_ui(
    self,
    page,
    *,
    render_config: Optional[Mapping[str, Any]] = None,
    #     {"format": "jpeg" | "png",   # default "jpeg"
    #      "quality": 30..100}          # default 80, clamped to floor 30
    draw_overlay: bool = True,
    min_size: int = 12,
    max_elements: int = 200,
    # Legacy kwargs — folded into render_config when not provided.
    max_image_dim: Optional[int] = None,   # DEPRECATED no-op (logs once)
    image_format: Optional[str] = None,    # legacy alias for render_config.format
    image_quality: Optional[int] = None,   # legacy alias for render_config.quality
) -> ParseContext: ...                     # raises AliaxInvalidKeyError / AliaxOutOfCreditsError

# --- Pointer / keyboard ---
await aliax.execute(page, {"action": "CLICK",          "element_id": "el_41"})
await aliax.execute(page, {"action": "HOVER",          "element_id": "el_14"})
await aliax.execute(page, {"action": "TYPE",           "element_id": "el_7",  "value": "Nike Shoes"})
await aliax.execute(page, {"action": "TYPE_AND_ENTER", "element_id": "el_7",  "value": "Nike Shoes"})
await aliax.execute(page, {"action": "PRESS",          "element_id": "el_7",  "key": "Enter"})

# --- Scrolling (directional shortcuts scroll ~80% of viewport / container) ---
await aliax.execute(page, {"action": "SCROLL_DOWN"})
await aliax.execute(page, {"action": "SCROLL_UP",   "element_id": "el_22"})  # scroll inside a container
await aliax.execute(page, {"action": "SCROLL_LEFT"})
await aliax.execute(page, {"action": "SCROLL_RIGHT"})
await aliax.execute(page, {"action": "SCROLL",      "dy": 600})              # explicit delta

# --- Navigation / pacing / completion ---
await aliax.execute(page, {"action": "NAVIGATE", "url": "https://example.com"})  # http(s) only — SSRF guard
await aliax.execute(page, {"action": "WAIT",     "ms": 1500})
await aliax.execute(page, {"action": "DONE"})    # also: "FINISH", "NOOP"

# --- Composite verbs ---
await aliax.execute(page, {
    "action": "BATCH_TYPE",
    "inputs": [
        {"element_id": "el_3", "value": "jane@example.com"},
        {"element_id": "el_4", "value": "hunter2"},
    ],
})
await aliax.execute(page, {
    "action": "COMBO",                    # exactly 2 sub-actions, no nesting
    "actions": [
        {"action": "CLICK", "element_id": "el_8"},
        {"action": "TYPE",  "element_id": "el_9", "value": "Hello"},
    ],
})

# --- Escalation (Gatekeeper-gated; rejected unless stagnation is proven) ---
await aliax.execute(page, {
    "action": "REPORT_ISSUE",
    "reason": "submit_unresponsive",
    "context": {
        "expected_outcome": "Navigate to /confirmation",
        "actual_outcome":   "Still on /checkout after 3 identical parses",
    },
})

# --- Raw-coords escape hatch (when you have your own mapper) ---
await aliax.execute(page, {"action": "CLICK", "x": 905, "y": 150})

# Direct capture — always posts to /v1/capture (or writes to tmp in debug_mode)
cap = await aliax.capture_failure(
    page,
    goal="Complete checkout",
    thoughts="The submit button is visible but clicking does nothing.",
    last_attempted_action={"action": "CLICK", "element_id": "el_42"},
    failure_reason="submit_unresponsive",
    step=7,
    context={"expected_outcome": "/confirmation", "actual_outcome": "still on /checkout"},
)
# cap["status"]: "success" | "error"
# cap["capture_id"]: str | None
# cap["screenshot_url"]: str (when success)

# Gatekeeper-gated escalation — rejected unless either
#   (a) three consecutive parse_ui rounds returned the same state hash, OR
#   (b) a period-2/3/4 action cycle was observed across distinct element_ids.
issue = await aliax.report_issue(
    page,
    reason="submit_button_dead_zone",
    expected_outcome="Page navigates to /confirmation after clicking Submit.",
    actual_outcome="Page stays on /checkout. 3 identical parse states observed.",
    goal="complete checkout",
    step=7,
    # force=True,   # bypass the Gatekeeper (developer assertion)
)
if issue["status"] == "rejected":
    print(issue["msg"])     # explains exactly what proof is still missing

# Pure in-memory snapshot — no network call.
status = aliax.billing_status()
# {"locked": bool, "reason": str | None, "balance": int | None, "credits_exhausted": bool}

# Force a fresh server-side balance check (one 0-cost telemetry ping):
await aliax.refresh_billing_status()

# Deterministic teardown — closes the httpx pool.
await aliax.aclose()

# Or use as an async context manager:
async with Aliax(api_key=os.environ["ALIAX_API_KEY"]) as aliax:
    ctx = await aliax.parse_ui(page)
    # ... aclose() runs at scope exit, even on exception

from aliax import (
    AliaxError,             # base — catch this for any SDK error
    AliaxInvalidKeyError,   # 401 — terminal; SDK locks permanently
    AliaxOutOfCreditsError, # 402 after grace loop; self-heals on top-up
)

try:
    await aliax.parse_ui(page)
except AliaxOutOfCreditsError as e:
    print("balance:", e.balance)        # last known balance (negative or None)
except AliaxInvalidKeyError as e:
    print("reason:", e.reason)          # mint a new key
except AliaxError:
    # any other SDK-originated failure
    raise

Aliax never sees raw customer data of your users — but only because you tell it what to blur. PII redaction is opt-in and runs in two layers.

Layer 1 — visual blur

Pass CSS selectors to redact_selectors. Anything matching is given a heavy CSS filter: blur(12px) for the duration of the screenshot, then unblurred.

aliax = Aliax(
    api_key=os.environ["ALIAX_API_KEY"],
    redact_selectors=[
        "[data-pii]",
        ".customer-email",
        "input[name='credit_card']",
    ],
)

Layer 2 — text strip on the spatial map

The DOM mapper runs the same selector list against label values before they leave the page, replacing them with [REDACTED]. Bboxes and roles are preserved so annotators still see the layout.

The SDK ships its DOM mapper bundled in the package — there is no runtime CDN fetch. This is deliberate: silent JS swaps would poison your training data mid-run and CISOs flag runtime JS download into a Chromium sandbox as RCE risk.

Active-degradation notice

On construction (when check_for_updates=True), the SDK pings GET /v1/version with a 1 s timeout. If you're behind, a single warning fires — no exceptions, no retry storm.

Pinning

pip install aliax

Pin in production. The mapper output schema is part of the SDK version contract, so upgrading mid-training would invalidate cached spatial maps.

The capture becomes training data the moment a human taps it — and with v0.2, every annotation produces a complete DPO pair.

1. The capture lands in Captures with status: unannotated.
2. The inspector renders the screenshot with a pulsing red-X at the agent's last_attempted_action.target_x/y (scaled by viewport.dpr), the goal, and ai_thoughts in the side panel.
3. The platform cross-references the red-X against the spatial map and surfaces which element the agent actually hit — usually a wrapper, backdrop, or empty space.
4. The annotator taps the element the agent should have hit. We snap by point-in-bbox + z-index and prompt for the action type (CLICK, TYPE, HOVER, SCROLL).
5. A corrected_action JSON blob is written back alongside the original last_attempted_action, and status flips to annotated.

Output shape (DPO-ready)

{
  "task_id": "8f3c2a64-9c2e-4a3b-9d20-1f4e1c6e2b51",
  "original_goal": "Add the Medium size to cart",
  "ai_thoughts": "Targeting the green dismiss button at top-right of modal.",
  "last_attempted_action": {
    "action": "CLICK",
    "target_x": 905,
    "target_y": 150,
    "selector": ".modal-close",
    "element_id": "el_88",
    "failure_reason": "element_not_found",
    "step": 3
  },
  "ai_attempt_hit_element_id": "el_88",
  "corrected_action": {
    "action": "CLICK",
    "target_element_id": "el_14",
    "target_role": "button",
    "target_label": "Close",
    "value": null,
    "annotated_by": "auth0|6710f1c2…",
    "annotated_at": "2026-06-22T14:03:11.482Z"
  },
  "viewport": { "width": 1920, "height": 1080, "dpr": 2 }
}

Negative example (last_attempted_action) + positive example (corrected_action) in the same row. Stream this straight into a DPO training loop.

Aliax is pay-as-you-go. No plans, no seats, no expiration. One parse_ui() call = one credit.

How the SDK sees your balance

The first parse_ui() after process start carries a telemetry ping; the response includes your current balance and stamps the in-memory BillingStatus the SDK exposes via aliax.billing_status() / aliax.billingStatus().

• While balance > 0 telemetry is fire-and-forget — sub-millisecond overhead per parse.
• When balance ≤ 0 the SDK awaits the next ping so a top-up flips the kill-switch off within one parse, and a still-empty wallet rejects deterministically with AliaxOutOfCreditsError instead of leaking work.
• refresh_billing_status() / refreshBillingStatus() sends a zero-cost ping if you want to re-check without parsing.

• Write a specific goal. "Add the Medium size to cart" beats "shop". The annotator sees this verbatim — a vague goal becomes a vague training row.
• Capture at the snag, not after. The popup or layout shift that broke your agent must still be on screen. Don't page.reload() first.
• Always send thoughts + last_attempted_action. A capture without them is half a training row. The chain-of-thought is the prompt; the attempted action is the rejected response. Skipping either drops the capture out of your DPO pipeline.
• Coordinates are CSS pixels, viewport-relative. Use the same coordinate space as page.mouse.click(x, y). DPR is captured automatically — don't pre-multiply by it.
• Don't dedupe client-side. Two captures of the same popup from different sessions are signal, not noise — a model that fails twice fails consistently.
• Cap your QPS per agent. aliax.capture_failure() uploads a screenshot — if you're firing >5/s per agent process, you're probably looping on a recoverable error.