stream execution engine

Turn any API into a reliable stream.

Retry, adaptive windowing, cursor pagination, and cost budgets handled internally. Your code sees only clean, ordered batches - in any language.

429 → shrink window5xx → exp. backoff200 → window growsprefetch: 1 ahead

↓ Explore the libraries

Stream Pipeline

live

Adapters

generic_rest

Cursor-paginated

datadog_metrics

Time-window

openai

Chat completions

anthropic

Messages API

Outbound HTTP Request

HTTP→cursor={token} · page_size=100 · auth=Bearer…

adapter.build_request(cursor) → transport.execute(req)

Response Router

200 OK

① parse body

② advance cursor

③ deliver batch

window × 1.5 ▲

429

read Retry-After

add ±10% jitter

window × 0.5 ▼

attempt+1, retry

5xx / Net

exp. backoff ×2

retry n / max

if n = max →

STREAM_ERROR

apiexec StreamEngineABI v1

Prefetch buffer (depth=1)

#42 ▸ ready

#43 ··· fetch

Cursor state

eyJhbGciOiJIUzI1N…

retries0/ max 3

Adaptive window (grows ×1.5 / shrinks ×0.5)3 600 ms

Your Code

streaming

for batch in stream:# any language
process(batch)# no retry logic needed

Open source

Libraries

dd_stream

beta

A vendor-agnostic, adaptive query execution engine for reliable large-scale API data retrieval. Built-in adapters for Datadog v1/v2, page-number REST, and token-based cursor APIs, with bindings for Python, Node.js, Go, Java, and Rust.

PythonNode.jsGoJavaRust

apiexec

beta

An adaptive execution layer that sits between applications and APIs/AI systems, turning unreliable API calls into a deterministic streaming data retrieval model. A query planner for APIs: callers declare what they want, and the engine handles how to fetch it reliably, efficiently, and within cost/rate constraints.

PythonNode.jsGoJavaRust