Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions design/00_inference_api_v2/00-preface.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Inference Server API 2.0

The current HTTP API has grown organically to 60+ endpoints and suffers from:

* **Inconsistent authentication.** API keys are accepted via query params, JSON body fields, or middleware — six different patterns across endpoints.
* **Model and resource IDs buried in bodies.** Identifiers travel inside request bodies rather than URLs, which prevents load-balancer-level routing to backends that already have the relevant models loaded.
* **Divergent response formats.** The same model (e.g., object detection) produces structurally different responses when called directly vs. through a workflow — `class` vs `class_name`, flat vs nested prediction structures, presence or absence of parent metadata.
* **Overlapping endpoints.** Multiple ways to do the same thing — e.g., `/infer/workflows/{ws}/{wf}` and `/{ws}/workflows/{wf}`; model-specific paths like `/clip/embed_image` alongside generic `/infer/object_detection`.

## Design principles

* **Resource-identifying URLs.** Every URL encodes the model or workflow resource being addressed, enabling distributed routing without body parsing.
* **Header-only authentication.** API keys always travel in the `Authorization` header, never in bodies or query params.
* **Unified execution path.** Direct model inference accepts the same inputs and produces the same results as a single-step workflow that wraps the same model.
* **One way to do each thing.** No duplicate endpoints. For most operations the API exposes a single, opinionated execution path. The one exception is performance vs. simplicity: where that trade-off is real, we expose both a simple low-entry-bar variant and an advanced high-performance variant. The two variants must never carry different semantics — only different transport.
* **Coexistence.** `v2` mounts alongside `v1` under a `/v2` prefix. `v1` remains in place for backward compatibility throughout the migration window.

## Authorization

Authentication is standardised so that it is both secure and usable without body parsing. Bearer-token auth is the only supported scheme:

```
Authorization: Bearer <api_key>
```

API keys are never accepted in query params or request bodies in `v2`.

---

*Original author of preamble: @Thomas. Modifications introduced by @Paweł.*
32 changes: 32 additions & 0 deletions design/00_inference_api_v2/01-general-api-structure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# API structure

This document lists the top-level shape of the `v2` API. Per-endpoint details are deferred to the subsequent documents.

## Models endpoints

* `POST /v2/models/run` — run model inference.
* `GET /v2/models/interface` — discover the request/response interface of a given model.
* `GET /v2/models/compatibility` — list model architectures compatible with the current server configuration.
* `GET /v2/models/loaded` — list models currently loaded into memory.
* `POST /v2/models/load` — load a given model.
* `DELETE /v2/models/unload` — unload a given model (or all loaded models, when no model is specified).

## Workflows endpoints

* `POST /v2/workflows/run` — run a workflow.
* `POST /v2/workflows/interface` — discover the request/response interface of a workflow. `POST` (rather than `GET`) is required because in-line workflows ship their definition in the request body.
* `POST /v2/workflows/validate` — validate a workflow definition (and optionally check runtime readiness — see `03-workflows.md`).
* `GET /v2/workflows/system/blocks` — describe blocks available on this server.
* `GET /v2/workflows/system/definition-schema` — return the JSON Schema for workflow definitions.
* `GET /v2/workflows/system/engine-versions` — list available engine versions.

## Video stream processing

TBD — specification deferred to a follow-up document.

## Server status

* `GET /v2/server/health` — liveness probe.
* `GET /v2/server/ready` — readiness probe.
* `GET /v2/server/info` — server build and configuration metadata.
* `GET /v2/server/metrics` — Prometheus-compatible metrics export.
Loading