Skip to content

Resource Engine

Overview

ResourceEngine is the central manager for provisioning, acquiring, releasing, and ending resources across backends (for example local, ray, aws, and k8s). It maintains pooled free resources and id-bound acquired resources to support reuse and isolation in multi-rollout execution.

API Reference

ResourceEngine

Single entry point for resource management.

Holds one store: free resources keyed by (spec, backend). Any resource type (container, vLLM, etc.) lives in this store. The engine creates resources via runners and arranges acquire/release/start/end directly.

Functions

monitor() -> Dict[str, Dict[str, int]] async classmethod

Return a statistics dictionary of existing resource counts per pool.

Returns:

  • Dict[str, Dict[str, int]] –

    Dict mapping pool_key -> {"total": N, "free": F, "acquired": A}.

  • Dict[str, Dict[str, int]] –

    Only includes pool keys that have at least one resource.

start(spec: BaseResourceSpec, size: int = 1, backend: str = 'local', **kwargs: Any) -> None async classmethod

Ensure at least size free resources for (spec, backend). Creates resources via the backend runner and adds them to the store.

acquire(id: str, spec: BaseResourceSpec, backend: str = 'local', timeout: Optional[float] = 600.0) -> BaseResource async classmethod

Acquire a resource for the given id. Same id returns the same resource.

Semantics: - Prefer taking from the free queue for (spec, backend). - If none free and under max_global_num, create a new resource (slot reserved under lock). - If at max_global_num and none free, wait until another thread releases one.

Parameters:

  • timeout (Optional[float], default: 600.0 ) –

    Max seconds for backend start_resource() only. Waiting for an existing/pending resource is unbounded.

release(id: str) -> None async classmethod

Return resource to the store and unregister from acquired.

reset(id: str, *args: Any, **kwargs: Any) -> Any async classmethod

Reset resource state for the given id.

Parameters:

  • id (str) –

    Identifier used when acquiring the resource.

get_status(id: str) -> ResourceStatus async classmethod

Current execution/lifecycle status for the resource associated with id.

Parameters:

  • id (str) –

    Identifier used when acquiring the resource.

control(id: str, **kwargs: Any) -> None async classmethod

Update resource limits for the resource associated with id where supported.

Parameters:

  • id (str) –

    Identifier used when acquiring the resource.

end(id: str) -> None async classmethod

End/kill the resource associated with id.

Removes it from the acquired map and does not return it to the free store.

Parameters:

  • id (str) –

    Identifier used when acquiring the resource.

close() -> None async classmethod

Shut down: end all free and acquired resources, clear store.