Control Plane

The Control Plane is the core of Inferable, it performs all LLM calls managing state, routing and fault-tolerance. When a new Run or Message is received, the Control Plane will trigger a ReAct agent to handle the request with the ability to call any available functions.

The Control Plane maintains context over all functions which have been registered by the SDKs, and routes function calls to the appropriate machines for execution.

The Control Plane does not host or execute these functions, instead it manages a queue of function calls and handles routing them to the appropriate machine for execution.

​

Comparison with Agent frameworks

Most agent frameworks are built with the expectation that your model calls and functions execution will be co-located within the same process.

Generally, when the model returns intention to call a tool, the agent library will execute the registered handler in-band by calling the function.

Inferable’s distributed approach provides a number of benefits:

Language Agnostic

Concerns around model calls, etc are handled within the Control Plane. SDKs are only responsible for polling the Control Plane, executing function handlers and returning the results.

This allows Inferable to support a more varied selection of languages as the SDKs do not need to re-implement the same logic.

Durable Execution

The Control Plane maintains state and can continue from any point in the lifecycle in case of failure, including model provider, function execution or Control Plane host failures.

This also allows Inferable to support features like indefinite pause and resume for Runs (Human in the loop).

​

Related Pages