Architecture
Learn how Inferable works
1. Registering your functions
The Inferable SDK registers your functions with the Control Plane, collecting information such as the function’s input schema, name and description for use by the agent.
2. Control Plane
The Inferable Control Plane will receive the metadata about the registered functions.
At any given time, the Control Plane has a list of all the functions that are registered with Inferable, and the health of the machines that are hosting them.
The Control Plane does not establish a connection to your runtime directly. Instead the SDK establishes a connection with the Control Plane and polls.
3. Executing Runs
When inferable receives a new Run, it is executed using a ReAct-like agent strategy which iteratively reasons about the task and takes actions to achieve the requested outcome. This includes calling the attached functions as necessary.
Inferable performs all model calls and gracefully handles transient errors such as timeouts. The Run is a durable execution construct which supports indefinitely long Function calls and pausing for human interaction.
All Run execution takes place within the Control Plane including model calls.
4. Function calls
The SDKs will long poll the Control Plane for new function calls. When a new call is available, it will be executed using the provided handler.
All Function execution takes place within the your runtime environment, i.e the machines running the Inferable SDK.
5. Monitoring and Observability
The Inferable Playground and API allow developers to monitor the health of the SDKs, runs, and any functions that are being executed.
For more details, please see the Observability page.
Was this page helpful?