Introduction to Agents

In Inferable context, agents are autonomous LLM-based reasoning engines that may use zero or more tools to iteratively achieve a pre-defined goal.

They exhibit characteristics of a long-running durable task in a distributed system, except that it uses an LLM to model its internal control flow.

​

Autonomous

Agents are self-contained, “probabilistic reasoning engines”. The agent is responsible for its own control flow. That is, that as a developer you’re not responsible for the control flow of the Agent. The only way you can influence its control flow is via:

• The system prompt
• The data you provide
• The result schema
• Tool attachments

​

Multi-step

Inferable Agents are designed to be multi-step, and may make one or more LLM calls to achieve their goal. This is in stark contrast to the single-step nature of most LLM-based workflows. At each step, the agent will reason about the current state of the world, and the next step it should take. Therefore, it will make at least one LLM call to achieve its goal, but potentially many more.

Building effective agents is a great primer on how to reason about what an agent should do.

​

Tool-use

Agents may use zero or more tools to achieve their goal. They can dynamically choose which tools to use at each iteration, based on the current state of the world, and the goals they have. You can influence which tools are available to the agent by naming them in the goals.

​

Fault-tolerance

Internally, the agent implementation in Inferable is based on a distributed job queue. At each iteration, the agent saves its current state, and enqueues a task to derive the next step. This means that the actual agent can reason and act as a long running task, and recover from the point of failure, even if one of your machines that run a tool crashes.

​

Durability

During the execution of an agent, it’s possible that it will run out of machines to run on, either because all the machines are down, or because of a temporary outage. At these points, the agent will pause indefinitely, and will not be able to continue.

Another potential scenario to pause is the human-in-the-loop request approvals of the function calls. If the agent is waiting for a human to approve a request, it will pause indefinitely, and will not be able to continue.