Agents

Agents are autonomous LLM-based reasoning engines that may use zero or more tools to iteratively achieve a pre-defined goal.

Agents exhibit characteristics of a long-running durable task in a distributed system, except that it uses an LLM to model its internal control flow. Inferable will run the agent in a durable way and return the structured result of the agent to the workflow.

​

Agent Types

The ctx.agents object is used to interact with agents. Inferable provides a default agent type called react that is a good starting point for most use cases.

​

React Agent

The ctx.agents.react object is used to interact with the React agent. It takes in 5 arguments:

1. name: The name of the agent.
2. instructions: The instructions for the agent. (This gets passed to the system prompt of the LLM)
3. input: Stringified input for the agent.
4. tools: The tools that the agent can use. (These are registered with the workflow context)
5. resultSchema: The schema of the result that the agent will return.

const agentInstructions = `
  Evaluate the provided support ticket body and extract the user from the database.

  When searching for users, if you don't get specific results, try to search with a more general term with sub strings with unique nouns.
  For example, "John Smith": searchUser("John Smith"), searchUser("John"), searchUser("Smith"), etc.
`;

workflow.version(1).define(async (ctx, input) => {
  const { venueIds } = await ctx.llm.agents.react({
    name: "userSearch",
    instructions: agentInstructions,
    input: JSON.stringify({ ticket }),
    tools: ["searchUsers"],
    resultSchema: z.object({
      userId: z.string(),
    }),
  });
});

Internally, this agent will:

1. Call the searchUsers tool with the search query.
2. Inspect the response from the tool to examine the results.
3. If no results are found, call the searchUsers tool again with a new search query.
4. Repeat step 2 and 3 until a result is found or the agent is stopped.
5. Return the user ID.

Semantics

• The agent will either resolve with a result or throw an error.
• If the agent throws an error, the workflow will stop and the error will be surfaced.
• If the agent returns a result, the workflow will continue to execute the next step.

​

Tool Usage

Tools are functions that are used to interact with external systems. They are defined with your workflow context and are called by the agent.

A tool takes in the following arguments:

1. name: The name of the tool.
2. schema: The schema of the tool.
3. func: The function that is called when the tool is used.
4. config: Optional configuration for the tool.

orderSearchWorkflow.tools.register({
  name: "searchUser",
  schema: z.object({
    searchTerms: z.object({
      email: z.string().optional(),
      orderId: z.string().optional(),
    }),
  }),
  func: async ({ searchTerms }) => {
    if (searchTerms.length === 0) {
      throw new Error("Search terms are required");
    }

    if (searchTerms.email) {
      return searchUserByEmail(searchTerms.email);
    }

    if (searchTerms.orderId) {
      const order = await getOrder(searchTerms.orderId);

      if (users.length === 0) {
        throw new Error("No users found for order ID");
      }

      return getUsersByEmail(order.email);
    }

    return [];
  },
  config: {
    retryCountOnStall: 3, // Times to retry the tool call if it stalls
    timeoutSeconds: 120, // Timeout for the tool call in seconds
  },
});

​

Model Selection

By default ctx.llm.agents will use the Claude Sonnet 3.5 model provided by Inferable.

However, this is intended for testing only. You can specify your own Anthropic API details when calling ctx.llm.agents.

const { venueIds } = await ctx.llm.agents.react({
  name: "userSearch",
  instructions: agentInstructions,
  input: JSON.stringify({ ticket }),
  tools: ["searchUsers"],
  resultSchema: z.object({
    userId: z.string(),
  }),
  provider: {
    model: "claude-3-7-sonnet-20250219"
    key: ""
  }
});

Currently ctx.agents only supports the Anthropic API and the following model Ids:

"claude-3-7-sonnet-20250219",
"claude-3-7-sonnet-latest",
"claude-3-5-sonnet-20241022",
"claude-3-5-sonnet-latest",
"claude-3-5-sonnet-20240620",
"claude-3-5-haiku-20241022",
"claude-3-5-haiku-latest"

Configuration

• retryCountOnStall: The number of times to retry the tool call if it stalls.
• timeoutSeconds: The timeout for the tool call in seconds.

Semantics

• When you call workflow.listen(), it will automatically register all the tools defined and make them available to the agent.