Functions

Functions within your code are registered using one of the Inferable SDKs, this makes the function available as a tool for the agent to use within Runs.

The execution of functions takes place within your environment. Inferable manages a job queue of function calls which is polled by the SDK. This means in order to use a function,

1. Your service does not need to expose an endpoint to receive function calls.
2. There are no network configuration changes required to run the function.
3. The function will be executed in your own VPC, as long as it can connect to the Inferable cluster.

• TypeScript
• Go
• .NET

inferable.default.register({
  name: "submitTicket",
  description: "Submit a support ticket",
  func: (input) => {
    console.log("Submitting support ticket", input);
    return `Ticket submitted for ${input.userName}`;
  },
  schema: {
    input: z.object({
      userName: z.string(),
      userEmail: z.string(),
      issue: z.string(),
    }),
  },
});

The snippet above registered a function submitTicket with the default service. Agents within the same cluster can now invoke this function as needed.

​

Failure

The result of a function’s handler is always returned to the agent for evaluation, this includes any Errors or Exceptions thrown.

​

Utilities

​

approvalRequest()

The approvalRequest() utility is used to pause a Run and request approval before continuing. The the run can be paused indefinitely until approval is provided.

Approval can be provided via the Playground or manually via the API.

Within the handler, the call’s approval can be checked via context.approval which is provided as the second argument.

service.register({
  name: 'deleteUser',
  description: "Deletes a user from the database",
  schema: {
    input: z.object({
      customerId: z.string(),
    })
  },
  func: async (input, context) => {
    if (!context.approved) {
      return approvalRequest();
    }

    return await db
      .customers
      .delete({
        customerId: input.customerId
      });

},
});

​

Options

Valid options when registering a function with the SDK.

​

name

The name of the function that will be provided to the agent.

​

description

A human-readable description of the function, use this to augment the agent’s understanding of the function and when to use it.

​

schema

A zod schema that defines the input shape of the function. The schema is used to explain how to use the function, and to validate the input before the function is executed.

​

func

The handler that will be executed when Inferable invokes the function. This handler runs within your environment.

service.register({
  name: 'printOrder',
  description: "Prints an order using the specified printer",
  schema: {
    input: z.object({
      orderId: z.string(),
      printerId: z.string()
    })
  },
  func: async (input: { orderId: string, printerId: string }) => {
    printer.triggerPaperJam();
  }
});

​

config

Additional configuration options for the function.

​

config.retryCountOnStall

The number of times Inferable will retry the function call if the function stalls.

A function can stall if it does not complete within the allotted time. This usually happens if:

• Function fails to complete within the allotted timeoutSeconds time.
• Worker stalls due to a machine crash or network problems. (See Recovering from machine failures for more information)

Without this configuration, Inferable will not retry the function call.

​

config.timeoutSeconds

The number of seconds Inferable will wait before timing out the function call.

​

config.cache

Used to cache the result of the function for a specified amount of time.

• ttl - The time in milliseconds that the result of the function will be cached.
• keyPath - The path (described using JSONPath) to the unique identifier in the function’s input.

service.register({
  name: 'getCustomer',
  description: "Gets information about a customer",
  schema: {
    input: z.object({
      customerId: z.string(),
    })
  },
  config: {
    cache: {
      ttl: 60 * 60 * 1000 // 1 hour
      keyPath: '$.customerId'
    }
  }
  func: async (input: { customerId: string }) => {
    return await db.customers.find({ customerId: input.customerId });
  },
});

​

config.private

When set to true, the function will not be made available to the agent. This is can be used for functions registered as onStatusChange handlers.

​

Limitations

​

Function return values must be JSON serializable

Since the LLM at the heart of Inferable is generating instructions based on the return values of your functions, it’s important that the return values are JSON serializable, so that the AI model can understand them.

​

Number of functions

Inferable currently supports a maximum of 1000 functions per cluster. This is a soft limit and customers can request an increase by contacting support at [email protected].

​

Go Struct Reference Limitation in JSON Schema Generation

When defining input arguments for your functions, avoid using struct references in Go. Our JSON schema generation doesn’t fully support $ref in the resulting schema, which can lead to issues during function registration and execution.

Here’s an example of what to avoid:

type Address struct {
    Street  string
    City    string
    Country string
}

type Person struct {
    Name    string
    Address *Address  // This reference will cause issues
}

func ProcessPerson(input Person) {
    // Function logic here
}

This will cause issues with function registration and execution.

Recommended Approach:

type Person struct {
    Name    string
    Address struct {
        Street  string
        City    string
        Country string
    }
}

func ProcessPerson(input Person) {
    // Function logic here
}

If the SDK detects that you’re using a struct reference, it will log a warning and not register the function.