Structured Outputs

Inferable provides a way to get structured outputs from LLMs. This is useful for a variety of use cases.

• Parsing API responses
• Extracting information from natural language
• Scanning images for text

All structured outputs are obtained via ctx.llm.structured() call.

​

Usage

​

Text Processing

This is a simple example of how to get a structured output within a workflow.

// ... existing workflow code ...

const { events } = await ctx.llm.structured({
  input: `Federal reserve was founded in 1913 after the banking panic of 1907.`,
  schema: z.object({
    events: z.array(
      z.object({
        event: z.string(),
        year: z.number(),
      }),
    ),
  }),
});

console.log(events);
// [
//   {
//     event: "Federal reserve founded",
//     year: 1913,
//   },
//   {
//     event: "Banking panic",
//     year: 1907,
//   },
// ]

​

Image Processing

You can also process images and extract structured information from them by providing a base64 encoded image string to the input field.

const image = await fetch("https://example.com/menu.png")
  .then((res) => res.arrayBuffer())
  .then((buffer) => Buffer.from(buffer).toString("base64"));

const { items } = await ctx.llm.structured({
  input: image,
  schema: z.object({
    items: z.array(
      z.object({
        name: z.string(),
        price: z.number(),
      }),
    ),
  }),
});

​

Configuration

Input

The input is always a string. If you need to process an image, you can pass a base64 encoded image string to the input field.

Schema

The schema is a JSON schema that describes the structure of the output you want. Or in TypeScript, a Zod schema.

Instructions

Sometimes, you may want to provide additional instructions to the LLM to help it understand the structure of the output you want.

You can do this by providing the instructions field.

const { events } = await ctx.llm.structured({
  instructions:
    "Extract the events from the text. The events should be in the past tense.",
  input: `Federal reserve was founded in 1913 after the banking panic of 1907.`,
  schema: z.object({
    events: z.array(
      z.object({
        event: z.string(),
        year: z.number(),
      }),
    ),
  }),
});

​

Model Selection

By default ctx.llm.structured will use the Claude Sonnet 3.5 model provided by Inferable.

However, this is intended for testing only. You can specify your own LLM provider details when calling ctx.llm.structured.

const { events } = await ctx.llm.structured({
  instructions:
    "Extract the events from the text. The events should be in the past tense.",
  input: `Federal reserve was founded in 1913 after the banking panic of 1907.`,
  schema: z.object({
    events: z.array(
      z.object({
        event: z.string(),
        year: z.number(),
      }),
    ),
  }),
}, {
  provider: {
    key: ""
    url: ""
    model: ""
  }
});

OpenAI (Or OpenAI compatible), Anthropic and Google AI providers are supported.

​

Semantics

​

JSON Schema Descriptions

Inferable will use the JSON schema descriptions (or Zod schema descriptions) to provide the LLM with additional context about the structure of the output you want.

​

Retries

If the LLM returns a structured output that does not conform to the schema you provided, Inferable will retry the request 2 more times, providing the LLM with the previous error.

If the LLM still returns an output that does not conform to the schema, ctx.llm.structured() will throw an error.