> ## Documentation Index
> Fetch the complete documentation index at: https://docs.inferable.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Structured Outputs

> Getting structured outputs from LLMs

Inferable provides a way to get structured outputs from LLMs. This is useful for a variety of use cases.

* Parsing API responses
* Extracting information from natural language
* Scanning images for text

All structured outputs are obtained via `ctx.llm.structured()` call.

## Usage

### Text Processing

This is a simple example of how to get a structured output within a workflow.

```typescript
// ... existing workflow code ...

const { events } = await ctx.llm.structured({
  input: `Federal reserve was founded in 1913 after the banking panic of 1907.`,
  schema: z.object({
    events: z.array(
      z.object({
        event: z.string(),
        year: z.number(),
      }),
    ),
  }),
});

console.log(events);
// [
//   {
//     event: "Federal reserve founded",
//     year: 1913,
//   },
//   {
//     event: "Banking panic",
//     year: 1907,
//   },
// ]
```

### Image Processing

You can also process images and extract structured information from them by providing a base64 encoded image string to the `input` field.

```typescript
const image = await fetch("https://example.com/menu.png")
  .then((res) => res.arrayBuffer())
  .then((buffer) => Buffer.from(buffer).toString("base64"));

const { items } = await ctx.llm.structured({
  input: image,
  schema: z.object({
    items: z.array(
      z.object({
        name: z.string(),
        price: z.number(),
      }),
    ),
  }),
});
```

## Configuration

**Input**

The input is always a string. If you need to process an image, you can pass a base64 encoded image string to the `input` field.

**Schema**

The schema is a JSON schema that describes the structure of the output you want. Or in TypeScript, a Zod schema.

**Instructions**

Sometimes, you may want to provide additional instructions to the LLM to help it understand the structure of the output you want.

You can do this by providing the `instructions` field.

```typescript
const { events } = await ctx.llm.structured({
  instructions:
    "Extract the events from the text. The events should be in the past tense.",
  input: `Federal reserve was founded in 1913 after the banking panic of 1907.`,
  schema: z.object({
    events: z.array(
      z.object({
        event: z.string(),
        year: z.number(),
      }),
    ),
  }),
});
```

### Model Selection

By default `ctx.llm.structured` will use the Claude Sonnet 3.5 model provided by Inferable.

However, this is intended for testing only. You can specify your own LLM provider details when calling `ctx.llm.structured`.

```typescript

const { events } = await ctx.llm.structured({
  instructions:
    "Extract the events from the text. The events should be in the past tense.",
  input: `Federal reserve was founded in 1913 after the banking panic of 1907.`,
  schema: z.object({
    events: z.array(
      z.object({
        event: z.string(),
        year: z.number(),
      }),
    ),
  }),
}, {
  provider: {
    key: ""
    url: ""
    model: ""
  }
});
```

OpenAI (Or OpenAI compatible), Anthropic and Google AI providers are supported.

## Semantics

### JSON Schema Descriptions

Inferable will use the JSON schema descriptions (or Zod schema descriptions) to provide the LLM with additional context about the structure of the output you want.

### Retries

If the LLM returns a structured output that does not conform to the schema you provided, Inferable will retry the request 2 more times, providing the LLM with the previous error.

If the LLM still returns an output that does not conform to the schema, `ctx.llm.structured()` will throw an error.