Blobs

Blobs in Inferable provide an efficient way to handle large data payloads without incurring the cost and latency of processing them through the language model. When functions need to return substantial amounts of data directly to users or APIs, blobs ensure this data bypasses model processing while maintaining data integrity.

​

What are Blobs?

Blobs are base64 encoded strings that can be returned by functions but are not processed by the language model. They are particularly useful when:

• Returning large amounts of data
• Handling binary data (like images)
• Needing to preserve exact data without risk of model hallucination
• Optimizing for cost and latency

​

Benefits

1. Cost Efficiency: By bypassing model processing, blobs reduce token usage and associated costs
2. Lower Latency: Direct data transfer without model processing means faster response times
3. Data Integrity: Eliminates the risk of model hallucination or data modification
4. Binary Data Support: Easily handle non-text data like images or documents

​

Drawbacks

1. Opaque Data: Blobs make the wrapped data completely opaque to the language model. If the data is required at inference time, it should be included in the function’s output, and kept out of the blob.

​

Language Support

​

Using Blobs

Any function result can contain blob data by including a blob() decorated field in the response. Here’s how to structure blob data:

// Example function returning a blob
client.default.register({
  name: "getArticle",
  description: "Returns an article",
  func: async () => {
    return {
      title: "HAL 9000",
      description:
        "HAL 9000 is a fictional artificial intelligence computer system in Arthur C. Clarke's 1968 science fiction novel and 1979 film 2001: A Space Odyssey.",
      image: blob({
        name: "hal9000.png",
        type: "image/png",
        data: "base64EncodedImageData...",
      }),
    };
  },
});

In the above example, the function returns an article with an image. The image is returned as a blob and is not processed by the language model. However, the title and description are processed by the language model and included in the agent context.

​

Supported Blob Types

Currently, Inferable supports the following blob types:

• application/json: For JSON data
• image/png: For PNG images
• image/jpeg: For JPEG images

​

Example Use Cases

​

1. Image Processing

client.default.register({
  name: "processImage",
  description: "Processes an image and returns the result",
  func: async (input) => {
    // Process image...
    return {
      image: blob({
        name: "processed.png",
        type: "image/png",
        data: processedImageBase64,
      }),
    };
  },
});

​

2. Large JSON Responses

client.default.register({
  name: "fetchLargeDataset",
  description: "Returns a large dataset as JSON",
  func: async () => {
    const largeData = await fetchData();
    return {
      data: blob({
        name: "dataset.json",
        type: "application/json",
        data: Buffer.from(JSON.stringify(largeData)).toString("base64"),
      }),
    };
  },
});