​
Introducing the Inferable Architecture

Inferable provides a robust, distributed architecture designed for building reliable AI-powered applications. Unlike traditional frameworks that require co-location of model calls and function execution, Inferable takes a fundamentally different approach, separating these concerns while maintaining persistent state throughout the entire application lifecycle.

​
Architectural Overview

At a high level, Inferable’s architecture consists of three primary components:

  1. Control Plane: The central nervous system of Inferable, orchestrating all component interactions
  2. Workflows: Stateful orchestration units that coordinate complex, multi-step processes
  3. Client SDKs: Language-specific libraries that connect your functions to the Inferable ecosystem

These 3 components underpin all LLM-native abstractions like structured outputs, human-in-the-loop, agents, tool use and etc.

​
The Control Plane: State Persistence at its Core

The Control Plane is the foundation of Inferable’s architecture, providing critical services including:

​
Persistent State Management

Unlike many AI orchestration frameworks, Inferable’s Control Plane includes a built-in persistence layer that automatically maintains the complete state of all running workflows:

Key benefits of this design:

  • Automatic Checkpointing: Every state change is persisted without developer intervention
  • Transactional Integrity: State updates use ACID transactions to prevent corruption
  • Zero-Loss Recovery: Workflows resume from exactly where they left off, even after system failures
  • Temporal Durability: State can be maintained for any duration - minutes, hours, or months

​
Distributed Job Queue

The Control Plane incorporates a sophisticated distributed job queue that routes function calls to the appropriate execution environments:

This job queue provides:

  • Fair Distribution: Balances workloads across available worker pools
  • Prioritization: Handles urgent tasks ahead of lower-priority ones
  • Backpressure Management: Gracefully handles capacity constraints
  • Routing Intelligence: Directs jobs to workers with appropriate capabilities

​
Long Polling: No Network Configuration Required

One of Inferable’s most distinctive architectural features is its use of long polling for communication between the Control Plane and function execution environments:

This approach offers significant advantages:

  • Zero Inbound Ports: Your execution environments need no open inbound ports
  • Firewall-Friendly: Works with standard outbound HTTPS (port 443)
  • NAT Traversal: Functions behind NAT can still participate
  • No VPN Required: Connect from any network with internet access
  • No DNS Configuration: No need to set up DNS records or certificates

For organizations with strict security requirements, this means:

  • Functions can run in private subnets with no internet-accessible endpoints
  • No need to expose internal services to the public internet
  • No complex network configuration or tunneling required
  • Works seamlessly with existing security boundaries

​
Workflows: Orchestrating Complex Processes

Workflows are the coordination layer in Inferable, responsible for:

  • Maintaining execution context across multiple steps
  • Managing transitions between different processing stages
  • Handling retries, timeouts, and error conditions
  • Coordinating parallel execution paths when appropriate

Each workflow maintains its own state machine, automatically persisted by the Control Plane:

This persistence enables powerful capabilities:

  • Indefinite Pausing: Workflows can pause for any duration while awaiting input
  • Context Preservation: Complete state is maintained during pauses
  • Seamless Resumption: Execution continues with all context intact
  • Multi-Step Processes: Complex multi-stage workflows maintain coherence