Inferable is open source (MIT license) and can be self-hosted on your own infrastructure.

This guide will walk you through setting up a self-hosted instance of Inferable using Docker Compose.

Requirements:

  • Docker
  • Docker Compose

​
Infrastructure Requirements

Inferable is designed to run efficiently on commodity hardware. The system has minimal infrastructure dependencies, requiring only:

  • PostgreSQL database (with pgvector extension)
  • Redis for caching and high-throughput message processing

Both of these services can be run on modest hardware, making Inferable easy to deploy in any environment.

1

Download Docker Compose definition

We provide a Docker Compose definition within the Inferable repository.

Use curl to download the file:

curl -O https://raw.githubusercontent.com/inferablehq/inferable/refs/heads/main/control-plane/docker-compose.yml
2

Define environment variables

When running using docker-compose.yml, most environment variables have defaults but the following need to be provided explicitly:

  • MANAGEMENT_API_SECRET (This can be generated using openssl rand -hex 32, take note of the value)
  • ANTHROPIC_API_KEY (With access to claude-3-5-sonnet / claude-3-haiku - our default reasoning models)
  • COHERE_API_KEY (With access to embed-english-v3 - our default embedding model)

You can alternatively (or in addition) use AWS Bedrock as the model provider. This will require you to provide AWS credentials. Please see the configuration section.

export MANAGEMENT_API_SECRET=XXXXXXX
export ANTHROPIC_API_KEY=XXXXXXX
export COHERE_API_KEY=XXXXXXX
3

Start Docker Compose

docker-compose up

This will start The Inferable control plane at http://localhost:4000 and the following auxiliary resources:

  • PostgreSQL database with pgvector
  • Redis for caching
4

Manage via the CLI

The Inferable CLI can be used to manage your Inferable instance. Export INFERABLE_API_ENDPOINT to target your instance, rather than Inferable Cloud.

See the section on headless mode for more details.

npm install -g @inferable/cli
export INFERABLE_API_ENDPOINT=http://localhost:4000

# If running in headless mode, you will be prompted for the `MANAGEMENT_API_SECRET` generated above
inf auth login

# Create a new cluster
inf clusters create

# Create a cluster key (for registering machines)
inf auth keys create "MY_KEY"
5

Register tools and workflows

You can now use the SDKs to register tools and workflows (Node) using your-self hosted instance by providing the baseUrl option to initialize the client.

const inferable = new Inferable({
  apiSecret: "", // Generated in the previous step
  baseUrl: "http://localhost:4000",
});

For more details, see the quick start.

​
Configuration

The Inferable control-plane accepts the following environment variables:

VariableDescriptionRequired
MANAGEMENT_API_SECRETA deployment secret for the management API.Yes (If running in headless mode)
ANTHROPIC_API_KEYAPI key for the Anthropic API.Yes (If Bedrock is not available)
COHERE_API_KEYAPI key for the Cohere API.Yes (If Bedrock is not available)
BEDROCK_AVAILABLEIf set to true, Bedrock will be used as the model provider (You must also provide AWS credentials).No
DATABASE_URLThe connection string for the PostgreSQL database.Yes
DATABASE_SSL_DISABLEDIf set to true, SSL will be disabled for the PostgreSQL database.No
REDIS_URLThe connection string for the Redis instance.Yes
APP_ORIGINThe origin of the Inferable Web UI for CORS.No
JWKS_URLThe URL of the Clerk JWKS url.No
CLERK_SECRET_KEYThe Clerk secret key.No
HYPERDX_API_KEYAPI key for sending telemetry to Hyperdx.No
ROLLBAR_ACCESS_TOKENAPI key for sending error events to Rollbar.No
POSTHOG_API_KEYAPI key for sending analytic events to Posthog.No
POSTHOG_HOSTThe host for sending analytic events to Posthog.No
EE_DEPLOYMENTEnable startup check for availability of all features.No

​
Headless Mode

This self-hosting guide will produce a “headless” instance of Inferable, meaning the the control plane will be running but not the Web UI. Web UI is not essential to interacting with the control-plane.

The Web UI relies on Clerk for authentication with Cluster being associated with a Clerk Organization. In headless mode, a MANAGEMENT_API_SECRET is used instead.

The MANAGEMENT_API_SECRET should only be used for managing clusters. You can use it to create API Keys which are used by the SDK to register tools.

inf auth keys create "MY_KEY"

The Web UI is also open source. We will be adding a guide for self-hosting the Web UI in the future.

Was this page helpful?

Suggest edits
Cluster API KeysAdopting Inferable