- Docker
- Docker Compose
Infrastructure Requirements
Inferable is designed to run efficiently on commodity hardware. The system has minimal infrastructure dependencies, requiring only:- PostgreSQL database (with pgvector extension)
- Redis for caching and high-throughput message processing
1
Download Docker Compose definition
We provide a Docker Compose definition within the Inferable repository.Use
curl
to download the file:2
Define environment variables
When running using
docker-compose.yml
, most environment variables have defaults but the following need to be provided explicitly:MANAGEMENT_API_SECRET
(This can be generated usingopenssl rand -hex 32
, take note of the value)ANTHROPIC_API_KEY
(With access toclaude-3-5-sonnet
/claude-3-haiku
- our default reasoning models)COHERE_API_KEY
(With access toembed-english-v3
- our default embedding model)
You can alternatively (or in addition) use AWS Bedrock as the model provider. This will require you to provide AWS credentials.
Please see the configuration section.
3
Start Docker Compose
http://localhost:4000
and the following auxiliary resources:- PostgreSQL database with pgvector
- Redis for caching
4
Manage via the CLI
The Inferable CLI can be used to manage your Inferable instance.
Export
INFERABLE_API_ENDPOINT
to target your instance, rather than Inferable Cloud.See the section on headless mode for more details.5
Register tools and workflows
You can now use the SDKs to register tools and workflows (Node) using your-self hosted instance by providing the For more details, see the quick start.
baseUrl
option to initialize the client.Configuration
The Inferable control-plane accepts the following environment variables:Variable | Description | Required |
---|---|---|
MANAGEMENT_API_SECRET | A deployment secret for the management API. | Yes (If running in headless mode) |
ANTHROPIC_API_KEY | API key for the Anthropic API. | Yes (If Bedrock is not available) |
COHERE_API_KEY | API key for the Cohere API. | Yes (If Bedrock is not available) |
BEDROCK_AVAILABLE | If set to true , Bedrock will be used as the model provider (You must also provide AWS credentials). | No |
DATABASE_URL | The connection string for the PostgreSQL database. | Yes |
DATABASE_SSL_DISABLED | If set to true , SSL will be disabled for the PostgreSQL database. | No |
REDIS_URL | The connection string for the Redis instance. | Yes |
APP_ORIGIN | The origin of the Inferable Web UI for CORS. | No |
JWKS_URL | The URL of the Clerk JWKS url. | No |
CLERK_SECRET_KEY | The Clerk secret key. | No |
HYPERDX_API_KEY | API key for sending telemetry to Hyperdx. | No |
ROLLBAR_ACCESS_TOKEN | API key for sending error events to Rollbar. | No |
POSTHOG_API_KEY | API key for sending analytic events to Posthog. | No |
POSTHOG_HOST | The host for sending analytic events to Posthog. | No |
EE_DEPLOYMENT | Enable startup check for availability of all features. | No |
Headless Mode
This self-hosting guide will produce a “headless” instance of Inferable, meaning the the control plane will be running but not the Web UI. Web UI is not essential to interacting with the control-plane. The Web UI relies on Clerk for authentication with Cluster being associated with a Clerk Organization. In headless mode, aMANAGEMENT_API_SECRET
is used instead.
The MANAGEMENT_API_SECRET
should only be used for managing clusters. You can use it to create API Keys which are used by the SDK to register tools.
The Web UI is also open
source. We will be
adding a guide for self-hosting the Web UI in the future.