Choosing Inferable

Inferable is a managed developer platform. We believe it’s a great fit for many projects, but it’s not the right choice for every project.

Inferable is opinionated, and we’ve made a number of choices about how we think developers should build complex LLM-powered applications.

This guide is designed to help you understand whether Inferable is a good fit for your project.

​

Inferable managed vs. Developer managed

There are two types of choices made in LLM-powered agentic applications:

1. How the model works (LLM-Ops): Managing the LLM provider, model, inference pipeline, model routing etc.
2. How the application works (Integration of Business Logic): Connecting LLMs to backend services, databases, execution environments, etc.

Inferable is opinionated about the first type of choice, and leaves the second type of choice open to the developer.

	Inferable managed	Developer managed
Tool Execution Environment: Inferable executes your functions (tools) within your own environment.	⛔️	✅
CI/CD and Deployment: Inferable does not introduce any new build / deployment steps top your application	⛔️	✅
Integration with services: Developers explicitly allow access to functions	⛔️	✅
Function Observability: Developers use existing observability tools for function observability	⛔️	✅
Model Observability: Inferable provides built-in observability for models interacting with functions	✅	⛔️
LLM management: Inferable manages all interaction with the LLM providers.	✅	⛔️
LLM routing: Inferable manages the LLM routing logic, handling rate limits and unexpected provider downtime.	✅	⛔️
Model choice: Inferable choses the most approbate LLM based on the task at hand.	✅	⛔️
Model Data privacy: Inferable ensures that all model data is private to your organization, and not retained by model providers	✅	⛔️

​

Ideal Use Cases

​

1. Extending Existing Services

Inferable is an excellent choice when you want to add AI capabilities to existing services without major refactoring:

• You have established REST/GraphQL APIs or internal services
• You want to maintain your existing codebase
• You want to build AI-based applications without rebuilding your infrastructure

​

2. Highly-Regulated Industries

Inferable is well-suited for organizations with strict security requirements:

• All compute runs in your infrastructure
• No inbound connections required
• Private networking support
• Data never leaves your environment, unless you explicitly return it through functions
• Optional Sentinel integration for complete data privacy

​

3. Complex Multi-Step Workflows

Ideal for scenarios requiring orchestrated sequences of operations:

• Multiple service interactions
• Context-aware decision making
• Structured workflows with approval steps
• Operations requiring both automation and human oversight

​

4. Enterprise Integration Projects

Perfect for enterprise environments needing:

• On-premise execution
• Integration with existing authentication systems
• Requiring observability and auditability with extensive tooling
• Support for multiple programming languages

​

5. Building Vertical AI SaaS Platforms

If you’re looking to build a vertical AI SaaS platform, then Inferable is a great choice.

• Inferable manages the LLM-Ops choices, so you can accelerate your time to value.
• Inferable allows your developers to focus on building your vertical SaaS platform, not the underlying infrastructure for inference.

​

Not Ideal For

​

1. Simple Chatbots

If you just need a basic chatbot interface:

• Single-step interactions
• No complex orchestration needed
• No integration with backend services required

​

2. Pure Content Generation

When your primary need is:

• Text generation without service integration
• Image generation
• Basic prompt-completion workflows

​

3. Consumer Applications without Backend Services

May not be the best fit for:

• B2C applications requiring end-user LLM access
• High-volume, low-complexity interactions
• Applications without backend services

​

4. LLM research and experimentation

If you’re looking to experiment with LLMs, Inferable is not the best choice.

• We do not support fine-tuning or instructing models
• We do not support running custom inference pipelines
• We do not allow arbitrary code execution

​

Alternatives

If Inferable is not the right choice for your project, due to above reasons, there are other platforms that may be a better fit:

​

1. LLM Evaluation & Observability Platforms

These are platforms that focus on observability and evaluation of LLM applications, not necessarily building LLM applications.

Langfuse - Open-source observability and analytics for LLM applications DeepEval - Framework for LLM testing and evaluation Weights & Biases - ML experiment tracking platform with LLM support Helicone - OpenAI API monitoring and optimization TruLens - LLM evaluation metrics and feedback collection

​

2. LLM Application Development Frameworks

These are frameworks that focus on building LLM applications, but do not offer managed inference or managed distributed execution for tool calling.

LangChain - Popular framework for building LLM applications LlamaIndex - Data framework for LLM applications Semantic Kernel - Microsoft’s open-source orchestration framework Haystack - Production-ready NLP pipelines DSPy - Stanford’s framework for prompt engineering OpenLLMetry - Open-source observability tools Guidance - Structured output generation framework

​

3. No-Code/Low-Code AI Application Platforms

These are platforms that focus on building general-purpose applications, with a focus on no-code or low-code development.

Bubble - General purpose no-code platform with AI features Retool - Internal tool builder with AI capabilities Superblock - No-code platform for building internal tools FlowiseAI - Visual programming for LLM applications Superagent - No-code platform for AI agents Mendable - Specialized chatbot builder Tooljet - Open-source platform with AI integration