Muthoot Finance lifts branch-visit conversion 150% with Blue MachinesRead more
Blue Machines

One place to run and control your AI models.

Run frontier, open and custom models through a shared AI gateway. Blue Machines Relay connects each task to an approved model, with access, spend and request controls around it.

See it in action
Connect models and appsProvider, open-weight and native models in one catalog.Usage and cost controlSee workload spend and enforce daily scope caps.Access policiesDecide which models each application can use.Smart routingMatch tasks to approved models, with fallbacks.Security and privacyConfigure input checks and personal-data redaction.ObservabilityInspect tokens, cost, latency and routing outcomes.Reusable promptsKeep templates, variables and model settings organized.Private inferenceRun open and business-tuned models on your infrastructure.
Inference Cloud / Live traffic
Illustrative data

Applications

Request controls

Scope access checked✓
Daily budget checked✓
Input checked✓

Model route

Blue Machines

Conversation model

Answer a customer question

ApplicationProviderTokensStatus
Service agentBlue Machines640Routed
CortexAnthropic2,800Routed
Document workflowGoogle1,420Routed
Customer operationsOpenAI980Routed

Daily scope budget · illustrative

80%of configured cap

Request protection

Checks before routing

Access · daily budget · configured input policy

Your models. One operating layer.

Bring provider models, open-weight endpoints and Blue Machines models into a shared catalog. Connect applications through scoped tokens and apply controls to the requests you route through Inference Cloud.

Inference Cloud / Model catalog
Illustrative configuration
FrontierOpen-weightBlue Machines models

Anthropic

Provider models

OpenAI

Provider models

Google

Provider models

Amazon

Provider models

NVIDIA

Open model serving

Blue Machines

Native models

Application → tokenToken → model permissionsRequest → provider

Model availability depends on your credentials and configured deployment.

Spend & traffic limits

Enforce daily workload budgets and limit request bursts.

How it works

Daily scope cost caps stop further requests when the limit is reached. Scope request-rate and burst controls, plus token request-rate limits, help contain traffic. Organization, team, project and monthly budgets support allocation and reporting.

Exact caching matches the model, messages and parameters. Semantic caching can reuse answers to similar user questions when enabled. Configure request-level expiry and bypass behavior for workloads that need fresh responses. Voice turns bypass caching. Cache-hit reporting is separate from usage-event analytics.

Daily scope spending caps are enforced; monthly and parent-level allocations are accounting controls.

Inference Cloud / Scope usage
Illustrative

Daily scope cap

Service operations

80% used
Daily cost capEnforced
Request rate + burstConfigured
At the daily capFurther requests stopped
Monthly allocationReporting and accounting

Provider keys & scoped access

Connect your provider credentials and control which models each workload can use.

How it works

Attach bring-your-own-key credentials to tokens, configure platform-credential fallback, and restrict model access at token and scope level. Organize usage through organizations, teams, projects and scopes.

Use scopes to separate applications, environments or business processes.

Inference Cloud / Model access
Illustrative

Example scope permissions

ScopeFrontierOpenNative
ClaimsAllowAllowAllow
ServiceRestrictedAllowAllow
DevelopmentAllowAllowRestricted

Provider credentials attached to tokens · scope permissions applied before routing

Task-aware routing with Relay

Choose a model for the task, with quality, speed or cost as the priority.

How it works

Relay classifies the request by task and complexity, filters candidates against token and scope model permissions, and selects a model with an ordered fallback list. Speed modes also consider observed provider latency and errors.

Use ordered per-request model fallbacks for rate limits, provider errors and network failures. Platform provider keys support weighted rotation and cooldowns. For streamed responses, fallback is available before the first response chunk is sent.

Routing modes are available; custom candidate-table changes are configured with our team.

Inference Cloud / Routing decision
Illustrative

Incoming task

Summarize a claim file and identify missing evidence.

ClassifyDocument analysis · complex
FilterApproved models for claims scope
PrioritizeQuality
SelectEligible reasoning model
PrepareOrdered fallback models

Input guardrails

Check incoming chat requests before they reach the model provider.

How it works

Apply request-level input checks, custom patterns, PII handling and webhook checks. Block or redact matching content and inspect the reason in request metadata.

These controls apply to chat-completion inputs. Configure them per request through your integration.

Inference Cloud / Input protection
Illustrative

Chat-completion input

Before input checks

Email alex@example.com about account 12345678.

After configured redaction

Email [EMAIL] about account [ACCOUNT].

Policy result and reason attached to request metadata.

Request-level visibility

Understand model usage, cost, latency and failures across your workloads.

How it works

Review input and output tokens, estimated cost, model and provider, request status and latency. Use routing metadata to understand model selection and fallback behavior, and aggregate spending by scope and team.

Model-request latency and end-to-end voice latency are measured separately.

Inference Cloud / Request inspector
Illustrative

Input tokens

2,480

Output tokens

320

Request latency

840 ms

Status

Completed

ScopeCustomer service
Model / providerResolved at request time
Estimated costAttached to usage event
Route reasonSelected within scope permissions

Model catalog & prompt versions

Keep model choices and reusable prompts organized as applications grow.

How it works

Review model aliases, context windows and pricing dimensions in the catalog. Manage named prompt templates with versions, variables and inference parameters so teams can reuse a consistent starting point.

Available models and serving capacity depend on the configured deployment.

Inference Cloud / Prompt registry
Illustrative

Named prompt template

service-summaryVersion 3

Summarize {{conversation}} for {{team}}. Include the request, completed steps and next action.

Variablesconversation · team
ParametersStored with the template
Model catalogAliases · context windows · rate cards

Choose where inference runs.

Match hosting, infrastructure control and model ownership to your workload.

Provider-hosted inference

Connect provider models for reasoning, analysis and tasks that benefit from general-purpose intelligence.

  • Model catalog with aliases and rate cards
  • Bring your own provider credentials
  • Relay selection within approved models

Complex analysis · long summaries · general reasoning

Explore Custom Models

AI is spreading across your business.
Give it one operating layer.

Bring model choices, request controls and usage into view. Inference Cloud helps your team choose where AI runs, manage what each workload can use, and understand the cost of the work it performs.

What changes for your business.

Your AI in one place

Bring configured providers, private endpoints and workloads into a shared operating view.

Choose with context

Match tasks to approved models and deployment options.

Control each workload

Apply scope permissions, input checks and daily spending limits.

Explain the usage

Trace model selection, tokens, estimated cost and request outcomes.

Put Inference Cloud to work.

Bring us your first use case. We’ll map the path from your current stack to production.

How we operate