LLM Providers

Configure OpenAI, Anthropic, Groq, or Ollama as your AI model provider for Astralform projects.

Supported Providers

Astralform supports multiple LLM providers:

Provider	Models	Best For
OpenAI	GPT-4o, GPT-4o-mini	General purpose, vision
Anthropic	Claude Sonnet 4, Claude Haiku	Long context, reasoning
Groq	Llama 3, Mixtral	Fast inference
Ollama	Any local model	Self-hosted, privacy

Configuration

LLM providers are configured per agent through the agent detail page:

1. Go to Agents and click an agent
2. On the Model tab, select your provider
3. Enter the provider API key
4. Choose a model
5. Click Save

Each agent can have its own LLM provider and model. If a named agent has no LLM configured, it inherits from the Main Agent.

See Agents for more on agent configuration.

OpenAI

{
  "provider": "openai",
  "api_key": "sk-...",
  "model": "gpt-4o"
}

Available Models:

• gpt-4o - Most capable
• gpt-4o-mini - Faster, cheaper
• gpt-4-turbo - Large context window

Anthropic

{
  "provider": "anthropic",
  "api_key": "sk-ant-...",
  "model": "claude-sonnet-4-20250514"
}

Available Models:

• claude-sonnet-4-20250514 - Best balance
• claude-haiku-3-5-20241022 - Fast and efficient

Groq

{
  "provider": "groq",
  "api_key": "gsk_...",
  "model": "llama-3.3-70b-versatile"
}

Available Models:

• llama-3.3-70b-versatile - Most capable
• llama-3.1-8b-instant - Fastest
• mixtral-8x7b-32768 - Good balance

Ollama (Self-Hosted)

For local models:

{
  "provider": "ollama",
  "base_url": "http://localhost:11434",
  "model": "llama3.2"
}

Ollama requires the model to be running on your server. No API key needed.

Model Parameters

Fine-tune model behavior:

Parameter	Description	Default
temperature	Randomness (0-2)	0.7
max_tokens	Maximum response length	4096
top_p	Nucleus sampling	1.0

Fallback Configuration

Configure a fallback provider for reliability:

{
  "primary": {
    "provider": "anthropic",
    "model": "claude-sonnet-4-20250514"
  },
  "fallback": {
    "provider": "openai",
    "model": "gpt-4o"
  }
}

Cost Optimization

Tips for reducing costs:

1. Use smaller models for simple tasks
2. Set appropriate max_tokens limits
3. Use Groq for high-volume, simple queries
4. Consider Ollama for development

Next Steps

Agents

Configure agents with LLM, skills, and tools

MCP Servers

Add server-side tools