Skip to content

🔌 Providers

SLM-first defaults. Generic harness underneath. If it speaks OpenAI Chat Completions (or Ollama’s native API), you’re in. 🎟️

🧠

Mental model: the harness owns structure. The model owns tokens. Swap brains without rewriting the play. Bring your own GPU drama.

The adapter picture 🖼️

flowchart LR
  M[Model server 🤖] --> C[provider + endpoint + model + key]
  C --> H[SLMCode harness 🧰]
  H --> Out[Plan / specialists / critic / memory ✨]

Quick switches ⚡

slmcode config set provider omlx
slmcode config set model Qwen3-Coder-30B-A3B-Instruct-MLX-4bit
slmcode doctor

slmcode run --provider ollama --model qwen2.5-coder:14b \
  --endpoint http://127.0.0.1:11434 "Fix the flaky test"

slmcode run --provider lmstudio --model local-coder \
  --endpoint http://127.0.0.1:1234/v1 "…"

slmcode run --provider openai --model gpt-4o-mini \
  --endpoint https://api.openai.com/v1 \
  --api-key "$OPENAI_API_KEY" "…"

export SLMCODE_PROVIDER=openrouter
export SLMCODE_MODEL=anthropic/claude-3.5-sonnet
export SLMCODE_ENDPOINT=https://openrouter.ai/api/v1
export SLMCODE_API_KEY=…
slmcode run -v "…"

Persist:

slmcode config set provider ollama
slmcode config set model qwen2.5-coder:14b
slmcode config set endpoint http://127.0.0.1:11434

Studio → Settings writes the same file. One source of truth. Less “which config is lying?”.

Built-in presets 🗂️

Provider Default endpoint Notes
omlx http://127.0.0.1:8000/v1 Default on Apple Silicon 🍎
ollama http://127.0.0.1:11434 Native Ollama API 🦙
lmstudio http://127.0.0.1:1234/v1 OpenAI-compat
openai https://api.openai.com/v1 Needs key 🔑
openrouter https://openrouter.ai/api/v1 Many models 🌈
groq https://api.groq.com/openai/v1 Fast ⚡
together https://api.together.xyz/v1
deepseek https://api.deepseek.com/v1
mistral https://api.mistral.ai/v1
fireworks https://api.fireworks.ai/inference/v1
gemini / google Google OpenAI-compat URL
vllm / litellm / custom http://127.0.0.1:8000/v1 Self-hosted
anything else you set endpoint Treated as OpenAI-compat ✨

🎁 Unknown names are features

provider: my-corp-gateway + endpoint + key just works. Fancy DNS optional.

API keys 🔑

  1. --api-key / config.api_key
  2. SLMCODE_API_KEY
  3. Provider env (OPENAI_API_KEY, OPENROUTER_API_KEY, OMLX_API_KEY, …)
  4. oMLX ~/.omlx/settings.json

🚨 Please don't

Commit keys. Future-you will sigh in stereo. HR will invent new verbs.

Per-agent providers 🧩

Cheap local explorer + sharper cloud reviewer = budget diplomacy.

/agent edit reviewer provider=openai model=gpt-4o-mini endpoint=https://api.openai.com/v1
/agent edit worker provider=ollama model=qwen2.5-coder:14b

YAML: .slmcode/agents/<id>.yaml. Unique backend keys prevent gateway mix-ups (“why is my local worker billing OpenAI?” — never again).

Knobs by model size 📏

Class Try
🐣 7–14B SLM think_passes 2, low context KB, retries 2+, parallel 1
🐦 ~30B Defaults
🦅 Frontier Raise parallel; keep permission: review on serious repos

→ ⚙️ Config · 🧪 Recipes · ❓ FAQ

☀️ Made with ♥ by UnicoLab