OpenAI-Compatible Provider - Connect Any OpenAI-Compatible API to Askimo

OpenAI-Compatible Provider

Askimo supports any server that implements the OpenAI-compatible API, giving you the flexibility to connect cloud-hosted providers, local inference servers, or self-hosted gateways that expose an OpenAI-style /v1 endpoint.

What is an OpenAI-Compatible Server?

Many AI tools and services implement the OpenAI REST API format, meaning they accept the same request structure as api.openai.com/v1. Examples include:

• Cloud providers - OpenRouter, Together AI, Groq, Mistral AI, Fireworks AI
• Local inference servers - vLLM, llama.cpp, LocalAI
• AI gateways and proxies - LiteLLM, any custom proxy that forwards requests to an LLM backend

Why Use the OpenAI-Compatible Provider?

• Broad compatibility - Connect any backend that speaks the OpenAI API format, whether cloud or local
• Flexible authentication - API key required for cloud-hosted endpoints, optional for local servers
• No vendor lock-in - Switch between providers without changing your Askimo workflow

Setting Up an OpenAI-Compatible Provider in Askimo

Required Configuration

• Base URL - The full endpoint URL of your server, including the API version path (e.g. /v1)

  Server	Base URL
  OpenRouter	https://openrouter.ai/api/v1
  Together AI	https://api.together.xyz/v1
  Groq	https://api.groq.com/openai/v1
  vLLM (local)	http://localhost:8000/v1
  llama.cpp (local)	http://localhost:8080/v1
  LiteLLM proxy	http://localhost:4000/v1

The full path matters

Askimo uses the exact URL you enter to discover models and send messages. If your server serves the API at /v1, you must include /v1 in the URL. Entering just http://localhost:8000 will fail because the /models and /chat/completions endpoints will not be found.

• API Key - Authentication token for your server
  • Required for cloud-hosted endpoints (e.g. OpenRouter, Together AI, Groq)
  • Optional for local servers that do not enforce authentication — leave blank if your server does not require it

Step-by-step Setup

1. Start your server if using a local inference server (see examples below), or have your API key ready for cloud providers
2. Select the provider in the footer of the Askimo app, or go to Settings > AI Providers
3. Select OpenAI Compatible from the provider list
4. Enter the Base URL including the API path (e.g. https://openrouter.ai/api/v1)
5. Enter your API key if required, or leave blank for local servers
6. Askimo will automatically fetch the available models once the connection is established

Examples by Server

OpenRouter (cloud)

Sign up at openrouter.ai and get an API key from openrouter.ai/keys.

Base URL in Askimo: https://openrouter.ai/api/v1 API Key: your OpenRouter API key

Together AI (cloud)

Sign up at together.ai and get an API key from your dashboard.

Base URL in Askimo: https://api.together.xyz/v1 API Key: your Together AI API key

vLLM (local)

# Start vLLM with a model
python -m vllm.entrypoints.openai.api_server \
  --model mistralai/Mistral-7B-Instruct-v0.2 \
  --port 8000

Base URL in Askimo: http://localhost:8000/v1 API Key: leave blank

llama.cpp (local)

# Start llama.cpp server
./server -m your-model.gguf --port 8080

Base URL in Askimo: http://localhost:8080/v1 API Key: leave blank

LiteLLM proxy (local or cloud)

# Start LiteLLM proxy
litellm --model ollama/llama2 --port 4000

Base URL in Askimo: http://localhost:4000/v1 API Key: leave blank (or set one if you configured auth on your proxy)

Troubleshooting

Could not connect to the provider?

• For cloud providers, confirm your API key is correct and has sufficient credits
• For local servers, make sure the server is running and accessible
• Verify the URL includes the full API path (e.g. /v1), not just the host and port
• Check that no firewall is blocking the port

No models showing after connecting?

• Confirm the server exposes a GET /v1/models endpoint
• Some minimal servers do not implement /v1/models - check your server’s documentation
• For llama.cpp, make sure you started the server with --port and that it is fully loaded before connecting

SSL / certificate errors?

• For local HTTP servers, ensure the URL starts with http:// not https://
• For self-signed HTTPS, configure your system to trust the certificate

Learn More

• Ollama provider guide - For local Ollama models
• LocalAI provider guide - For LocalAI self-hosted models
• LM Studio provider guide - For LM Studio local server