Docker AI Setup for Askimo App

Docker AI Configuration

Connect Askimo App to AI models running in Docker containers for portable and reproducible AI deployments.

Using Askimo with a team?

There are two ways your team can access AI through Askimo today:

Available now

Bring your own key

Each team member adds their own API keys in Askimo. Keys are encrypted and stored locally on their machine. Best for individuals and small groups who want full independence.

Available now

Shared AI Gateway

Run an OpenAI-compatible gateway on a shared server. Each user connects Askimo with just a URL, with no personal API keys required. The real keys stay on the server with central spend control. See an example setup using LiteLLM.

Coming soon

Askimo Team

A native team edition with shared workspaces and collaboration features is in development. Learn more.

Server Configuration

• Server URL: Docker AI container endpoint
  • Default: http://localhost:12434 (Docker AI default port)
  • For remote containers: http://your-server:12434
• API Key: (Optional) If your container requires authentication
• Timeout: Connection timeout (default: 120s)
• Available Models: Detected from your running Docker AI containers

Setting Up Docker AI

1. Install Docker Desktop from docker.com
2. Enable the OpenAI-compatible API server:

docker desktop enable model-runner --tcp 12434

3. Pull an AI model from Docker Hub (it will be automatically served):

docker model pull ai/<model>

4. In Askimo, configure the provider:
   • Provider Type: OpenAI (OpenAI-compatible)
   • Base URL: http://localhost:12434/v1
   • API Key: (leave empty - not required for local Docker AI)
   • Model: The model name (e.g., ai/gemma3:4B-F16)
5. Click “Test Connection” to verify

OpenAI-Compatible Server

Docker AI models run as OpenAI-compatible servers. After enabling model-runner with docker desktop enable model-runner --tcp 12434 and pulling a model, it is automatically served on port 12434. No additional serve command is needed.

Accessing Provider Settings

Since Docker AI models run as OpenAI-compatible servers, you configure them through the OpenAI provider settings:

1. Click on the menu bar
2. Select “Settings”
3. Navigate to the “AI Providers” tab
4. Select “OpenAI” from the provider list
5. Configure the settings:
   • Base URL: http://localhost:12434/v1 (or your custom port)
   • API Key: Leave empty (not required for local Docker AI)
   • Model: The name of your Docker AI model

Keyboard Shortcut: ⌘ + , (macOS) or Ctrl + , (Windows/Linux) then click “AI Providers”

Docker Hub AI Models

Visit hub.docker.com/u/ai to browse available AI models.

Prerequisites:

# Enable OpenAI-compatible API server (one-time setup)
docker desktop enable model-runner --tcp 12434

Example: Running gemma3:4B-F16 model

12434/v1

# Pull the model (it will be automatically served)
docker model pull ai/gemma3:4B-F16

# Verify it's running
docker model ls

Configure in Askimo:

• Provider: OpenAI
• Base URL: http://localhost:12434/v1
• API Key: (leave empty)
• Model: ai/gemma3:4B-F16

Model Management

List Pulled Models:

docker model ls

Remove a Model:

docker model rm <model-name>

Multiple Models

Docker AI can serve multiple models simultaneously. All pulled models are available through the same endpoint at http://localhost:12434/v1. Switch between them in Askimo by changing the model name in your provider configuration.

Troubleshooting

Cannot Connect?

• Ensure model-runner is enabled: docker desktop enable model-runner --tcp 12434
• Verify models are pulled: docker model ls
• Ensure correct base URL: http://localhost:12434/v1
• Test endpoint: curl http://localhost:12434/v1/models

Model Not Available?

• Check if Docker Desktop is running
• Verify model-runner is enabled (see above)
• Verify the model was pulled successfully: docker model ls
• Try pulling the model again: docker model pull ai/<model-name>
• Restart Docker Desktop if needed

Slow Performance?

• Docker AI will use GPU automatically if available
• Use smaller models for faster inference
• Close other resource-intensive applications
• Check system resources (CPU/RAM usage)

Need Different Port? If port 12434 is already in use, you can specify a different port when enabling model-runner:

docker desktop enable model-runner --tcp 12435

Then update Askimo’s base URL to http://localhost:12435/v1.