Start using the kluster.ai API#
The kluster.ai API provides a straightforward way to work with Large Language Models (LLMs) at scale. It is compatible with OpenAI's API and SDKs, making it easy to integrate into your existing workflows with minimal code changes.
Get your API key#
Navigate to the kluster.ai developer console API Keys section and create a new key. You'll need this for all API requests.
For step-by-step instructions, refer to the Get an API key guide.
Set up the OpenAI client library#
Developers can use the OpenAI libraries with kluster.ai with no changes. To start, you need to install the library:
pip install "openai>=1.0.0"
Once the library is installed, you can instantiate an OpenAI client pointing to kluster.ai with the following code and replacing INSERT_API_KEY
:
from openai import OpenAI
client = OpenAI(
base_url="https://api.kluster.ai/v1",
api_key="INSERT_API_KEY", # Replace with your actual API key
)
Check the kluster.ai OpenAI compatibility page for detailed information about the integration.
API request limits#
The following limits apply to API requests based on your plan tier (notation is free tier | standard tier
):
Model | Context size |
Max output |
Max batch requests |
Concurrent requests |
Requests per minute |
---|---|---|---|---|---|
DeepSeek R1 | 32k | 162k | 4k | 162k | <1000 | No limit | 2 | 10 | 1 | 60 |
DeepSeek V3 | 32k | 131k | 4k | 131k | <1000 | No limit | 2 | 10 | 1 | 60 |
DeepSeek V3 0324 | 32k | 131k | 4k | 131k | <1000 | No limit | 2 | 10 | 1 | 60 |
Gemma 3 27B | 32k | 32k | 4k | 8k | <1000 | No limit | 2 | 10 | 1 | 60 |
Llama 3.1 8B | 32k | 131k | 4k | 131k | <1000 | No limit | 2 | 10 | 1 | 60 |
Llama 3.1 405B | 32k | 131k | 4k | 131k | <1000 | No limit | 2 | 10 | 1 | 60 |
Llama 3.3 70B | 32k | 131k | 4k | 131k | <1000 | No limit | 2 | 10 | 1 | 60 |
Qwen 2.5 7B | 32k | 32k | 4k | 8k | <1000 | No limit | 2 | 10 | 1 | 60 |
Where to go next#
-
Guide Real-time inference
Build AI-powered applications that deliver instant, real-time responses.
-
Guide Batch inference
Process large-scale data efficiently with AI-powered batch inference.
-
Reference API reference
Explore the complete kluster.ai API documentation and usage details.