API reference

API request limits

The following limits apply to API requests based on your plan:

TrialCoreScaleEnterprise

Model	Context size [tokens]	Max output [tokens]	Max batch requests	Concurrent requests	Requests per minute	Hosted fine-tuned models
DeepSeek-R1	32k	4k	1000	20	30	1
DeepSeek-R1-0528	32k	4k	1000	20	30	1
DeepSeek-V3-0324	32k	4k	1000	20	30	1
Gemma 3 27B	32k	4k	1000	20	30	1
Meta Llama 3.1 8B	32k	4k	1000	20	30	1
Meta Llama 3.3 70B	32k	4k	1000	20	30	1
Meta Llama 4 Maverick	32k	4k	1000	20	30	1
Meta Llama 4 Scout	32k	4k	1000	20	30	1
Mistral NeMo	32k	4k	1000	20	30	1
Qwen2.5-VL 7B	32k	4k	1000	20	30	1
Qwen3-235B-A22B	32k	4k	1000	20	30	1

Model	Context size [tokens]	Max output [tokens]	Max batch requests	Concurrent requests	Requests per minute	Hosted fine-tuned models
DeepSeek-R1	163k	163k	100k	100	600	10
DeepSeek-R1-0528	163k	163k	100k	100	600	10
DeepSeek-V3-0324	163k	163k	100k	100	600	10
Gemma 3 27B	64k	8k	100k	100	600	10
Meta Llama 3.1 8B	131k	131k	100k	100	600	10
Meta Llama 3.3 70B	131k	131k	100k	100	600	10
Meta Llama 4 Maverick	1M	1M	100k	100	600	10
Meta Llama 4 Scout	131k	131k	100k	100	600	10
Mistral NeMo	131k	131k	100k	100	600	10
Qwen2.5-VL 7B	32k	32k	100k	100	600	10
Qwen3-235B-A22B	40k	40k	100k	100	600	10

Model	Context size [tokens]	Max output [tokens]	Max batch requests	Concurrent requests	Requests per minute	Hosted fine-tuned models
DeepSeek-R1	163k	163k	500k	100	1200	25
DeepSeek-R1-0528	163k	163k	500k	100	1200	25
DeepSeek-V3-0324	163k	163k	500k	100	1200	25
Gemma 3 27B	64k	8k	500k	100	1200	25
Meta Llama 3.1 8B	131k	131k	500k	100	1200	25
Meta Llama 3.3 70B	131k	131k	500k	100	1200	25
Meta Llama 4 Maverick	1M	1M	500k	100	1200	25
Meta Llama 4 Scout	131k	131k	500k	100	1200	25
Mistral NeMo	131k	131k	500k	100	1200	25
Qwen2.5-VL 7B	32k	32k	500k	100	1200	25
Qwen3-235B-A22B	40k	40k	500k	100	1200	25

Model	Context size [tokens]	Max output [tokens]	Max batch requests	Concurrent requests	Requests per minute	Hosted fine-tuned models
DeepSeek-R1	163k	163k	Unlimited	100	Unlimited	Unlimited
DeepSeek-R1-0528	163k	163k	Unlimited	100	Unlimited	Unlimited
DeepSeek-V3-0324	163k	163k	Unlimited	100	Unlimited	Unlimited
Gemma 3 27B	64k	8k	Unlimited	100	Unlimited	Unlimited
Meta Llama 3.1 8B	131k	131k	Unlimited	100	Unlimited	Unlimited
Meta Llama 3.3 70B	131k	131k	Unlimited	100	Unlimited	Unlimited
Meta Llama 4 Maverick	1M	1M	Unlimited	100	Unlimited	Unlimited
Meta Llama 4 Scout	131k	131k	Unlimited	100	Unlimited	Unlimited
Mistral NeMo	131k	131k	Unlimited	100	Unlimited	Unlimited
Qwen2.5-VL 7B	32k	32k	Unlimited	100	Unlimited	Unlimited
Qwen3-235B-A22B	40k	40k	Unlimited	100	Unlimited	Unlimited

---

Chat

Create chat completion

POST https://api.kluster.ai/v1/chat/completions

To create a chat completion, send a request to the chat/completions endpoint. Please ensure your request is compliant with the API request limits.

Request

model string required

ID of the model to use. You can use the models endpoint to retrieve the list of supported models.

---

messages array required

A list of messages comprising the conversation so far. The messages object can be one of system, user, or assistant.

Show possible types

System message object

Show properties

content string or array

The contents of the system message.

---

role string or null required

The role of the messages author, in this case, system.

---

User message object

Show properties

content string or array

The contents of the user message.

---

role string or null required

The role of the messages author, in this case, user.

---

Assistant message object

Show properties

content string or array

The contents of the assistant message.

---

role string or null required

The role of the messages author, in this case, assistant.

---

store boolean or null

Whether or not to store the output of this chat completion request. Defaults to false.

---

metadata object Required

Set of key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API.

Show properties

@kluster.ai object

kluster.ai-specific options for the request.

Show properties

callback_url string

A URL to which the system will send a callback when the request is complete.

---

async boolean

Indicates whether the request should be asynchronous. For more information, see the Submit an async request section.

---

strict_completion_window boolean

Indicates whether the request must be completed within the specified completion_window. If enabled and the request isn't completed within the window, it will be considered unsuccessful.

---

completion_window string

The time frame within which the batch should be processed. The supported completion windows are 24, 48, and 72 hours to accommodate a range of use cases and budget requirements.

Learn more about how completion window selection affects cost by visiting the pricing section of the kluster.ai website.

---

additionalProperties any

Allows any other properties to be included in the metadata object without enforcing a specific schema for them. These properties can have any key and any value type.

---

frequency_penalty number or null

Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood of repeating the same line verbatim. Defaults to 0.

---

logit_bias map

Modify the likelihood of specified tokens appearing in the completion. Defaults to null.

Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model, but values between -1 and 1 should decrease or increase the likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token.

---

logprobs boolean or null

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message. Defaults to false.

---

top_logprobs integer or null

An integer between 0 and 20 specifying the number of most likely tokens to return at each token position, each with an associated log probability. logprobs must be set to true if this parameter is used.

---

max_completion_tokens integer or null

An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.

---

presence_penalty number or null

Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. Defaults to 0.

---

seed integer or null

If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed.

---

stop string or array or null

Up to four sequences where the API will stop generating further tokens. Defaults to null.

---

stream boolean or null

If set, partial message deltas will be sent. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message. Defaults to false.

---

temperature number or null

The sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Defaults to 1.

It is generally recommended to alter this or top_p but not both.

---

top_p number or null

An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. Defaults to 1.

It is generally recommended to alter this or temperature but not both.

---

Returns

The created Chat completion object.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1", 
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

chat_completion = client.chat.completions.create(
    model="klusterai/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of Argentina?"},
    ],
)

print(chat_completion.to_dict())

Example request

curl -s https://api.kluster.ai/v1/chat/completions \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "model": "klusterai/Meta-Llama-3.1-8B-Instruct-Turbo",
        "messages": [
            {
                "role": "system",
                "content": "You are a helpful assistant."
            },
            {
                "role": "user",
                "content": "What is the capital of Argentina?"
            }
        ]
    }'

Response

{
    "id": "chat-d187c103e189483485b3bcd3eb899c62",
    "object": "chat.completion",
    "created": 1736136422,
    "model": "klusterai/Meta-Llama-3.1-8B-Instruct-Turbo",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "The capital of Argentina is Buenos Aires.",
                "tool_calls": []
            },
            "logprobs": null,
            "finish_reason": "stop",
            "stop_reason": null
        }
    ],
    "usage": {
        "prompt_tokens": 48,
        "total_tokens": 57,
        "completion_tokens": 9
    },
    "prompt_logprobs": null
}

---

Chat completion object

id string

Unique identifier for the chat completion.

---

object string

The object type, which is always chat.completion.

---

created integer

The Unix timestamp (in seconds) of when the chat completion was created.

---

model string

The model used for the chat completion. You can use the models endpoint to retrieve the list of supported models.

---

choices array

A list of chat completion choices.

Show properties

index integer

The index of the choice in the list of returned choices.

---

message object

A chat completion message generated by the model. Can be one of system, user, or assistant.

Show properties

content string or array

The contents of the message.

---

role string or null

The role of the messages author. Can be one of system, user, or assistant.

---

logprobs boolean or null

Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the content of message. Defaults to false.

---

finish_reason string

The reason the model stopped generating tokens. This will be stop if the model hit a natural stop point or a provided stop sequence, length if the maximum number of tokens specified in the request was reached, content_filter if content was omitted due to a flag from our content filters, tool_calls if the model called a tool, or function_call (deprecated) if the model called a function.

---

stop_reason string or null

The reason the model stopped generating text.

---

usage object

Usage statistics for the completion request.

Show properties

completion_tokens integer

Number of tokens in the generated completion.

---

prompt_tokens integer

Number of tokens in the prompt.

---

total_tokens integer

Total number of tokens used in the request (prompt + completion).

Chat completion object

{
    "id": "chat-d187c103e189483485b3bcd3eb899c62",
    "object": "chat.completion",
    "created": 1736136422,
    "model": "klusterai/Meta-Llama-3.1-8B-Instruct-Turbo",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "The capital of Argentina is Buenos Aires.",
                "tool_calls": []
            },
            "logprobs": null,
            "finish_reason": "stop",
            "stop_reason": null
        }
    ],
    "usage": {
        "prompt_tokens": 48,
        "total_tokens": 57,
        "completion_tokens": 9
    },
    "prompt_logprobs": null
}

---

Async

Submit an async request

Asynchronous inference is a cost-effective option when you don't need immediate results, such as for workloads that vary or have unpredictable timelines. Submitting an asynchronous inference request to the chat completions endpoint works like submitting a real-time request. The main difference is that you include a metadata object specifying the request as async and a completion_window, defining the time window you expect the response.

Request

model string required

ID of the model to use. You can use the models endpoint to retrieve the list of supported models.

---

messages array required

A list of messages comprising the conversation so far. The messages object can be one of system, user, or assistant.

---

endpoint string required

The endpoint to be used for all requests in the batch. Currently, only /v1/chat/completions is supported.

---

metadata object Required

Set of key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API.

Show properties

@kluster.ai object Required

kluster.ai-specific options for the request.

Show properties

callback_url string

A URL to which the system will send a callback when the request is complete.

---

async boolean Required

Indicates whether the request should be asynchronous.

---

strict_completion_window boolean

Indicates whether the request must be completed within the specified completion_window. If enabled and the request isn't completed within the window, it will be considered unsuccessful.

---

completion_window string required

The time frame within which the batch should be processed. The supported completion windows are 24, 48, and 72 hours to accommodate a range of use cases and budget requirements.

Learn more about how completion window selection affects cost by visiting the pricing section of the kluster.ai website.

---

additionalProperties any

Allows any other properties to be included in the metadata object without enforcing a specific schema for them. These properties can have any key and any value type.

---

Returns

The Batch object including the job ID. All async jobs are treated as batch jobs but submitted through the real-time chat completions endpoint.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1",
    api_key="INSERT_API_KEY"  # Replace with your actual API key
)

# Create a chat completion request with async flag in metadata
chat_completion = client.chat.completions.create(
    model="google/gemma-3-27b-it",
    messages=[
        {"role": "user", "content": "Please give me your honest opinion on the best stock as an investment."}
    ],
    metadata={
        "@kluster.ai": {
            "async": True,
            "completion_window": "24h"
        }
    }
)

print(chat_completion.to_dict())

Example request

    curl -s https://api.kluster.ai/v1/chat/completions \
    -H "Authorization: Bearer INSERT_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
  "model": "google/gemma-3-27b-it",
  "messages": [
    {
      "role": "user",
      "content": "Please give me your honest opinion on the best stock as an investment."
    }
  ],
  "metadata": {
    "@kluster.ai": {
      "async": true,
      "completion_window": "24h"
    }
  }
}'

Response

{
    "id": "67783976bf636f79b49643ee_1743267849127",
    "object": "chat.completion",
    "created": 1743267849,
    "model": "google/gemma-3-27b-it",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "Your request has been queued for adaptive inference. Check your batch jobs for results."
            },
            "finish_reason": "stop"
        }
    ]
}

---

Batch

Submit a batch job

POST https://api.kluster.ai/v1/batches

To submit a batch job, send a request to the batches endpoint. Please ensure your request is compliant with the API request limits.

Request

input_file_id string required

The ID of an uploaded file that contains requests for the new batch.

Warning

For the free tier, the maximum number of batch request (lines in the JSONL file) must be less than 1000, and each file must not exceed 100 MB. For the standard tier, there is no limit to the number of batch requests, but the maximum batch file size is 100 MB per file.

---

endpoint string required

The endpoint to be used for all requests in the batch. Currently, only /v1/chat/completions is supported.

---

completion_window string required

The supported completion windows are 24, 48, and 72 hours to accommodate a range of use cases and budget requirements. The code samples provided utilize the 24-hour completion window.

Learn more about how completion window selection affects cost by visiting the pricing section of the kluster.ai website.

---

metadata Object or null

Custom metadata for the batch.

---

Returns

The created Batch object.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1",
    api_key="INSERT_API_KEY",  # Replace with your actual API key
)

batch_request = client.batches.create(
    input_file_id="myfile-123",
    endpoint="/v1/chat/completions",
    completion_window="24h",
)

print(batch_request.to_dict())

Example request

curl -s https://api.kluster.ai/v1/batches \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
    "input_file_id": "myfile-123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
    }'

Response

{
    "id": "mybatch-123",
    "completion_window": "24h",
    "created_at": 1733832777,
    "endpoint": "/v1/chat/completions",
    "input_file_id": "myfile-123",
    "object": "batch",
    "status": "validating",
    "cancelled_at": null,
    "cancelling_at": null,
    "completed_at": null,
    "error_file_id": null,
    "errors": null,
    "expired_at": null,
    "expires_at": 1733919177,
    "failed_at": null,
    "finalizing_at": null,
    "in_progress_at": null,
    "metadata": {},
    "output_file_id": null,
    "request_counts": {
        "completed": 0,
        "failed": 0,
        "total": 0
    }
}

---

Retrieve a batch

GET https://api.kluster.ai/v1/batches/{batch_id}

To retrieve a batch job, send a request to the batches endpoint with your batch_id.

You can also monitor jobs in the Batch tab of the kluster.ai platform UI.

Path parameters

batch_id string required

The ID of the batch to retrieve.

---

Returns

The Batch object matching the specified batch_id.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1",
    api_key="INSERT_API_KEY",  # Replace with your actual API key
)

client.batches.retrieve("mybatch-123")

Example request

curl -s https://api.kluster.ai/v1/batches/mybatch-123 \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json"

Response

{
  "id": "mybatch-123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "myfile-123",
  "completion_window": "24h",
  "status": "completed",
  "output_file_id": "myfile-123-output",
  "error_file_id": null,
  "created_at": "1733832777",
  "in_progress_at": "1733832777",
  "expires_at": "1733919177",
  "finalizing_at": "1733832781",
  "completed_at": "1733832781",
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": null,
  "cancelled_at": null,
  "request_counts": {
    "total": 4,
    "completed": 4,
    "failed": 0
  },
  "metadata": {}
}

---

Cancel a batch

POST https://api.kluster.ai/v1/batches/{batch_id}/cancel

To cancel a batch job that is currently in progress, send a request to the cancel endpoint with your batch_id. Note that cancellation may take up to 10 minutes to complete, during which time the status will show as cancelling.

Path parameters

batch_id string required

The ID of the batch to cancel.

---

Returns

The Batch object matching the specified ID.

Pythoncurl

Example

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1",  
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

client.batches.cancel("mybatch-123") # Replace with your batch id

Example

curl -s https://api.kluster.ai/v1/batches/$BATCH_ID/cancel \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -X POST

Response

{
  "id": "mybatch-123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "myfile-123",
  "completion_window": "24h",
  "status": "cancelling",
  "output_file_id": "myfile-123-output",
  "error_file_id": null,
  "created_at": "1730821906",
  "in_progress_at": "1730821911",
  "expires_at": "1730821906",
  "finalizing_at": null,
  "completed_at": null,
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": "1730821906",
  "cancelled_at": null,
  "request_counts": {
    "total": 3,
    "completed": 3,
    "failed": 0
  },
  "metadata": {}
}

---

List all batch jobs

GET https://api.kluster.ai/v1/batches

To list all batch jobs, send a request to the batches endpoint without specifying a batch_id. To constrain the query response, you can also use a limit parameter.

Query parameters

after string

A cursor for use in pagination. after is an object ID that defines your place in the list. For instance, if you make a list request and receive 100 objects, ending with obj_foo, your subsequent call can include after=obj_foo in order to fetch the next page of the list.

---

limit integer

A limit on the number of objects to be returned. Limit can range between 1 and 100. Default is 20.

---

Returns

A list of paginated Batch objects.

The status of a batch object can be one of the following:

Status	Description
validating	The input file is being validated.
failed	The input file failed the validation process.
in_progress	The input file was successfully validated and the batch is in progress.
finalizing	The batch job has completed and the results are being finalized.
completed	The batch has completed and the results are ready.
expired	The batch was not completed within the 24-hour time window.
cancelling	The batch is being cancelled (may take up to 10 minutes).
cancelled	The batch was cancelled.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1", 
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

print(client.batches.list(limit=2).to_dict())

Example request

curl -s https://api.kluster.ai/v1/batches \
    -H "Authorization: Bearer $API_KEY"

Response

{
"object": "list",
"data": [
    {
    "id": "mybatch-123",
    "object": "batch",
    "endpoint": "/v1/chat/completions",
    "errors": null,
    "input_file_id": "myfile-123",
    "completion_window": "24h",
    "status": "completed",
    "output_file_id": "myfile-123-output",
    "error_file_id": null,
    "created_at": "1733832777",
    "in_progress_at": "1733832777",
    "expires_at": "1733919177",
    "finalizing_at": "1733832781",
    "completed_at": "1733832781",
    "failed_at": null,
    "expired_at": null,
    "cancelling_at": null,
    "cancelled_at": null,
    "request_counts": {
        "total": 4,
        "completed": 4,
        "failed": 0
    },
    "metadata": {}
    },
{ ... },
],
"first_id": "mybatch-123",
"last_id": "mybatch-789",
"has_more": false,
"count": 1,
"page": 1,
"page_count": -1,
"items_per_page": 9223372036854775807
}

---

Batch object

id string

The ID of the batch.

---

object string

The object type, which is always batch.

---

endpoint string

The kluster.ai API endpoint used by the batch.

---

errors object

Show properties

object string

The object type, which is always list.

---

data array

Show properties

code string

An error code identifying the error type.

---

message string

A human-readable message providing more details about the error.

---

param string or null

The name of the parameter that caused the error, if applicable.

---

line integer or null

The line number of the input file where the error occurred, if applicable.

---

input_file_id string

The ID of the input file for the batch.

---

completion_window string

The time frame within which the batch should be processed.

---

status string

The current status of the batch.

---

output_file_id string

The ID of the file containing the outputs of successfully executed requests.

---

error_file_id string

The ID of the file containing the outputs of requests with errors.

---

created_at integer

The Unix timestamp (in seconds) for when the batch was created.

---

in_progress_at integer

The Unix timestamp (in seconds) for when the batch started processing.

---

expires_at integer

The Unix timestamp (in seconds) for when the batch will expire.

---

finalizing_at integer

The Unix timestamp (in seconds) for when the batch started finalizing.

---

completed_at integer

The Unix timestamp (in seconds) for when the batch was completed.

---

failed_at integer

The Unix timestamp (in seconds) for when the batch failed.

---

expired_at integer

The Unix timestamp (in seconds) for when the batch expired.

---

cancelling_at integer

The Unix timestamp (in seconds) for when the batch started cancelling.

---

cancelled_at integer

The Unix timestamp (in seconds) for when the batch was cancelled.

---

request_counts object

The request counts for different statuses within the batch.

Show properties

total integer

Total number of requests in the batch.

---

completed integer

Number of requests that have been completed successfully.

---

failed integer

Number of requests that have failed.

Batch object

{
    "id": "mybatch-123",
    "completion_window": "24h",
    "created_at": 1733832777,
    "endpoint": "/v1/chat/completions",
    "input_file_id": "myfile-123",
    "object": "batch",
    "status": "validating",
    "cancelled_at": null,
    "cancelling_at": null,
    "completed_at": null,
    "error_file_id": null,
    "errors": null,
    "expired_at": null,
    "expires_at": 1733919177,
    "failed_at": null,
    "finalizing_at": null,
    "in_progress_at": null,
    "metadata": {},
    "output_file_id": null,
    "request_counts": {
        "completed": 0,
        "failed": 0,
        "total": 0
    }
}

---

The request input object

The per-line object of the batch input file.

custom_id string

A developer-provided per-request ID.

---

method string

The HTTP method to be used for the request. Currently, only POST is supported.

---

url string

The /v1/chat/completions endpoint.

---

body map

The JSON body of the input file.

Request input object

[
    {
        "custom_id": "request-1",
        "method": "POST",
        "url": "/v1/chat/completions",
        "body": {
            "model": "klusterai/Meta-Llama-3.1-8B-Instruct-Turbo",
            "messages": [
                {
                    "role": "system",
                    "content": "You are a helpful assistant."
                },
                {
                    "role": "user",
                    "content": "What is the capital of Argentina?"
                }
            ],
            "max_tokens": 1000
        }
    }
]

---

The request output object

The per-line object of the batch output files.

id string

A unique identifier for the batch request.

---

custom_id string

A developer-provided per-request ID that will be used to match outputs to inputs.

---

response object or null

Show properties

status_code integer

The HTTP status code of the response.

---

request_id string

A unique identifier for the request. You can reference this request ID if you need to contact support for assistance.

---

body map

The JSON body of the response.

---

error object or null

For requests that failed with a non-HTTP error, this will contain more information on the cause of the failure.

Show properties

code string

A machine-readable error code.

---

message string

A human-readable error message.

Request output object

{
    "id": "batch-req-123",
    "custom_id": "request-1",
    "response": {
        "status_code": 200,
        "request_id": "req-123",
        "body": {
            "id": "chatcmpl-5a5ba6c6-2f95-4136-815b-23275c4f1efb",
            "object": "chat.completion",
            "created": 1737472126,
            "model": "klusterai/Meta-Llama-3.1-8B-Instruct-Turbo",
            "choices": [
                {
                    "index": 0,
                    "message": {
                        "role": "assistant",
                        "content": "The capital of Argentina is Buenos Aires.",
                        "tool_calls": []
                    },
                    "logprobs": null,
                    "finish_reason": "stop",
                    "stop_reason": null
                }
            ],
            "usage": {
                "prompt_tokens": 48,
                "total_tokens": 57,
                "completion_tokens": 9,
                "prompt_tokens_details": null
            },
            "prompt_logprobs": null
        }
    }
}

---

Files

Upload files

POST https://api.kluster.ai/v1/files/

Upload a JSON Lines file to the files endpoint. Please ensure your file is compliant with the API request limits.

You can also view all your uploaded files in the Files tab of the kluster.ai platform.

Request

file file required

The file object (not file name) to be uploaded.

Warning

For the free tier, the maximum number of batch request (lines in the JSONL file) must be less than 1000, and each file must not exceed 100 MB. For the standard tier, there is no limit to the number of batch requests, but the maximum batch file size is 100 MB per file.

---

purpose string required

The intended purpose of the uploaded file. Use batch for the batch API.

---

Returns

The uploaded File object.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1", 
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

batch_input_file = client.files.create(
    file=open(file_name, "rb"),
    purpose="batch"
)

print(batch_input_file.to_dict())

Example request

curl -s https://api.kluster.ai/v1/files \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: multipart/form-data" \
    -F "file=@my_batch_request.jsonl" \
    -F "purpose=batch"

Response

{
  "id": "myfile-123",
  "bytes": 2797,
  "created_at": "1733832768",
  "filename": "my_batch_request.jsonl",
  "object": "file",
  "purpose": "batch"
}

---

Retrieve file content

GET https://api.kluster.ai/v1/files/{output_file_id}/content

To retrieve the content of your batch jobs output file, send a request to the files endpoint specifying the output_file_id. The output file will be a JSONL file, where each line contains the custom_id from your input file request, and the corresponding response.

Path parameters

file_id string required

The ID of the file to use for this request

---

Returns

The file content. Refer to the input and output format specifications for batch requests.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1", 
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

# Get the status of the batch, which returns the output_file_id
batch_status = client.batches.retrieve(batch_request.id)

# Check if the batch completed successfully
if batch_status.status.lower() == "completed":
    # Retrieve the results
    result_file_id = batch_status.output_file_id
    results = client.files.content(result_file_id).content

    # Save results to a file
    result_file_name = "batch_results.jsonl"
    with open(result_file_name, "wb") as file:
        file.write(results)
    print(f"Results saved to {result_file_name}")
else:
    print(f"Batch failed with status: {batch_status.status}")

Example request

curl -s https://api.kluster.ai/v1/files/kluster-output-file-123/content \
    -H "Authorization: Bearer $API_KEY" > batch_output.jsonl

---

File object

id string

The file identifier, which can be referenced in the API endpoints.

---

object string

The object type, which is always file.

---

bytes integer

The size of the file, in bytes.

---

created_at integer

The Unix timestamp (in seconds) for when the file was created.

---

filename string

The name of the file.

---

purpose string

The intended purpose of the file. Currently, only batch is supported.

File object

{
  "id": "myfile-123",
  "bytes": 2797,
  "created_at": "1733832768",
  "filename": "my_batch_request.jsonl",
  "object": "file",
  "purpose": "batch"
}

---

Uploads

Create upload

POST https://api.kluster.ai/v1/uploads

Creates an intermediate upload object that accepts additional parts. Each upload can hold up to 8 GB of data and expires one hour after creation.

Upon completion, the uploaded parts are assembled into a single File object that functions as a regular file across the platform.

Request

bytes integer required

The number of bytes in the file you are uploading.

---

filename string required

The name of the file to upload.

---

mime_type string required

The MIME type of the file.

This must fall within the supported MIME types for your file purpose.

---

purpose string required

The intended purpose of the uploaded file. Accepted values are assistants, vision, batch, batch_output, fine-tune, or fine-tune-results.

---

Returns

The Upload object with status pending.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1", 
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

upload = client.uploads.create(
    purpose="fine-tune",
    filename="training_examples.jsonl",
    bytes=2147483648,
    mime_type="text/jsonl"
)

print(upload.to_dict())

Example request

curl https://api.kluster.ai/v1/uploads \
  -H "Authorization: Bearer INSERT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "purpose": "batch",
    "filename": "training_examples.jsonl",
    "bytes": 2147483648,
    "mime_type": "text/jsonl"
  }'

Response

{
  "id": "67feae39d63649ef421416f8",
  "object": "upload",
  "bytes": 2147483648,
  "created_at": 1744743993,
  "filename": "training_examples.jsonl",
  "status": "pending",
  "expires_at": 1744747593
}

---

Add upload part

POST https://api.kluster.ai/v1/uploads/{upload_id}/parts

Adds an upload part to an Upload object. A upload part represents a chunk of bytes from the file you are trying to upload.

Each upload part can be at most 64 MB, and you can add upload parts until you hit the upload maximum of 8 GB.

It is possible to add multiple upload parts in parallel. You can decide the intended order of the upload parts when you complete the upload.

Path parameters

upload_id string required

The ID of the upload.

---

Request

data file required

The chunk of bytes for this upload part.

---

Returns

The Upload part object.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1", 
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

# Open the file you want to upload and replace upload ID
with open("training_examples.jsonl", "rb") as file:
    part = client.uploads.parts.create(
        upload_id="INSERT_UPLOAD_ID",
        data=file
    )

print(part.to_dict())

Example request

curl -s https://api.kluster.ai/v1/uploads/INSERT_UPLOAD_ID/parts \
    -H "Authorization: Bearer $API_KEY" \
    -F "data=@training_examples.jsonl"

Response

{
  "id": "67feb0c9f6f1dad39fec8d39",
  "object": "upload.part",
  "created_at": 1744744649,
  "upload_id": "67feae39d63649ef421416f8"
}

---

Complete upload

POST https://api.kluster.ai/v1/uploads/{upload_id}/complete

Completes the upload.

Within the returned Upload object, there is a nested file object that is ready to use in the rest of the platform.

You can specify the order of the upload parts by passing in an ordered list of the part IDs.

The number of bytes uploaded upon completion must match the number of bytes initially specified when creating the upload object. No upload parts may be added after an upload is completed.

Path parameters

upload_id string required

The ID of the upload.

---

Request

part_ids array required

The ordered list of Part IDs.

---

md5 string

The optional md5 checksum for the file contents to verify if the bytes uploaded matches what you expect.

---

Returns

The Upload object with status completed with an additional file property containing the created usable file object. If the complete upload operation is unsuccessful, the upload object will remain in pending state.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1", 
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

completed_upload = client.uploads.complete(
    upload_id="INSERT_UPLOAD_ID",
    part_ids=["INSERT_PART_ID", "INSERT_PART_ID"]
)

print(completed_upload.to_dict())

Example request

curl -s https://api.kluster.ai/v1/uploads/INSERT_UPLOAD_ID/complete \
    -H "Authorization: Bearer $API_KEY" \
    -d '{
        "part_ids": ["INSERT_PART_ID", "INSERT_PART_ID"]
    }'

Response

{
  "id": "67feae39d63649ef421416f8",
  "object": "upload",
  "bytes": 2147483648,
  "created_at": 1744743993,
  "filename": "training_examples.jsonl",
  "status": "completed",
  "expires_at": 1744747593,
  "file": {
    "id": "67feb2bff6f1dad39feca157",
    "object": "file",
    "created_at": 1744745151,
    "filename": "67783976bf636f79b49643ee/d0b911d4-1fcf-4809-b8af-ca65c6a689a0-e7726314-de6d-4cb7-bed7-822cc12f9bbc",
    "purpose": "batch",
    "bytes": 1776
  }
}

---

Cancel upload

POST https://api.kluster.ai/v1/uploads/{upload_id}/cancel

Cancels the upload. No upload parts may be added after an upload is cancelled.

Path parameters

upload_id string required

The ID of the upload.

---

Returns

The upload object with status cancelled.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1", 
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

cancelled_upload = client.uploads.cancel(
    upload_id="INSERT_UPLOAD_ID"
)

print(cancelled_upload.to_dict())

Example request

curl -X POST -s https://api.kluster.ai/v1/uploads/INSERT_UPLOAD_ID/cancel \
-H "Authorization: Bearer INSERT_API_KEY"

Response

{
  "id": "67feb41fb779611ad5b3b635",
  "object": "upload",
  "bytes": 2147483648,
  "created_at": 1744745503,
  "filename": "training_examples.jsonl",
  "status": "cancelled",
  "expires_at": 1744749103
}

---

Upload object

id string

The upload unique identifier, which can be referenced in API endpoints.

---

object string

The object type, which is always upload.

---

bytes integer

The intended number of bytes to be uploaded.

---

created_at integer

The Unix timestamp (in seconds) for when the upload was created.

---

expires_at integer

The Unix timestamp (in seconds) for when the upload will expire.

---

filename string

The name of the file to upload.

---

purpose string

The intended purpose of the uploaded file.

---

status string

The current status of the upload. Possible values are pending, completed, or cancelled.

---

file undefined or null

The ready file object after the upload is completed.

Upload object

{
  "id": "upload_abc123",
  "object": "upload",
  "bytes": 2147483648,
  "created_at": 1719184911,
  "filename": "training_examples.jsonl",
  "purpose": "fine-tune",
  "status": "completed",
  "expires_at": 1719127296,
  "file": {
    "id": "file-xyz321",
    "object": "file",
    "bytes": 2147483648,
    "created_at": 1719186911,
    "filename": "training_examples.jsonl",
    "purpose": "fine-tune"
  }
}

---

Upload part object

id string

The upload part unique identifier, which can be referenced in API endpoints.

---

object string

The object type, which is always upload.part.

---

created_at integer

The Unix timestamp (in seconds) for when the UploadPart was created.

---

upload_id string

The ID of the Upload object that this upload part was added to.

Upload part object

{
  "id": "part_def456",
  "object": "upload.part",
  "created_at": 1719185911,
  "upload_id": "upload_abc123"
}

---

Embeddings

Create embeddings

POST https://api.kluster.ai/v1/embeddings

Creates an embedding vector representing the input text. Machine learning models and algorithms can easily consume these vector representations to understand semantic relationships between pieces of text.

Request

input string or array required

Input text to embed, encoded as a string or array of tokens. To embed multiple inputs in a single request, pass an array of strings or an array of token arrays. The input must not exceed the max input tokens for the model (8192 tokens for BAAI/bge-m3), it cannot be an empty string, and any array must be 2048 dimensions or less.

---

model string required

ID of the model to use.

---

encoding_format string

The format to return the embeddings in. Can be either float or base64. Defaults to float.

---

dimensions not supported

Please note, the embeddings endpoint doesn’t support the dimensions parameter. Models such as BAAI/bge‑m3 always return a fixed 1024‑dimensional vector. Supplying dimensions will trigger a 400 Bad Request.

Returns

A list of Embedding objects.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1",
    api_key="INSERT_API_KEY",  # Replace with your actual API key
)

response = client.embeddings.create(
    model="BAAI/bge-m3",
    input="The food was delicious and the waiter...",
    encoding_format="float"
)

print(response)

Example request

curl -s https://api.kluster.ai/v1/embeddings \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
    "input": "The food was delicious and the waiter...",
    "model": "BAAI/bge-m3",
    "encoding_format": "float"
    }'

Response

{
  "object": "list",
  "created": 1744935248,
  "model": "BAAI/bge-m3",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [
        0.00885772705078125,
        -0.0010204315185546875,
        -0.045135498046875,
        0.00478363037109375,
        -0.02642822265625,
        /* ... truncated for brevity ... */
        -0.0168304443359375,
        0.0229339599609375,
        0.007648468017578125,
        -0.03875732421875,
        0.05487060546875
      ]
    }
  ],
  "usage": {
    "prompt_tokens": 13
  }
}

---

Embedding object

Represents an embedding vector returned by the embeddings endpoint.

Properties

embedding array

The embedding vector, which is a list of floats.

---

index integer

The index of the embedding in the list of embeddings.

---

object string

The object type, which is always "embedding".

The embedding object

{
  "object": "embedding",
  "embedding": [
    0.00885772705078125,
    -0.0010204315185546875,
    -0.045135498046875,
    0.00478363037109375,
    -0.02642822265625,
    /* ... truncated for brevity ... */
    -0.0168304443359375,
    0.0229339599609375,
    0.007648468017578125,
    -0.03875732421875,
    0.05487060546875
  ],
  "index": 0
}

---

Models

List supported models

GET https://api.kluster.ai/v1/models

Lists the currently available models.

You can use this endpoint to retrieve a list of all available models for the kluster.ai API. If you have created any fine-tuned models, they will also appear when you query this endpoint.

Returns

id string

The model identifier, which can be referenced in the API endpoints.

---

created integer

The Unix timestamp (in seconds) when the model was created.

---

object string

The object type, which is always model.

---

owned_by string

The organization that owns the model.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="http://api.kluster.ai/v1",
    api_key="INSERT_API_KEY" # Replace with your actual API key
)

print(client.models.list().to_dict())

Example request

curl https://api.kluster.ai/v1/models \
    -H "Authorization: Bearer $API_KEY" 

Response

{
  "object": "list",
  "data": [
    {
      "id": "klusterai/Meta-Llama-3.1-8B-Instruct-Turbo",
      "object": "model",
      "created": 1731336610,
      "owned_by": "klusterai"
    },
    {
      "id": "klusterai/Meta-Llama-3.3-70B-Instruct-Turbo",
      "object": "model",
      "created": 1733777629,
      "owned_by": "klusterai"
    },
    {
      "id": "deepseek-ai/DeepSeek-R1",
      "object": "model",
      "created": 1737385699,
      "owned_by": "klusterai"
    },
    {
      "id": "Qwen/Qwen2.5-VL-7B-Instruct",
      "object": "model",
      "created": 1741303075,
      "owned_by": "qwen"
    },
    {
      "id": "deepseek-ai/DeepSeek-V3-0324",
      "object": "model",
      "created": 1742848965,
      "owned_by": "klusterai"
    },
    {
      "id": "google/gemma-3-27b-it",
      "object": "model",
      "created": 1742913870,
      "owned_by": "google"
    },
    {
      "id": "meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8",
      "object": "model",
      "created": 1743944115,
      "owned_by": "meta"
    },
    {
      "id": "meta-llama/Llama-4-Scout-17B-16E-Instruct",
      "object": "model",
      "created": 1743944115,
      "owned_by": "meta"
    }
  ]
}

---

Fine-tuning

Fine-tuning is the process of refining a pre-trained model on specialized data. By adjusting the parameters with new, domain-specific examples, the model performs better on targeted tasks while retaining the general knowledge learned in its original training.

Supported models

Currently, two base models are supported for fine-tuning:

• klusterai/Meta-Llama-3.1-8B-Instruct-Turbo - has a 64,000 tokens max context window, best for long-context tasks, cost-sensitive scenarios
• klusterai/Meta-Llama-3.3-70B-Instruct-Turbo - has a 32,000 tokens max context window, best for complex reasoning, high-stakes accuracy

Create a fine-tuning job

POST https://api.kluster.ai/v1/fine_tuning/jobs

To initiate a fine-tuning job for one of the supported models, first upload the dataset file (see Files section for instructions).

Request

training_file string required

ID of an uploaded file that will serve as training data. This file must have purpose="fine-tune".

---

model string required

The base model ID to fine-tune. Must be a fine-tunable model, for example meta-llama/Meta-Llama-3.1-8B-Instruct or meta-llama/Meta-Llama-3.3-70B-Instruct-Turbo.

---

validation_file string or null

Optionally specify a separate file to serve as your validation dataset.

---

hyperparameters object or null

Optionally specify an object containing hyperparameters for fine-tuning:

Show properties

batch_size number

The number of training examples processed in one forward/backward pass. Larger batch sizes reduce the frequency of weight updates per epoch, leading to more stable gradients but slower updates. Gradient accumulation is used, so larger batches may increase the duration of the job.

---

learning_rate_multiplier number

A multiplier for the base step size used in model weight updates. Lower values slow training but improve precision (helping avoid overshooting optimal weights or overfitting). Higher values speed up convergence but risk instability. Adjust carefully to balance training efficiency and model performance.

---

n_epochs number

The number of times the entire training dataset is passed through the model. More epochs can improve learning but risk overfitting if the model memorizes training data. Monitor validation metrics to determine the optimal number.

---

nickname string or null

Add a custom suffix that will be appended to the output model name. This can help identify a fine tuned model.

---

Returns

A Fine-tuning job object.

Pythoncurl

Example request

from openai import OpenAI

# Configure OpenAI client
client = OpenAI(
    base_url="https://api.kluster.ai/v1",
    api_key="INSERT_API_KEY"  # Replace with your actual API key
)

job = client.fine_tuning.jobs.create(
    training_file="INSERT_TRAINING_FILE_ID",  # ID from uploaded training file
    model="meta-llama/Meta-Llama-3.1-8B-Instruct",
    hyperparameters={
        "batch_size": 4,
        "learning_rate_multiplier": 1,
        "n_epochs": 3
    }
)
print(job.to_dict())

Example request

curl -X POST https://api.kluster.ai/v1/fine_tuning/jobs \
    -H "Authorization: Bearer INSERT_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
        "training_file": "INSERT_TRAINING_FILE_ID",
        "model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
        "hyperparameters": {
            "batch_size": 4,
            "learning_rate_multiplier": 1,
            "n_epochs": 3
        }
    }'

Response

{
  "object": "fine_tuning.job",
  "id": "67ae81b59b08392687ea5f69",
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "created_at": 1739489717,
  "result_files": [],
  "status": "queued",
  "training_file": "67ae81587772e8a89c8fd5cf",
  "hyperparameters": {
    "batch_size": 4,
    "learning_rate_multiplier": 1,
    "n_epochs": 3
  },
  "method": {
    "type": "supervised",
    "supervised": {
      "batch_size": 4,
      "learning_rate_multiplier": 1,
      "n_epochs": 3
    }
  },
  "integrations": []
}

Retrieve a fine-tuning job

GET https://api.kluster.ai/v1/fine_tuning/jobs/{fine_tuning_job_id}

Fetch details of a single fine-tuning job by specifying its fine_tuning_job_id.

Path parameters

fine_tuning_job_id string required

The ID of the fine-tuning job to retrieve.

---

Returns

A Fine-tuning job object.

Pythoncurl

Example request

from openai import OpenAI
client = OpenAI(
    base_url="https://api.kluster.ai/v1",
    api_key="INSERT_API_KEY"
)
job_details = client.fine_tuning.jobs.retrieve("INSERT_JOB_ID")
print(job_details.to_dict())

Example request

curl -s https://api.kluster.ai/v1/fine_tuning/jobs/INSERT_JOB_ID \
    -H "Authorization: Bearer INSERT_API_KEY"

Response

{
  "object": "fine_tuning.job",
  "id": "67ae81b59b08392687ea5f69",
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "created_at": 1739489717,
  "result_files": [],
  "status": "running",
  "training_file": "67ae81587772e8a89c8fd5cf",
  "hyperparameters": {
    "batch_size": 4,
    "learning_rate_multiplier": 1,
    "n_epochs": 3
  },
  "method": {
    "type": "supervised",
    "supervised": {
      "batch_size": 4,
      "learning_rate_multiplier": 1,
      "n_epochs": 3
    }
  },
  "integrations": []
}

List all fine-tuning jobs

GET https://api.kluster.ai/v1/fine_tuning/jobs

Retrieve a paginated list of all fine-tuning jobs.

Query parameters

after string

A cursor for use in pagination.

---

limit integer

A limit on the number of objects returned (1 to 100). Default is 20.

---

Returns

A paginated list of Fine-tuning job objects.

Pythoncurl

Example request

from openai import OpenAI

client = OpenAI(
    base_url="https://api.kluster.ai/v1",
    api_key="INSERT_API_KEY"
)

jobs = client.fine_tuning.jobs.list(limit=3)
print(jobs.to_dict())

Example request

curl -s https://api.kluster.ai/v1/fine_tuning/jobs \
    -H "Authorization: Bearer $API_KEY"

Response

{
  "object": "list",
  "data": [
    {
      "object": "fine_tuning.job",
      "id": "67ae81b59b08392687ea5f69",
      "model": "meta-llama/Llama-3.1-8B-Instruct",
      "created_at": 1739489717,
      "result_files": [],
      "status": "running",
      "training_file": "67ae81587772e8a89c8fd5cf",
      "hyperparameters": {
        "batch_size": 4,
        "learning_rate_multiplier": 1,
        "n_epochs": 2
      },
      "method": {
        "type": "supervised",
        "supervised": {
          "batch_size": 4,
          "learning_rate_multiplier": 1,
          "n_epochs": 2
        }
      },
      "trained_tokens": 3065,
      "integrations": []
    },
    {
      "object": "fine_tuning.job",
      "id": "67ae7f7d965c187d5cda039f",
      "model": "meta-llama/Llama-3.1-8B-Instruct",
      "created_at": 1739489149,
      "result_files": [],
      "status": "cancelled",
      "training_file": "67ae7f7c965c187d5cda0397",
      "hyperparameters": {
        "batch_size": 1,
        "learning_rate_multiplier": 1,
        "n_epochs": 10
      },
      "method": {
        "type": "supervised",
        "supervised": {
          "batch_size": 1,
          "learning_rate_multiplier": 1,
          "n_epochs": 10
        }
      },
      "integrations": []
    }
  ],
  "first_id": "67ae81b59b08392687ea5f69",
  "last_id": "67abefddbee1f22fb0a742ef",
  "has_more": true
}

Cancel a fine-tuning job

POST https://api.kluster.ai/v1/fine_tuning/jobs/{fine_tuning_job_id}/cancel

To cancel a job that is in progress, send a POST request to the cancel endpoint with the job ID.

Path parameters

fine_tuning_job_id string required

The ID of the fine-tuning job to cancel.

---

Returns

The Fine-tuning job object with updated status.

Pythoncurl

Example request

from openai import OpenAI
client = OpenAI(
    base_url="https://api.kluster.ai/v1",
    api_key="INSERT_API_KEY"
)
cancelled_job = client.fine_tuning.jobs.cancel("67ae7f7d965c187d5cda039f")
print(cancelled_job.to_dict())

Example request

curl -X POST https://api.kluster.ai/v1/fine_tuning/jobs/67ae7f7d965c187d5cda039f/cancel \
    -H "Authorization: Bearer INSERT_API_KEY" \
    -H "Content-Type: application/json"

Response

{
  "id": "67ae7f7d965c187d5cda039f",
  "object": "fine_tuning.job",
  "model": "meta-llama/Meta-Llama-3.1-8B-Instruct",
  "fine_tuned_model": null,
  "status": "cancelling",
  "created_at": 1738382911,
  "training_file": "file-123abc",
  "validation_file": null,
  "hyperparameters": {
    "batch_size": 4,
    "learning_rate_multiplier": 1,
    "n_epochs": 3
  },
  "metrics": {},
  "error": null
}

Fine-tuning job object

object string

The object type, which is always fine_tuning.job.

---

id string

Unique identifier for the fine-tuning job.

---

model string

ID of the base model being fine-tuned.

---

created_at integer

Unix timestamp (in seconds) when the fine-tuning job was created.

---

finished_at integer

Unix timestamp (in seconds) when the fine-tuning job was completed.

---

fine_tuned_model string or null

The ID of the resulting fine-tuned model if the job succeeded; otherwise null.

---

result_files array

Array of file IDs associated with the fine-tuning job results.

---

status string

The status of the fine-tuning job (e.g., pending, running, succeeded, failed, or cancelled).

---

training_file string

ID of the uploaded file used for training data.

---

hyperparameters object

Training hyperparameters used in the job (e.g., batch_size, n_epochs, learning_rate_multiplier).

---

method object

Details about the fine-tuning method used, including type and specific parameters.

---

trained_tokens integer

The total number of tokens processed during training.

---

integrations array

Array of integrations associated with the fine-tuning job.

Example

{
  "object": "fine_tuning.job",
  "id": "67ad3877720af9f9ba78b684",
  "model": "meta-llama/Llama-3.1-8B-Instruct",
  "created_at": 1739405431,
  "finished_at": 1739405521,
  "fine_tuned_model": "ft:meta-llama:Llama-3.1-8B-Instruct:personal:805b5d69",
  "result_files": [],
  "status": "succeeded",
  "training_file": "67ad38760272045e7006171b",
  "hyperparameters": {
    "batch_size": 4,
    "learning_rate_multiplier": 1,
    "n_epochs": 2
  },
  "method": {
    "type": "supervised",
    "supervised": {
      "batch_size": 4,
      "learning_rate_multiplier": 1,
      "n_epochs": 2
    }
  },
  "trained_tokens": 3065,
  "integrations": []
}

Verify

Reliability check

POST https://api.kluster.ai/v1/verify/reliability

The Verify endpoint evaluates whether an LLM answer is reliable or hallucinatory. It can operate in two modes:

1. General-knowledge verification – if you omit context, the service cross-checks the answer against publicly available sources.
2. Context validation mode – if you supply context, the answer is judged only against that reference text (perfect for RAG pipelines).

Request

prompt string required The original question or instruction.

---

output string required The LLM response you want to verify.

---

context string or null Optional reference material. When present, reliability is checked only against this context.

---

return_search_results boolean or null Whether to return the web search snippets the judge used to fact-check the answer. Defaults to false.

---

Returns

A Reliability response object indicating if hallucination was detected.

Pythoncurl

Example request

from typing import Any, Dict 

from openai import OpenAI

client = OpenAI(
    base_url="https://api.kluster.ai/v1",
    api_key="INSERT_API_KEY"
)

payload = {
    "prompt": "Is earth flat?",
    "output": "Yes, absolutely.",
    "return_search_results": False
}

resp = client.post(
    "/verify/reliability",
    cast_to=Dict[str, Any], 
    body=payload
)

print(resp)                      

Example request

curl -s https://api.kluster.ai/v1/verify/reliability \
  -H "Authorization: Bearer INSERT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "prompt": "Is earth flat?",
        "output": "Yes, absolutely.",
        "return_search_results": false
      }'

Response

{
  "is_hallucination": true,
  "usage": {
    "completion_tokens": 301,
    "prompt_tokens": 925,
    "total_tokens": 1226
  },
  "explanation": "The response 'Yes, absolutely' is factually incorrect as it contradicts the established scientific fact that the Earth is not flat.\nThe statement is an unsupported claim because it goes against the overwhelming scientific consensus and evidence.\nThe response confidently presents a falsehood, which is a key characteristic of hallucination.",
  "search_results": []
}

---

Reliability response object

is_hallucination boolean

true if the answer contains unreliable or unsupported information.

---

explanation string

Reasoning behind the judgment. Useful for audit logs and model evaluation.

---

usage object

Token-level usage statistics for billing / monitoring.

Show properties

prompt_tokens integer

Tokens consumed by the request.

---

completion_tokens integer

Tokens generated during the verification step.

---

total_tokens integer

Sum of prompt + completion tokens.

---

search_results array

Only present when return_search_results=true. Each element contains:

Field	Type	Description
title	string	Page title
snippet	string	Passage supporting/refuting the claim
link	string	Source URL

Reliability response object

{
  "is_hallucination": false,
  "explanation": "Answer matches the supplied context exactly.",
  "usage": {
    "prompt_tokens": 58,
    "completion_tokens": 142,
    "total_tokens": 200
  },
  "search_results": [
    {
      "title": "NASA – Shape of the Earth",
      "snippet": "...the planet is an oblate spheroid...",
      "link": "https://www.nasa.gov/..."
    }
  ]
}

---