Text classification with kluster.ai API and Bespoke Curator¶

This notebook goes through the same example as in our previous Text classification notebook, but this time, we'll be using Bespoke Curator instead of the OpenAI Python library

To recap, the notebook uses kluster.ai batch API to classify a data set based on a predefined set of categories.

The example uses an extract from the IMDB top 1000 movies dataset and categorizes them into "Action," "Adventure," "Comedy," "Crime," "Documentary," "Drama," "Fantasy," "Horror," "Romance," or "Sci-Fi."

You can adapt this example by using your data and categories relevant to your use case. With this approach, you can effortlessly process datasets of any scale, big or small, and obtain categorized results powered by a state-of-the-art language model.

Prerequisites¶

Before getting started, ensure you have the following:

• A kluster.ai account - sign up on the kluster.ai platform if you don't have one
• A kluster.ai API key - after signing in, go to the API Keys section and create a new key. For detailed instructions, check out the Get an API key guide

Setup¶

In this notebook, we'll use Python's getpass module to input the key safely. After execution, please provide your unique kluster.ai API key (ensure no spaces).

In [3]:

from getpass import getpass
api_key = getpass("Enter your kluster.ai API key: ")

from getpass import getpass api_key = getpass("Enter your kluster.ai API key: ")

Enter your kluster.ai API key: ··········

Next, ensure you've the Bespoke Curator Python library:

In [1]:

pip install -q bespokelabs-curator

pip install -q bespokelabs-curator

WARNING: google-cloud-aiplatform 1.71.1 does not provide the extra 'all'

Now that we've the library, we can initialize the LLM object for batch. Note that Curator supports kluster.ai natively, so you just need to provide the model to use, API key, and completion window.

This example uses klusterai/Meta-Llama-3.1-8B-Instruct-Turbo, but feel free to comment it and uncomment any other model you want to try out. Please refer to the Supported models section for a list of the models we support.

In [4]:

from bespokelabs import curator

# Models
#model="deepseek-ai/DeepSeek-R1"
#model="deepseek-ai/DeepSeek-V3"
model="klusterai/Meta-Llama-3.1-8B-Instruct-Turbo"
#model="klusterai/Meta-Llama-3.1-405B-Instruct-Turbo"
#model="klusterai/Meta-Llama-3.3-70B-Instruct-Turbo"
#model="Qwen/Qwen2.5-VL-7B-Instruct"

llm = curator.LLM(
    model_name=model,
    batch=True,
    backend="klusterai",
    backend_params={"api_key": api_key, "completion_window": "24h"})

from bespokelabs import curator # Models #model="deepseek-ai/DeepSeek-R1" #model="deepseek-ai/DeepSeek-V3" model="klusterai/Meta-Llama-3.1-8B-Instruct-Turbo" #model="klusterai/Meta-Llama-3.1-405B-Instruct-Turbo" #model="klusterai/Meta-Llama-3.3-70B-Instruct-Turbo" #model="Qwen/Qwen2.5-VL-7B-Instruct" llm = curator.LLM( model_name=model, batch=True, backend="klusterai", backend_params={"api_key": api_key, "completion_window": "24h"})

DEBUG:curator.bespokelabs.curator.log:Adjusting file descriptor limit from 1048576 to 1048576 (hard limit: 1048576)

Get the data¶

With the Curator LLM object ready, let's define the data and prompt.

This notebook includes a preloaded sample dataset derived from the Top 1000 IMDb Movies dataset. It contains movie descriptions ready for classification. No additional setup is needed. Proceed to the next steps to begin working with this data.

For this particular scenario, the prompt consists of the request to the model and the data (movie) to be classified. Because this is a batch job, each separate request must contain both.

In [5]:

movies = ["Breakfast at Tiffany's: A young New York socialite becomes interested in a young man who has moved into her apartment building, but her past threatens to get in the way.",
        "Giant: Sprawling epic covering the life of a Texas cattle rancher and his family and associates.",
        "From Here to Eternity: In Hawaii in 1941, a private is cruelly punished for not boxing on his unit's team, while his captain's wife and second-in-command are falling in love.",
        "Lifeboat: Several survivors of a torpedoed merchant ship in World War II find themselves in the same lifeboat with one of the crew members of the U-boat that sank their ship.",
        "The 39 Steps: A man in London tries to help a counter-espionage Agent. But when the Agent is killed, and the man stands accused, he must go on the run to save himself and stop a spy ring which is trying to steal top secret information."]

movies = ["Breakfast at Tiffany's: A young New York socialite becomes interested in a young man who has moved into her apartment building, but her past threatens to get in the way.", "Giant: Sprawling epic covering the life of a Texas cattle rancher and his family and associates.", "From Here to Eternity: In Hawaii in 1941, a private is cruelly punished for not boxing on his unit's team, while his captain's wife and second-in-command are falling in love.", "Lifeboat: Several survivors of a torpedoed merchant ship in World War II find themselves in the same lifeboat with one of the crew members of the U-boat that sank their ship.", "The 39 Steps: A man in London tries to help a counter-espionage Agent. But when the Agent is killed, and the man stands accused, he must go on the run to save himself and stop a spy ring which is trying to steal top secret information."]

In [6]:

prompts = [f"Classify the main genre of the given movie description based on the following genres(Respond with only the genre): “Action”, “Adventure”, “Comedy”, “Crime”, “Documentary”, “Drama”, “Fantasy”, “Horror”, “Romance”, “Sci-Fi”.\n{movie}" for movie in movies]

# Log the prompt
for prompt in prompts:
    print(prompt)

prompts = [f"Classify the main genre of the given movie description based on the following genres(Respond with only the genre): “Action”, “Adventure”, “Comedy”, “Crime”, “Documentary”, “Drama”, “Fantasy”, “Horror”, “Romance”, “Sci-Fi”.\n{movie}" for movie in movies] # Log the prompt for prompt in prompts: print(prompt)

Classify the main genre of the given movie description based on the following genres(Respond with only the genre): “Action”, “Adventure”, “Comedy”, “Crime”, “Documentary”, “Drama”, “Fantasy”, “Horror”, “Romance”, “Sci-Fi”.
Breakfast at Tiffany's: A young New York socialite becomes interested in a young man who has moved into her apartment building, but her past threatens to get in the way.
Classify the main genre of the given movie description based on the following genres(Respond with only the genre): “Action”, “Adventure”, “Comedy”, “Crime”, “Documentary”, “Drama”, “Fantasy”, “Horror”, “Romance”, “Sci-Fi”.
Giant: Sprawling epic covering the life of a Texas cattle rancher and his family and associates.
Classify the main genre of the given movie description based on the following genres(Respond with only the genre): “Action”, “Adventure”, “Comedy”, “Crime”, “Documentary”, “Drama”, “Fantasy”, “Horror”, “Romance”, “Sci-Fi”.
From Here to Eternity: In Hawaii in 1941, a private is cruelly punished for not boxing on his unit's team, while his captain's wife and second-in-command are falling in love.
Classify the main genre of the given movie description based on the following genres(Respond with only the genre): “Action”, “Adventure”, “Comedy”, “Crime”, “Documentary”, “Drama”, “Fantasy”, “Horror”, “Romance”, “Sci-Fi”.
Lifeboat: Several survivors of a torpedoed merchant ship in World War II find themselves in the same lifeboat with one of the crew members of the U-boat that sank their ship.
Classify the main genre of the given movie description based on the following genres(Respond with only the genre): “Action”, “Adventure”, “Comedy”, “Crime”, “Documentary”, “Drama”, “Fantasy”, “Horror”, “Romance”, “Sci-Fi”.
The 39 Steps: A man in London tries to help a counter-espionage Agent. But when the Agent is killed, and the man stands accused, he must go on the run to save himself and stop a spy ring which is trying to steal top secret information.

Perform batch inference with Curator¶

Now that everything is set, we can execute the inference job. With Curator it is extremely simple, we just need to pass the prompts to the LLM object, and log the response.

In [7]:

responses = llm(prompts)

responses = llm(prompts)

Generating train split: 0 examples [00:00, ? examples/s]

DEBUG:curator.bespokelabs.curator.log:Curator Cache Fingerprint String: 36e766fc298f4350_5ac272e3bc92b32e_klusterai/Meta-Llama-3.1-8B-Instruct-Turbo_text_True
DEBUG:curator.bespokelabs.curator.log:Curator Cache Fingerprint: 3acb0d5efbda6beb

[03/24/25 09:51:35] INFO     Running OpenAIBatchRequestProcessor completions with     base_request_processor.py:130
                             model: klusterai/Meta-Llama-3.1-8B-Instruct-Turbo                                     

INFO:curator.bespokelabs.curator.log:Running OpenAIBatchRequestProcessor completions with model: klusterai/Meta-Llama-3.1-8B-Instruct-Turbo

                    INFO     Preparing request file(s) in                             base_request_processor.py:230
                             /root/.cache/curator/3acb0d5efbda6beb                                                 

INFO:curator.bespokelabs.curator.log:Preparing request file(s) in /root/.cache/curator/3acb0d5efbda6beb

                    INFO     Wrote 5 requests to                                      base_request_processor.py:312
                             /root/.cache/curator/3acb0d5efbda6beb/requests_0.jsonl.                               

INFO:curator.bespokelabs.curator.log:Wrote 5 requests to /root/.cache/curator/3acb0d5efbda6beb/requests_0.jsonl.
DEBUG:curator.bespokelabs.curator.log:Batch file content size: 0.00 MB (3,367 bytes)

Output()

DEBUG:curator.bespokelabs.curator.log:skipping uploaded file status check, provider does not support file checks.
DEBUG:curator.bespokelabs.curator.log:File uploaded with id 67e12b272d9e5ba243fe9ba1
DEBUG:curator.bespokelabs.curator.log:Batch submitted with id 67e12b286afe1d706e726f73
DEBUG:curator.bespokelabs.curator.log:Marked /root/.cache/curator/3acb0d5efbda6beb/requests_0.jsonl as submitted with batch 67e12b286afe1d706e726f73
DEBUG:curator.bespokelabs.curator.log:Updated submitted batch 67e12b286afe1d706e726f73 with new request counts
DEBUG:curator.bespokelabs.curator.log:Batch 67e12b286afe1d706e726f73 status: in_progress requests: 0/0/5 succeeded/failed/total
DEBUG:curator.bespokelabs.curator.log:Batches returned: 0/1 Requests completed: 0/5
DEBUG:curator.bespokelabs.curator.log:Sleeping for 60 seconds...
DEBUG:curator.bespokelabs.curator.log:Updated submitted batch 67e12b286afe1d706e726f73 with new request counts
DEBUG:curator.bespokelabs.curator.log:Batch 67e12b286afe1d706e726f73 status: completed requests: 5/0/5 succeeded/failed/total
DEBUG:curator.bespokelabs.curator.log:Batch 67e12b286afe1d706e726f73 finished with status: completed
DEBUG:curator.bespokelabs.curator.log:Marked batch 67e12b286afe1d706e726f73 as finished
DEBUG:curator.bespokelabs.curator.log:Batch 67e12b286afe1d706e726f73 completed and downloaded

/usr/local/lib/python3.11/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: 
The secret `HF_TOKEN` does not exist in your Colab secrets.
To authenticate with the Hugging Face Hub, create a token in your settings tab 
(https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.
You will be able to reuse this secret in all of your notebooks.
Please note that authentication is recommended but still optional to access public models or datasets.
  warnings.warn(

tokenizer.json:   0%|          | 0.00/9.08M [00:00<?, ?B/s]

DEBUG:curator.bespokelabs.curator.log:Batch 67e12b286afe1d706e726f73 written to /root/.cache/curator/3acb0d5efbda6beb/responses_0.jsonl
DEBUG:curator.bespokelabs.curator.log:Marked batch 67e12b286afe1d706e726f73 as downloaded
DEBUG:curator.bespokelabs.curator.log:Batches returned: 1/1 Requests completed: 5/5

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% • Time Elapsed 0:01:04 • Time Remaining 0:00:00

Curator Viewer: Disabled                                                            
Set HOSTED_CURATOR_VIEWER=1 to view your data live at https://curator.bespokelabs.ai
Batches: Total: 1 • Submitted: 0⋯ • Downloaded: 1✓                                  
Requests: Total: 5 • Submitted: 0⋯ • Succeeded: 5✓ • Failed: 0✗                     
Tokens: Avg Input: 133 • Avg Output: 7                                              
Cost: Current: $0.000 • Projected: $0.000 • Rate: $0.000/request                    
Model: Name: klusterai/Meta-Llama-3.1-8B-Instruct-Turbo                             
Model Pricing: Per 1M tokens: Input: $0.050 • Output: $0.050                        

                         Final Curator Statistics                          
╭────────────────────────────┬────────────────────────────────────────────╮
│ Section/Metric             │ Value                                      │
├────────────────────────────┼────────────────────────────────────────────┤
│ Model                      │                                            │
│ Model                      │ klusterai/Meta-Llama-3.1-8B-Instruct-Turbo │
│ Batches                    │                                            │
│ Total Batches              │ 1                                          │
│ Submitted                  │ 0                                          │
│ Downloaded                 │ 1                                          │
│ Requests                   │                                            │
│ Total Requests             │ 5                                          │
│ Successful                 │ 5                                          │
│ Failed                     │ 0                                          │
│ Tokens                     │                                            │
│ Total Tokens Used          │ 0                                          │
│ Total Input Tokens         │ 664                                        │
│ Total Output Tokens        │ 35                                         │
│ Average Tokens per Request │ 0                                          │
│ Average Input Tokens       │ 132                                        │
│ Average Output Tokens      │ 7                                          │
│ Costs                      │                                            │
│ Total Cost                 │ $0.000                                     │
│ Average Cost per Request   │ $0.000                                     │
│ Input Cost per 1M Tokens   │ $0.050                                     │
│ Output Cost per 1M Tokens  │ $0.050                                     │
│ Performance                │                                            │
│ Total Time                 │ 64.37s                                     │
│ Average Time per Request   │ 12.87s                                     │
│ Requests per Minute        │ 4.7                                        │
│ Input Tokens per Minute    │ 618.9                                      │
│ Output Tokens per Minute   │ 32.6                                       │
╰────────────────────────────┴────────────────────────────────────────────╯

[03/24/25 09:52:39] INFO     Read 5 responses.                                        base_request_processor.py:437

INFO:curator.bespokelabs.curator.log:Read 5 responses.

                    INFO     Finalizing writer                                        base_request_processor.py:446

INFO:curator.bespokelabs.curator.log:Finalizing writer

Lastly, let's print the response.

In [8]:

responses['response']

responses['response']

Out[8]:

['Romance',
 'Drama',
 'Drama',
 'Drama',
 'Thriller is a possible genre but choosing an option from the above categories, it would be "Drama"']

Summary¶

This tutorial used the chat completion endpoint and Bespoke Curator to perform a simple text classification task with batch inference. This particular example clasified a series of movies based on their description.

Using Curator, submitting a batch job is extremely simple. It handles all the steps of creating the file, uploading it, submitting the batch job, monitoring the job, and retrieving results. Moreover, kluster.ai is natively supported, making things even easier!

Kluster.ai's batch API empowers you to scale your workflows seamlessly, making it an invaluable tool for processing extensive datasets. As next steps, feel free to create your own dataset, or expand on top of this existing example. Good luck!