Skip to content

Using CrewAI with the kluster.ai API＃

CrewAI is a multi-agent platform that organizes specialized AI agents—each with defined roles, tools, and goals—within a structured process to tackle complex tasks efficiently. CrewAI agents streamline workflows and deliver reliable, scalable solutions by coordinating tasks and ensuring smooth collaboration.

This guide walks you through integrating kluster.ai with CrewAI, from installation to creating and running a simple AI agent chatbot that leverages the kluster.ai API.

Prerequisites＃

Before starting, ensure you have the following prerequisites:

  • A kluster.ai account - sign up on the kluster.ai platform if you don't have one
  • A kluster.ai API key - after signing in, go to the API Keys section and create a new key. For detailed instructions, check out the Get an API key guide
  • CrewAI installed - the Installation Guide on the CrewAI website will walk you through installing CrewAI, setting up a virtual Python environment, and creating a new project. Note that CrewAI requires a Python version >=3.10 and <3.13

Create a project with the CrewAI CLI＃

Open your Python virtual environment, and then follow these steps to use the CrewAI CLI to create a new project:

  1. Create a project - following the installation guide, create your first project with the following command: 
    crewai create crew INSERT_PROJECT_NAME
  2. Select model and provider - during setup, the CLI will ask you to choose a provider and a model. Select openai as the provider and then choose any available model. Because you'll configure kluster.ai as a custom model, your initial model choice won't affect the final integration. The CLI will prompt you for an OpenAI API key, but this isn’t required. Simply press enter to skip

Build a simple AI agent＃

After finishing the CLI setup, you will see a src directory with files crew.py and main.py. This guide won't use these sample files because they include extra features outside the scope. Follow these steps to continue:

  1. Create your first file - Create a hello_crew.py file in src/YOUR_PROJECT_NAME to correspond to a simple AI agent chatbot

  2. Import modules and select model - open hello_crew.py to add imports and define a custom LLM for kluster.ai by setting the following parameters:

    • provider - you can specify openai_compatible
    • model - choose one of kluster.ai's available models based on your use case. Regardless of which model you choose, prepend its name with openai/ to ensure CrewAI, which relies on LiteLLM, processes your requests correctly. For more details, see kluster.ai's models
    • base_url - use https://api.kluster.ai/v1 to send requests to the kluster.ai endpoint
    • api_key - replace INSERT_API_KEY in the code below with your kluster.ai API key. If you don't have one yet, refer to the Get an API key guide
    hello_crew.py
    import random

from crewai import LLM, Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task


@CrewBase
class HelloWorldCrew:
    # Override any default YAML references
    agents_config = {}
    tasks_config = {}

    def __init__(self):
        """
        When this crew is instantiated, create a custom LLM with your base_url.
        """
        self.custom_llm = LLM(
            provider="openai_compatible", 
            model="openai/klusterai/Meta-Llama-3.3-70B-Instruct-Turbo",
            base_url="https://api.kluster.ai/v1",
            api_key="INSERT_KLUSTER_API_KEY"
        )

    This example overrides agents_config and tasks_config with empty dictionaries to tell CrewAI to ignore all YAML files and rely solely on your code, keeping this guide as streamlined as possible.

  3. Define your agent - set the agent's role, goal, and backstory, and assign the custom LLM (via the kluster.ai API) for generating creative greetings:

    hello_crew.py
        @agent
    def hello_agent(self) -> Agent:
        """
        A super simple agent with a single purpose: greet the user in a friendly, varied way.
        """
        return Agent(
            role="HelloWorldAgent",
            goal="Greet the user in a fun and creative way.",
            backstory="I'm a friendly agent who greets everyone in a slightly different manner!",
            llm=self.custom_llm,
            verbose=True
        )

  4. Give the agent a task - define a task that prompts the agent for a unique, creative greeting using randomness to avoid repetition. Passing this prompt to hello_agent() ensures varied responses. CrewAI requires an expected_output field, defined here as a short greeting:

    hello_crew.py
        @task
    def hello_task(self) -> Task:
        """
        A task that asks the agent to produce a dynamic greeting.
        """
        random_factor = random.randint(100000, 999999)
        prompt = f"""
        You are a friendly greeting bot. 
        Please produce a short, creative greeting that changes each time. 
        Random factor: {random_factor}
        Example: "Hey there, how's your day going?"
        """

        return Task(
            description=prompt,
            expected_output="A short, creative greeting",
            agent=self.hello_agent()
        )

  5. Tie it all together with a @crew method - Add the following method to return the assembled Crew object with a single agent and task. This method enables CrewAI to coordinate the agent and task you defined:

    hello_crew.py
        @crew
    def hello_crew(self) -> Crew:
        """
        Our entire 'Hello World' crew—only 1 agent + 1 task in sequence.
        """
        return Crew(
            agents=self.agents,  
            tasks=self.tasks,    
            process=Process.sequential,
            verbose=True
        )

  6. Set up the entry point for the agent - Create a new file named hello_main.py. In hello_main.py, import and initialize the HelloWorldCrew class, call its hello_crew() method, and then kickoff() to launch the task sequence:

    hello_main.py
    #!/usr/bin/env python
from hello_crew import HelloWorldCrew


def run():
    """
    Kick off the HelloWorld crew with no inputs.
    """
    HelloWorldCrew().hello_crew().kickoff(inputs={})

if __name__ == "__main__":
    run()
Complete script
hello_crew.py
import random

from crewai import LLM, Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task


@CrewBase
class HelloWorldCrew:
    # Override any default YAML references
    agents_config = {}
    tasks_config = {}

    def __init__(self):
        """
        When this crew is instantiated, create a custom LLM with your base_url.
        """
        self.custom_llm = LLM(
            provider="openai_compatible", 
            model="openai/klusterai/Meta-Llama-3.3-70B-Instruct-Turbo",
            base_url="https://api.kluster.ai/v1",
            api_key="INSERT_KLUSTER_API_KEY"
        )

    @agent
    def hello_agent(self) -> Agent:
        """
        A super simple agent with a single purpose: greet the user in a friendly, varied way.
        """
        return Agent(
            role="HelloWorldAgent",
            goal="Greet the user in a fun and creative way.",
            backstory="I'm a friendly agent who greets everyone in a slightly different manner!",
            llm=self.custom_llm,
            verbose=True
        )

    @task
    def hello_task(self) -> Task:
        """
        A task that asks the agent to produce a dynamic greeting.
        """
        random_factor = random.randint(100000, 999999)
        prompt = f"""
        You are a friendly greeting bot. 
        Please produce a short, creative greeting that changes each time. 
        Random factor: {random_factor}
        Example: "Hey there, how's your day going?"
        """

        return Task(
            description=prompt,
            expected_output="A short, creative greeting",
            agent=self.hello_agent()
        )

    @crew
    def hello_crew(self) -> Crew:
        """
        Our entire 'Hello World' crew—only 1 agent + 1 task in sequence.
        """
        return Crew(
            agents=self.agents,  
            tasks=self.tasks,    
            process=Process.sequential,
            verbose=True
        )

Run the agent＃

To run your agent, ensure you are in the same directory as your hello_main.py file, then use the following command:

python hello_main.py

Upon running the script, you'll see output that looks like the following:

# Agent: HelloWorldAgent ## Task: You are a friendly greeting bot. Please produce a short, creative greeting that changes each time. Random factor: 896380 Example: "Hey there, how's your day going?"
# Agent: HelloWorldAgent ## Final Answer: Hello, it's a beautiful day to shine, how's your sparkle today?

And that's it! You've now successfully configured your AI agent harnessing CrewAI and the power of the kluster.ai API!