Skip to content

Integrate SillyTavern with kluster.ai＃

SillyTavern is a locally installed customizable LLM user interface that focuses on persona-driven LLM interactions—letting you create unique characters or group chats for tasks like code reviews and text editing. It provides custom prompt fields, bookmarks for revisiting specific points in a conversation, and a mobile-friendly design to manage your chat sessions easily.

By integrating SillyTavern with the kluster.ai API, you can tap into kluster.ai's high-performance language models as your primary or backup backend. This combination merges SillyTavern's customizable UI and advanced prompt options with kluster.ai's reliable inference, offering a scalable and tailored chat environment for casual users and AI enthusiasts.

Prerequisites＃

Before starting, ensure you have the following:

  • A kluster.ai account - sign up on the kluster.ai platform if you don't have one
  • A kluster.ai API key - after signing in, go to the API Keys section and create a new key. For detailed instructions, check out the Get an API key guide

Configure SillyTavern＃

  1. Launch SillyTavern and open it in your browser at http://127.0.0.1:8000/ (default port)
  2. Click on the API Connections icon (plug) in the top navigation menu
  3. In the API drop-down menu, select Chat Completion
  4. In the Chat Completion Source option, choose Custom (OpenAI-compatible)

  5. Enter the kluster.ai API endpoint in the Custom Endpoint (Base URL) field:

    https://api.kluster.ai/v1

    There should be no trailing slash (/) at the end of the URL

  6. Paste your kluster.ai API Key into the designated field

  7. Enter a Model ID. For this example, you can enter:

    klusterai/Meta-Llama-3.3-70B-Instruct-Turbo

  8. Click the Connect button. If you've configured the API correctly, you should see a 🟢 Valid message next to the button

  9. Select one of the kluster.ai-supported models from the Available Models drop-down menu

That's it! You're now ready to start chatting with your bot powered by kluster.ai.

Test the connection＃

Now that you've configured kluster.ai with SillyTavern, you can test the API connection by starting a new conversation.

Follow these steps to get started:

  1. Click the menu icon on the bottom-left corner of the page
  2. Select Start New Chat to open a new chat with the model
  3. Type a message in the Type a message bar at the bottom and send it
  4. Verify that the chatbot has returned a response successfully

Troubleshooting

If you encounter errors, revisit the configuration instructions and double-check your API key and base URL and that you've received a Valid response after connecting the API (see step 8).