Konko

All functionality related to Konko

Konko AI provides a fully managed API to help application developers

1. Select the right open source or proprietary LLMs for their application
2. Build applications faster with integrations to leading application frameworks and fully managed APIs
3. Fine tune smaller open-source LLMs to achieve industry-leading performance at a fraction of the cost
4. Deploy production-scale APIs that meet security, privacy, throughput, and latency SLAs without infrastructure set-up or administration using Konko AI's SOC 2 compliant, multi-cloud infrastructure

Installation and Setup

1. Sign in to our web app to create an API key to access models via our endpoints for chat completions and completions.
2. Enable a Python3.8+ environment
3. Install the SDK

pip install konko

4. Set API Keys as environment variables(KONKO_API_KEY,OPENAI_API_KEY)

export KONKO_API_KEY={your_KONKO_API_KEY_here}
export OPENAI_API_KEY={your_OPENAI_API_KEY_here} #Optional

Please see the Konko docs for more details.

LLM

Explore Available Models: Start by browsing through the available models on Konko. Each model caters to different use cases and capabilities.

Another way to find the list of models running on the Konko instance is through this endpoint.

See a usage example.

Examples of Endpoint Usage

• Completion with mistralai/Mistral-7B-v0.1:

  from langchain_community.llms import Konko
  llm = Konko(max_tokens=800, model='mistralai/Mistral-7B-v0.1')
  prompt = "Generate a Product Description for Apple Iphone 15"
  response = llm.invoke(prompt)

Chat Models

See a usage example.

• ChatCompletion with Mistral-7B:

  from langchain_core.messages import HumanMessage
  from langchain_community.chat_models import ChatKonko
  chat_instance = ChatKonko(max_tokens=10, model = 'mistralai/mistral-7b-instruct-v0.1')
  msg = HumanMessage(content="Hi")
  chat_response = chat_instance([msg])

For further assistance, contact [email protected] or join our Discord.