(straight-use-package 'gptel)

Installing the markdown-mode package is optional.

Clone or download this repository and run M-x package-install-file⏎ on the repository directory.

Installing the markdown-mode package is optional.

In packages.el

(package! gptel)

In config.el

(use-package! gptel
 :config
 (setq! gptel-api-key "your key"))

“your key” can be the API key itself, or (safer) a function that returns the key. Setting gptel-api-key is optional, you will be asked for a key if it’s not found.

In your .spacemacs file, add llm-client to dotspacemacs-configuration-layers.

(llm-client :variables
            llm-client-enable-gptel t)

Register a backend with

(gptel-make-azure "Azure-1"             ;Name, whatever you'd like
  :protocol "https"                     ;Optional -- https is the default
  :host "YOUR_RESOURCE_NAME.openai.azure.com"
  :endpoint "/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2023-05-15" ;or equivalent
  :stream t                             ;Enable streaming responses
  :key #'gptel-api-key
  :models '(gpt-3.5-turbo gpt-4))

Refer to the documentation of gptel-make-azure to set more parameters.

You can pick this backend from the menu when using gptel. (see Usage).

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model 'gpt-3.5-turbo
 gptel-backend (gptel-make-azure "Azure-1"
                 :protocol "https"
                 :host "YOUR_RESOURCE_NAME.openai.azure.com"
                 :endpoint "/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2023-05-15"
                 :stream t
                 :key #'gptel-api-key
                 :models '(gpt-3.5-turbo gpt-4)))

Register a backend with

(gptel-make-gpt4all "GPT4All"           ;Name of your choosing
 :protocol "http"
 :host "localhost:4891"                 ;Where it's running
 :models '(mistral-7b-openorca.Q4_0.gguf)) ;Available models

These are the required parameters, refer to the documentation of gptel-make-gpt4all for more.

You can pick this backend from the menu when using gptel (see Usage).

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above. Additionally you may want to increase the response token size since GPT4All uses very short (often truncated) responses by default.

;; OPTIONAL configuration
(setq
 gptel-max-tokens 500
 gptel-model 'mistral-7b-openorca.Q4_0.gguf
 gptel-backend (gptel-make-gpt4all "GPT4All"
                 :protocol "http"
                 :host "localhost:4891"
                 :models '(mistral-7b-openorca.Q4_0.gguf)))

Register a backend with

(gptel-make-ollama "Ollama"             ;Any name of your choosing
  :host "localhost:11434"               ;Where it's running
  :stream t                             ;Stream responses
  :models '(mistral:latest))          ;List of models

These are the required parameters, refer to the documentation of gptel-make-ollama for more.

You can pick this backend from the menu when using gptel (see Usage)

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model 'mistral:latest
 gptel-backend (gptel-make-ollama "Ollama"
                 :host "localhost:11434"
                 :stream t
                 :models '(mistral:latest)))

Register a backend with

;; :key can be a function that returns the API key.
(gptel-make-gemini "Gemini" :key "YOUR_GEMINI_API_KEY" :stream t)

These are the required parameters, refer to the documentation of gptel-make-gemini for more.

You can pick this backend from the menu when using gptel (see Usage)

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model 'gemini-pro
 gptel-backend (gptel-make-gemini "Gemini"
                 :key "YOUR_GEMINI_API_KEY"
                 :stream t))

(If using a llamafile, run a server llamafile instead of a “command-line llamafile”, and a model that supports text generation.)

Register a backend with

;; Llama.cpp offers an OpenAI compatible API
(gptel-make-openai "llama-cpp"          ;Any name
  :stream t                             ;Stream responses
  :protocol "http"
  :host "localhost:8000"                ;Llama.cpp server location
  :models '(test))                    ;Any names, doesn't matter for Llama

These are the required parameters, refer to the documentation of gptel-make-openai for more.

You can pick this backend from the menu when using gptel (see Usage)

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model   'test
 gptel-backend (gptel-make-openai "llama-cpp"
                 :stream t
                 :protocol "http"
                 :host "localhost:8000"
                 :models '(test)))

Kagi’s FastGPT model and the Universal Summarizer are both supported. A couple of notes:

1. Universal Summarizer: If there is a URL at point, the summarizer will summarize the contents of the URL. Otherwise the context sent to the model is the same as always: the buffer text upto point, or the contents of the region if the region is active.
2. Kagi models do not support multi-turn conversations, interactions are “one-shot”. They also do not support streaming responses.

Register a backend with

(gptel-make-kagi "Kagi"                    ;any name
  :key "YOUR_KAGI_API_KEY")                ;can be a function that returns the key

These are the required parameters, refer to the documentation of gptel-make-kagi for more.

You can pick this backend and the model (fastgpt/summarizer) from the transient menu when using gptel.

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model 'fastgpt
 gptel-backend (gptel-make-kagi "Kagi"
                 :key "YOUR_KAGI_API_KEY"))

The alternatives to fastgpt include summarize:cecil, summarize:agnes, summarize:daphne and summarize:muriel. The difference between the summarizer engines is documented here.

Register a backend with

;; Together.ai offers an OpenAI compatible API
(gptel-make-openai "TogetherAI"         ;Any name you want
  :host "api.together.xyz"
  :key "your-api-key"                   ;can be a function that returns the key
  :stream t
  :models '(;; has many more, check together.ai
            mistralai/Mixtral-8x7B-Instruct-v0.1
            codellama/CodeLlama-13b-Instruct-hf
            codellama/CodeLlama-34b-Instruct-hf))

You can pick this backend from the menu when using gptel (see Usage)

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model   'mistralai/Mixtral-8x7B-Instruct-v0.1
 gptel-backend
 (gptel-make-openai "TogetherAI"         
   :host "api.together.xyz"
   :key "your-api-key"                   
   :stream t
   :models '(;; has many more, check together.ai
             mistralai/Mixtral-8x7B-Instruct-v0.1
             codellama/CodeLlama-13b-Instruct-hf
             codellama/CodeLlama-34b-Instruct-hf)))

Register a backend with

;; Anyscale offers an OpenAI compatible API
(gptel-make-openai "Anyscale"           ;Any name you want
  :host "api.endpoints.anyscale.com"
  :key "your-api-key"                   ;can be a function that returns the key
  :models '(;; has many more, check anyscale
            mistralai/Mixtral-8x7B-Instruct-v0.1))

You can pick this backend from the menu when using gptel (see Usage)

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model   'mistralai/Mixtral-8x7B-Instruct-v0.1
 gptel-backend
 (gptel-make-openai "Anyscale"
                 :host "api.endpoints.anyscale.com"
                 :key "your-api-key"
                 :models '(;; has many more, check anyscale
                           mistralai/Mixtral-8x7B-Instruct-v0.1)))

Register a backend with

;; Perplexity offers an OpenAI compatible API
(gptel-make-openai "Perplexity"         ;Any name you want
  :host "api.perplexity.ai"
  :key "your-api-key"                   ;can be a function that returns the key
  :endpoint "/chat/completions"
  :stream t
  :models '(;; has many more, check perplexity.ai
            pplx-7b-chat
            pplx-70b-chat
            pplx-7b-online
            pplx-70b-online))

You can pick this backend from the menu when using gptel (see Usage)

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model   'pplx-7b-chat
 gptel-backend
 (gptel-make-openai "Perplexity"
   :host "api.perplexity.ai"
   :key "your-api-key"
   :endpoint "/chat/completions"
   :stream t
   :models '(;; has many more, check perplexity.ai
             pplx-7b-chat
             pplx-70b-chat
             pplx-7b-online
             pplx-70b-online)))

Register a backend with

(gptel-make-anthropic "Claude"          ;Any name you want
  :stream t                             ;Streaming responses
  :key "your-api-key")

The :key can be a function that returns the key (more secure).

You can pick this backend from the menu when using gptel (see Usage).

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model 'claude-3-sonnet-20240229 ;  "claude-3-opus-20240229" also available
 gptel-backend (gptel-make-anthropic "Claude"
                 :stream t :key "your-api-key"))

Register a backend with

;; Groq offers an OpenAI compatible API
(gptel-make-openai "Groq"               ;Any name you want
  :host "api.groq.com"
  :endpoint "/openai/v1/chat/completions"
  :stream t
  :key "your-api-key"                   ;can be a function that returns the key
  :models '(llama-3.1-70b-versatile
            llama-3.1-8b-instant
            llama3-70b-8192
            llama3-8b-8192
            mixtral-8x7b-32768
            gemma-7b-it))

You can pick this backend from the menu when using gptel (see Usage). Note that Groq is fast enough that you could easily set :stream nil and still get near-instant responses.

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq gptel-model   'mixtral-8x7b-32768
      gptel-backend
      (gptel-make-openai "Groq"
        :host "api.groq.com"
        :endpoint "/openai/v1/chat/completions"
        :stream t
        :key "your-api-key"
        :models '(llama-3.1-70b-versatile
                  llama-3.1-8b-instant
                  llama3-70b-8192
                  llama3-8b-8192
                  mixtral-8x7b-32768
                  gemma-7b-it)))

Register a backend with

;; OpenRouter offers an OpenAI compatible API
(gptel-make-openai "OpenRouter"               ;Any name you want
  :host "openrouter.ai"
  :endpoint "/api/v1/chat/completions"
  :stream t
  :key "your-api-key"                   ;can be a function that returns the key
  :models '(openai/gpt-3.5-turbo
            mistralai/mixtral-8x7b-instruct
            meta-llama/codellama-34b-instruct
            codellama/codellama-70b-instruct
            google/palm-2-codechat-bison-32k
            google/gemini-pro))

You can pick this backend from the menu when using gptel (see Usage).

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq gptel-model   'mixtral-8x7b-32768
      gptel-backend
      (gptel-make-openai "OpenRouter"               ;Any name you want
        :host "openrouter.ai"
        :endpoint "/api/v1/chat/completions"
        :stream t
        :key "your-api-key"                   ;can be a function that returns the key
        :models '(openai/gpt-3.5-turbo
                  mistralai/mixtral-8x7b-instruct
                  meta-llama/codellama-34b-instruct
                  codellama/codellama-70b-instruct
                  google/palm-2-codechat-bison-32k
                  google/gemini-pro)))

Register a backend with

(gptel-make-privategpt "privateGPT"               ;Any name you want
  :protocol "http"
  :host "localhost:8001"
  :stream t
  :context t                            ;Use context provided by embeddings
  :sources t                            ;Return information about source documents
  :models '(private-gpt))

You can pick this backend from the menu when using gptel (see Usage).

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq gptel-model   'private-gpt
      gptel-backend
      (gptel-make-privategpt "privateGPT"               ;Any name you want
        :protocol "http"
        :host "localhost:8001"
        :stream t
        :context t                            ;Use context provided by embeddings
        :sources t                            ;Return information about source documents
        :models '(private-gpt)))

Register a backend with

;; DeepSeek offers an OpenAI compatible API
(gptel-make-openai "DeepSeek"       ;Any name you want
  :host "api.deepseek.com"
  :endpoint "/chat/completions"
  :stream t
  :key "your-api-key"               ;can be a function that returns the key
  :models '(deepseek-chat deepseek-coder))

You can pick this backend from the menu when using gptel (see Usage).

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq gptel-model   'deepseek-chat
      gptel-backend
      (gptel-make-openai "DeepSeek"     ;Any name you want
        :host "api.deepseek.com"
        :endpoint "/chat/completions"
        :stream t
        :key "your-api-key"             ;can be a function that returns the key
        :models '(deepseek-chat deepseek-coder)))

Register a backend with

;; Cerebras offers an instant OpenAI compatible API
(gptel-make-openai "Cerebras"
  :host "api.cerebras.ai"
  :endpoint "/v1/chat/completions"
  :stream t                             ;optionally nil as Cerebras is instant AI
  :key "your-api-key"                   ;can be a function that returns the key
  :models '(llama3.1-70b
            llama3.1-8b))

You can pick this backend from the menu when using gptel (see Usage).

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq gptel-model   'llama3.1-8b
      gptel-backend
      (gptel-make-openai "Cerebras"
        :host "api.cerebras.ai"
        :endpoint "/v1/chat/completions"
        :stream nil
        :key "your-api-key"
        :models '(llama3.1-70b
                  llama3.1-8b)))

Register a backend with

;; Github Models offers an OpenAI compatible API
(gptel-make-openai "Github Models" ;Any name you want
  :host "models.inference.ai.azure.com"
  :endpoint "/chat/completions"
  :stream t
  :key "your-github-token"
  :models '(gpt-4o))

For all the available models, check the marketplace.

You can pick this backend from the menu when using (see Usage).

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq gptel-model  'gpt-4o
      gptel-backend
      (gptel-make-openai "Github Models" ;Any name you want
        :host "models.inference.ai.azure.com"
        :endpoint "/chat/completions"
        :stream t
        :key "your-github-token"
        :models '(gpt-4o))

Register a backend with

;; Novita AI offers an OpenAI compatible API
(gptel-make-openai "NovitaAI"         ;Any name you want
  :host "api.novita.ai"
  :endpoint "/v3/openai"
  :key "your-api-key"                   ;can be a function that returns the key
  :stream t
  :models '(;; has many more, check https://novita.ai/llm-api
            gryphe/mythomax-l2-13b
            meta-llama/llama-3-70b-instruct
            meta-llama/llama-3.1-70b-instruct))

You can pick this backend from the menu when using gptel (see Usage)

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model   'gryphe/mythomax-l2-13b
 gptel-backend
 (gptel-make-openai "NovitaAI"         
   :host "api.novita.ai"
   :endpoint "/v3/openai"
   :key "your-api-key"                   
   :stream t
   :models '(;; has many more, check https://novita.ai/llm-api
             mistralai/Mixtral-8x7B-Instruct-v0.1
             meta-llama/llama-3-70b-instruct
             meta-llama/llama-3.1-70b-instruct)))

Register a backend with

;; xAI offers an OpenAI compatible API
(gptel-make-openai "xAI"           ;Any name you want
  :host "api.x.ai"
  :key "your-api-key"              ;can be a function that returns the key
  :endpoint "/v1/chat/completions"
  :stream t
  :models '(;; xAI now only offers `grok-beta` as of the time of this writing
            grok-beta))

You can pick this backend from the menu when using gptel (see Usage)

The above code makes the backend available to select. If you want it to be the default backend for gptel, you can set this as the value of gptel-backend. Use this instead of the above.

;; OPTIONAL configuration
(setq
 gptel-model   'grok-beta
 gptel-backend
 (gptel-make-openai "xAI"           ;Any name you want
   :host "api.x.ai"
   :key "your-api-key"              ;can be a function that returns the key
   :endpoint "/v1/chat/completions"
   :stream t
   :models '(;; xAI now only offers `grok-beta` as of the time of this writing
             grok-beta)))

gptel supports sending media in Markdown and Org chat buffers, but this feature is disabled by default.

• You can enable it globally, for all models that support it, by setting gptel-track-media.
• Or you can set it locally, just for the chat buffer, via the header line:

There are two ways to include media with requests:

1. Adding media files to the context with gptel-add-file, described further below.
2. Including links to media in chat buffers, described here:

To send media – images or other supported file types – with requests in chat buffers, you can include links to them in the chat buffer. Such a link must be “standalone”, i.e. on a line by itself surrounded by whitespace.

In Org mode, for example, the following are all valid ways of including an image with the request:

• “Standalone” file link:

Describe this picture

[[file:/path/to/screenshot.png]]

Focus specifically on the text content.

• “Standalone” file link with description:

Describe this picture

[[file:/path/to/screenshot.png][some picture]]

Focus specifically on the text content.

• “Standalone”, angle file link:

Describe this picture

<file:/path/to/screenshot.png>

Focus specifically on the text content.

The following links are not valid, and the text of the link will be sent instead of the file contents:

• Inline link:

Describe this [[file:/path/to/screenshot.png][picture]].

Focus specifically on the text content.

• Link not “standalone”:

Describe this picture: 
[[file:/path/to/screenshot.png]]
Focus specifically on the text content.

• Not a valid Org link:

Describe the picture

file:/path/to/screenshot.png

Similar criteria apply to Markdown chat buffers.

Saving the file will save the state of the conversation as well. To resume the chat, open the file and turn on gptel-mode before editing the buffer.

To be minimally annoying, gptel does not move the cursor by default. Add the following to your configuration to enable auto-scrolling.

(add-hook 'gptel-post-stream-hook 'gptel-auto-scroll)

To be minimally annoying, gptel does not move the cursor by default. Add the following to your configuration to move the cursor:

(add-hook 'gptel-post-response-functions 'gptel-end-of-response)

You can also call gptel-end-of-response as a command at any time.

For dedicated chat buffers: customize gptel-prompt-prefix-alist and gptel-response-prefix-alist. You can set a different pair for each major-mode.

Anywhere in Emacs: Use gptel-pre-response-hook and gptel-post-response-functions, which see.

Any model options you set are saved for the current buffer. But the redirection options in the menu are set for the next query only:

You can make them persistent across this Emacs session by pressing C-x C-s:

(You can also cycle through presets you’ve saved with C-x p and C-x n.)

Now these will be enabled whenever you send a query from the transient menu. If you want to use these saved options without invoking the transient menu, you can use a keyboard macro:

;; Replace with your key to invoke the transient menu:
(keymap-global-set "<f6>" "C-u C-c <return> <return>")

Or see this wiki entry.

gptel’s default usage pattern is simple, and will stay this way: Read input in any buffer and insert the response below it. Some custom behavior is possible with the transient menu (C-u M-x gptel-send).

For more programmable usage, gptel provides a general gptel-request function that accepts a custom prompt and a callback to act on the response. You can use this to build custom workflows not supported by gptel-send. See the documentation of gptel-request, and the wiki for examples.

gptel uses text-properties to watermark LLM responses. Thus this text is interpreted as a response even if you copy it into another buffer. In regular buffers (buffers without gptel-mode enabled), you can turn off this tracking by unsetting gptel-track-responses.

When restoring a chat state from a file on disk, gptel will apply these properties from saved metadata in the file when you turn on gptel-mode.

gptel does not use any prefix or semantic/syntax element in the buffer (such as headings) to separate prompts and responses. The reason for this is that gptel aims to integrate as seamlessly as possible into your regular Emacs usage: LLM interaction is not the objective, it’s just another tool at your disposal. So requiring a bunch of “user” and “assistant” tags in the buffer is noisy and restrictive. If you want these demarcations, you can customize gptel-prompt-prefix-alist and gptel-response-prefix-alist. Note that these prefixes are for your readability only and purely cosmetic.

Doom binds RET in Org mode to +org/dwim-at-point, which appears to conflict with gptel’s transient menu bindings for some reason.

Two solutions:

• Press C-m instead of the return key.
• Change the send key from return to a key of your choice:

  (transient-suffix-put 'gptel-menu (kbd "RET") :key "<f8>")
      

(HTTP/2 429) You exceeded your current quota, please check your plan and billing details.

Using the ChatGPT (or any OpenAI) API requires adding credit to your account.

Other Emacs clients for LLMs prescribe the format of the interaction (a comint shell, org-babel blocks, etc). I wanted:

1. Something that is as free-form as possible: query the model using any text in any buffer, and redirect the response as required. Using a dedicated gptel buffer just adds some visual flair to the interaction.
2. Integration with org-mode, not using a walled-off org-babel block, but as regular text. This way the model can generate code blocks that I can run.