notebooks/3_Chatbot_with_LLM.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a simple chatbot with open-source LLMs using Python and Hugging Face\n",
    "\n",
    "In this notebook, we will create a very simple but functional chatbot!\n",
    "\n",
    "<img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMSkillsNetwork-GPXX04ESEN/images/DALL%C2%B7E%202023-06-06%2009.38.20%20-%20a%20robot%20driving%20a%20convertible%20sports%20car%20towards%20the%20sunset%2C%20digital%20art.png\" width=\"400\" alt=\"robot driving car\">\n",
    "\n",
    "## Learning outcomes:\n",
    "\n",
    "By the end of this lab, you will understand:\n",
    "\n",
    "- the main components of a chatbot\n",
    "- what an LLM is\n",
    "- how to choose an LLM for your application\n",
    "- how a transformer essentially works\n",
    "- how to feed input into a transformer (tokenization)\n",
    "- how to program your own simple chatbot in Python\n",
    "\n",
    "## Introduction: Under the hood of a chatbot\n",
    "\n",
    "\n",
    "### Intro: How does a chatbot work?\n",
    "\n",
    "A chatbot is a computer program that takes a text input, and returns a corresponding text output.\n",
    "\n",
    "Chatbots use a special kind of computer program called a transformer, which is like its brain. Inside this brain, there is something called a language model (LLM), which helps the chatbot understand and generate human-like responses. It looks at lots of examples of human conversations it has seen before to help it respond in a way that makes sense.\n",
    "\n",
    "Transformers and LLMs work together within a chatbot to enable conversation. Here's a simplified explanation of how they interact:\n",
    "\n",
    "    Input Processing: When you send a message to the chatbot, the transformer helps process your input. It breaks down your message into smaller parts and represents them in a way that the chatbot can understand. Each part is called a token.\n",
    "\n",
    "    Understanding Context: The transformer passes these tokens to the LLM, which is a language model trained on lots of text data. The LLM has learned patterns and meanings from this data, so it tries to understand the context of your message based on what it has learned.\n",
    "\n",
    "    Generating Response: Once the LLM understands your message, it generates a response based on its understanding. The transformer then takes this response and converts it into a format that can be easily sent back to you.\n",
    "\n",
    "    Iterative Conversation: As the conversation continues, this process repeats. The transformer and LLM work together to process each new input message, understand the context, and generate a relevant response.\n",
    "\n",
    "The key is that the LLM learns from a large amount of text data to understand language patterns and generate meaningful responses. The transformer helps with the technical aspects of processing and representing the input/output data, allowing the LLM to focus on understanding and generating language\n",
    "\n",
    "Once the chatbot understands your message, it uses the language model to generate a response that it thinks will be helpful or interesting to you. The response is sent back to you, and the process continues as you have a back-and-forth conversation with the chatbot.\n",
    "\n",
    "### Intro: Hugging Face\n",
    "\n",
    "Hugging Face is an organization that focuses on natural language processing (NLP) and AI. They provide a variety of tools, resources, and services to support NLP tasks.\n",
    "\n",
    "We'll be making use of their Python library `transformers`, as you'll see soon.\n",
    "\n",
    "Alright! Now that we know how a chatbot works at a high-level, let's get started with implementing a simple chatbot!\n",
    "\n",
    "## Step 1: Installing Requirements\n",
    "\n",
    "For this example, we will be using the `transformers` library, which is an open-source natural language processing (NLP) toolkit with many useful features.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pip install transformers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Import our required tools from the transformers library\n",
    "\n",
    "For this example, we will be using `AutoTokenizer` and `AutoModelForSeq2SeqLM` from the `transformers` library. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/adebayoakinlalu/.pyenv/versions/3.10.12/envs/llm-env/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer, AutoModelForSeq2SeqLM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Choosing a model\n",
    "\n",
    "Choosing the right model for your purposes is an important part of building chatbots! You can read on the different types of models available on the Hugging Face website: https://huggingface.co/models.\n",
    "\n",
    "LLMs differ from each other in how they are trained. Let's gloss over some examples to see how different models fit better in various contexts.\n",
    "\n",
    "- **Text Generation**:\n",
    "    If you need a general-purpose text generation model, consider using the GPT-2 or GPT-3 models. They are known for their impressive language generation capabilities.\n",
    "    Example: You want to build a chatbot that generates creative and coherent responses to user input.\n",
    "\n",
    "- **Sentiment Analysis**:\n",
    "    For sentiment analysis tasks, models like BERT or RoBERTa are popular choices. They are trained to understand the sentiment and emotional tone of text.\n",
    "    Example: You want to analyze customer feedback and determine whether it is positive or negative.\n",
    "\n",
    "- **Named Entity Recognition**:\n",
    "    LLMs such as BERT, GPT-2, or RoBERTa can be used for Named Entity Recognition (NER) tasks. They perform well in understanding and extracting entities like person names, locations, organizations, etc.\n",
    "    Example: You want to build a system that extracts names of people and places from a given text.\n",
    "\n",
    "- **Question Answering**:\n",
    "    Models like BERT, GPT-2, or XLNet can be effective for question answering tasks. They can comprehend questions and provide accurate answers based on the given context.\n",
    "    Example: You want to build a chatbot that can answer factual questions from a given set of documents.\n",
    "\n",
    "- **Language Translation**:\n",
    "    For language translation tasks, you can consider models like MarianMT or T5. They are designed specifically for translating text between different languages.\n",
    "    Example: You want to build a language translation tool that translates English text to French.\n",
    "\n",
    "However, these examples are very limited and the fit of an LLM may depend on many factors such as data availability, performance requirements, resource constraints, and domain-specific considerations. It's important to explore different LLMs thoroughly and experiment with them to find the best match for your specific application.\n",
    "\n",
    "Other important purposes that should be taken into consideration when choosing an LLM include (but are not limited to):\n",
    "- Licensing: Ensure you are allowed to use your chosen model the way you intend\n",
    "- Model size: Larger models may be more accurate, but might also come at the cost of greater resource requirements\n",
    "- Training data: Ensure that the model's training data aligns with the domain or context you intend to use the LLM for\n",
    "- Performance and accuracy: Consider factors like accuracy, runtime, or any other metrics that are important for your specific use case\n",
    "\n",
    "To explore all the different options, check out the available [models on the Hugging Face website](https://huggingface.co/models?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkGuidedProjectsIBMSkillsNetworkGPXX04ESEN3232-2023-01-01).\n",
    "\n",
    "For this example, we'll be using \"facebook/blenderbot-400M-distill\" because it has an open-source license and runs relatively fast.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = \"facebook/blenderbot-400M-distill\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Fetch the model and initialize a tokenizer\n",
    "\n",
    "When running this code for the first time, the host machine will download the model from Hugging Face API.\n",
    "However, after running the code once, the script will not re-download the model and will instead reference the local installation.\n",
    "\n",
    "We'll be looking at two terms here: `model` and `tokenizer`.\n",
    "\n",
    "In this script, we initiate variables using two handy classes from the `transformers` library:\n",
    "- `model` is an instance of the class `AutoModelForSeq2SeqLM`, which allows us to interact with our chosen language model.\n",
    "- `tokenizer` is an instance of the class `AutoTokenizer`, which optimizes our input and passes it to the language model efficiently. It does so by converting our text input to \"tokens\", which is how the model interprets the text.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Downloading (&hellip;)lve/main/config.json: 100%|&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;| 1.57k/1.57k [00:00<00:00, 4.11MB/s]\n",
      "Downloading pytorch_model.bin: 100%|&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;| 730M/730M [02:02<00:00, 5.98MB/s] \n",
      "Downloading (&hellip;)neration_config.json: 100%|&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;| 347/347 [00:00<00:00, 842kB/s]\n",
      "Downloading (&hellip;)okenizer_config.json: 100%|&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;| 1.15k/1.15k [00:00<00:00, 3.29MB/s]\n",
      "Downloading (&hellip;)olve/main/vocab.json: 100%|&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;| 127k/127k [00:00<00:00, 313kB/s]\n",
      "Downloading (&hellip;)olve/main/merges.txt: 100%|&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;| 62.9k/62.9k [00:00<00:00, 234kB/s]\n",
      "Downloading (&hellip;)in/added_tokens.json: 100%|&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;| 16.0/16.0 [00:00<00:00, 5.75kB/s]\n",
      "Downloading (&hellip;)cial_tokens_map.json: 100%|&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;&#9608;| 772/772 [00:00<00:00, 1.30MB/s]\n"
     ]
    }
   ],
   "source": [
    "# Load model (download on first run and reference local installation for consequent runs)\n",
    "model = AutoModelForSeq2SeqLM.from_pretrained(model_name)\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: Chat\n",
    "\n",
    "Now that we're all set up, let's start chatting!\n",
    "\n",
    "There are several things we'll do to have an effective conversation with our chatbot.\n",
    "\n",
    "Before interacting with our model, we need to initialize an object where we can store our conversation histiry.\n",
    "1. Initialize object to store conversation history\n",
    "\n",
    "Afterwards, we'll do the following for each interaction with the model:\n",
    "2. Encode conversation history as a string\n",
    "3. Fetch prompt from user\n",
    "4. Tokenize (optimize) prompt\n",
    "5. Generate output from model using prompt and history\n",
    "6. Decode output\n",
    "7. Update conversation history\n",
    "\n",
    "### Step 5.1: Keeping track of conversation history\n",
    "\n",
    "The conversation history is important when interacting with a chatbot because the chatbot will also reference the previous conversations when generating output.\n",
    "\n",
    "For our simple implementation in Python, we may simply use a list. Per the Hugging Face implementation, we will use this list to store the conversation history as follows:\n",
    "\n",
    "```\n",
    "conversation_history\n",
    "\n",
    ">> [input_1, output_1, input_2, output_2, ...]\n",
    "```\n",
    "\n",
    "Let's initialize this list before any conversations occur.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "conversation_history = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5.2: Encoding the conversation history\n",
    "\n",
    "During each interaction, we will pass our conversation history to the model along with our input so that it may also reference the previous conversation when generating the next answer.\n",
    "\n",
    "The `transformers` library function we are using expects to receive the conversation history as a string, with each element separated by the newline character `'\\n'`. Thus, we create such a string.\n",
    "\n",
    "We'll use the `join()` method in Python to do exactly that. (Initially, our history_string will be an empty string, which is okay, and will grow as the conversation goes on)&szlig;\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "''"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history_string = \"\\n\".join(conversation_history)\n",
    "history_string "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5.3: Fetch prompt from user\n",
    "\n",
    "Befor we start building a simple terminal chatbot, let's example, the input will be\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hello'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_text =\"hello\"\n",
    "input_text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 5.4: Tokenization of User Prompt and Chat History \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tokens in NLP are individual units or elements that text or sentences are divided into. Tokenization or vectorization is the process of converting tokens into numerical representations. In NLP tasks, we often use the `encode_plus` method from the `tokenizer` object to perform tokenization and vectorization. Let's encode our inputs (prompt & chat history) as tokens so that we may pass them to the model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': tensor([[1710,   86]]), 'attention_mask': tensor([[1, 1]])}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inputs = tokenizer.encode_plus(history_string, input_text, return_tensors=\"pt\")\n",
    "inputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In doing so, we've now created a Python `dictionary` which contains special keywords that allow the model to properly reference its contents.\n",
    "To learn more about tokens and their associated pretrained vocabulary files, you can explore the pretrained_vocab_files_map attribute. This attribute provides a mapping of pretrained models to their corresponding vocabulary files.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'vocab_file': {'facebook/blenderbot-3B': 'https://huggingface.co/facebook/blenderbot-3B/resolve/main/vocab.json'},\n",
       " 'merges_file': {'facebook/blenderbot-3B': 'https://huggingface.co/facebook/blenderbot-3B/resolve/main/merges.txt'},\n",
       " 'tokenizer_config_file': {'facebook/blenderbot-3B': 'https://huggingface.co/facebook/blenderbot-3B/resolve/main/tokenizer_config.json'}}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.pretrained_vocab_files_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "### Step 5.5: Generate output from model\n",
    "\n",
    "Now that we have our inputs ready, both past and present inputs, we can pass them to the model and generate a response. According to the documentation, we can use the `generate()` function and pass the inputs as keyword arguments ([kwargs](https://www.freecodecamp.org/news/args-and-kwargs-in-python/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkGuidedProjectsIBMSkillsNetworkGPXX04ESEN3232-2023-01-01)).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[   1, 1710, 1710, 1710, 6950, 6950, 6950, 1710, 1710, 7336, 7336, 7336,\n",
       "         4424, 7336, 7336, 1710, 1710, 4909, 1710, 1710, 3156, 1710, 1710,    2]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "outputs = model.generate(**inputs)\n",
    "outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great - now we have our outputs! However, the current output `outputs` is also a dictionary and contains tokens, not words in plaintext.\n",
    "Therefore, we just need to decode the first index of `outputs` to see the response in plaintext.\n",
    "\n",
    "### Step 5.6: Decode output\n",
    "\n",
    "We may decode the output using `tokenizer.decode()`. This is know as \"detokenization\" or \"reconstruction\". It is the process of combining or merging individual tokens back into their original form, typically to reconstruct the original text or sentence\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hell hell hell Hello Hello Hello hell hell hi hi hi Hi hi hi hell hell hey hell hell Hell hell hell'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = tokenizer.decode(outputs[0], skip_special_tokens=True).strip()\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alright! We've successfully had an interaction with our chatbot! We've given it a prompt, and we received its response.\n",
    "\n",
    "Now, all that's left to do is to update our conversation history, so that we may pass it with the next iteration.\n",
    "\n",
    "### Step 5.7: Update Conversation History\n",
    "\n",
    "All we need to do here is add both the input and response to `conversation_history` in plaintext.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['hello',\n",
       " 'hell hell hell Hello Hello Hello hell hell hi hi hi Hi hi hi hell hell hey hell hell Hell hell hell']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conversation_history.append(input_text)\n",
    "conversation_history.append(response)\n",
    "conversation_history"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Step 6: Repeat\n",
    "\n",
    "We have gone through all the steps of interacting with your chatbot. Now, we can put everything in a loop and run a whole conversation! \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "while True:\n",
    "    # Create conversation history string\n",
    "    history_string = \"\\n\".join(conversation_history)\n",
    "\n",
    "    # Get the input data from the user\n",
    "    input_text = input(\"> \")\n",
    "\n",
    "    # Tokenize the input text and history\n",
    "    inputs = tokenizer.encode_plus(history_string, input_text, return_tensors=\"pt\")\n",
    "\n",
    "    # Generate the response from the model\n",
    "    outputs = model.generate(**inputs)\n",
    "\n",
    "    # Decode the response\n",
    "    response = tokenizer.decode(outputs[0], skip_special_tokens=True).strip()\n",
    "\n",
    "    # Add interaction to conversation history\n",
    "    conversation_history.append(input_text)\n",
    "    conversation_history.append(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Voila! We have built a simple, functional chatbot that we can interact with through our terminal!\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "llm-env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}