stateset_ai.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "stateset ai.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5m6cysSH4log",
        "colab_type": "text"
      },
      "source": [
        "**Stateset AI Notebook**\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NmQOT_H_2EwU",
        "colab_type": "text"
      },
      "source": [
        "This is going to be a working notebook of the latest developments in Deep Learning and Natural Language Processing (NLP).\n",
        "\n",
        "In 2014 I took a course in Computational Linguistics that introduced me to logic programming in Prolog.\n",
        "\n",
        "In 2016 I worked on my first NLP based bots using a context, utterance, entity detection and was training the model based on chatbot interactions. I used a stack powered by luis.ai and microsoft botkit, node.js while working at Apttus which later become Max. \n",
        "\n",
        "Fast forward 4 years and we now have BERT and Transformer based model leveraging deep learning methods and otherstate of the art natural language processing libraries.\n",
        "\n",
        "Introductory books and resources:\n",
        "\n",
        "https://mitpress.mit.edu/books/introduction-deep-learning\n",
        "\n",
        "https://www.deeplearning.ai/\n",
        "\n",
        "*    Dan Jurafsky and James H. Martin.&nbsp;Speech and Language Processing (3rd ed. draft)\n",
        "*    Jacob Eisenstein.&nbsp;Natural Language Processing\n",
        "*    Yoav Goldberg.&nbsp;A Primer on Neural Network Models for Natural Language Processing\n",
        "*    Ian Goodfellow, Yoshua Bengio, and Aaron Courville.&nbsp;Deep Learning\n",
        "*    Delip Rao and Brian McMahan.&nbsp;Natural Language Processing with PyTorch.\n",
        "*    Programming PyTorch for Deep Learning - Ian Pointer\n",
        "\n",
        "The other online courses I am looking through are:\n",
        "\n",
        "Ling581-20: Advanced Computational Linguistics Arizona Spring 2020\n",
        "\n",
        "CS224n: Natural Language Processing with Deep Learning\n",
        "Stanford / Winter 2020\n",
        "\n",
        "I will be taking this class starting in March with Jeremy Howard of fast.ai:\n",
        "\n",
        "Data Science Institute at USF | Deep Learning I, Starting in March\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HGYFndwR3Jg0",
        "colab_type": "text"
      },
      "source": [
        "The program I am going to be working on is incorporating the latest NLP technologies and libraries into Stateset. These technolgies are the following:\n",
        "\n",
        "\n",
        "\n",
        "*   PyTorch\n",
        "*   FastAI\n",
        "*   BERT\n",
        "*   StanfordCoreNLP\n",
        "*   NLTK\n",
        "\n",
        "\n",
        "The master plan:\n",
        "\n",
        "- Build a State of the Art Natural Language Processing (NLP) engine and embed into Stateset for communicating with the platform via voice and text.\n",
        "- Raise money on the Stateset platform and use that money train a better AI engine using Billions of Parameters.\n",
        "- Leverage that model to create revenue to train a better AI engine and create better predictions on the global Stateset Network. \n",
        "\n",
        "\n",
        "At the same time I will be working on developing Reflektion's Shopify integration and working on implementing the Machine Learning with customers. \n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OxISNTdsXdai",
        "colab_type": "text"
      },
      "source": [
        "The full stack consist of a next.js application which we will use the /api routing to call a nginx proxy on a flask server for the natural language processing.\n",
        "\n",
        "This container is being served in Kubernetes Engine on GCP. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6iUSYUnW35bG",
        "colab_type": "text"
      },
      "source": [
        "**The Following is using fastai Tabular / Structured Data deel learning**\n",
        "\n",
        "Tabular data is the most commonly used type of data in industry, but deep learning on tabular data receives far less attention than deep learning for computer vision and natural language processing.\n",
        "\n",
        "0.   Data Bunch: A data format for fast ai input\n",
        "1.   Dependent variable: The variable to predict\n",
        "2.   Categorical columns: The text/label columns. Or the columns with low            cardinality, eg. gender, type, year etc.\n",
        "3.    Continuous columns: Numeric value columns, usually with higher cardinality, eg. salary, price, temperature.\n",
        "4.    Transformations: Feature engineering and handling data eg. Missing values, Normalisation etc\n",
        "\n",
        "tabular contains all the necessary classes to deal with tabular data, across two modules:\n",
        "\n",
        "tabular.transform: defines the TabularTransform class to help with preprocessing;\n",
        "\n",
        "tabular.data: defines the TabularDataset that handles that data, as well as the methods to quickly get a TabularDataBunch.\n",
        "\n",
        "**The data has been split into two groups:**\n",
        "\n",
        "training set (train.csv)\n",
        "test set (test.csv)\n",
        "\n",
        "The training set should be used to build your machine learning models. For the training set, we provide the outcome (also known as the &ldquo;ground truth&rdquo;) for each passenger. Your model will be based on &ldquo;features&rdquo; like passengers&rsquo; gender and class. You can also use feature engineering to create new features.\n",
        "\n",
        "The test set should be used to see how well your model performs on unseen data. For the test set, we do not provide the ground truth for each passenger. It is your job to predict these outcomes. For each passenger in the test set, use the model you trained to predict whether or not they survived the sinking of the Titanic.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BDlILPInDUhX",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from fastai.tabular import * "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "53Jdkn22DexQ",
        "colab_type": "code",
        "outputId": "0d195aa3-22ba-4309-dc05-429c7f4811d3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "path = untar_data(URLs.ADULT_SAMPLE)\n",
        "path"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading http://files.fast.ai/data/examples/adult_sample\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              ""
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "PosixPath('/root/.fastai/data/adult_sample')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NaZ4t1skDnQt",
        "colab_type": "code",
        "outputId": "2ac0df26-e9ec-41ef-8dae-9363e6cfc8e4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 281
        }
      },
      "source": [
        "df = pd.read_csv(path/'adult.csv')\n",
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>workclass</th>\n",
              "      <th>fnlwgt</th>\n",
              "      <th>education</th>\n",
              "      <th>education-num</th>\n",
              "      <th>marital-status</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>race</th>\n",
              "      <th>sex</th>\n",
              "      <th>capital-gain</th>\n",
              "      <th>capital-loss</th>\n",
              "      <th>hours-per-week</th>\n",
              "      <th>native-country</th>\n",
              "      <th>salary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>49</td>\n",
              "      <td>Private</td>\n",
              "      <td>101320</td>\n",
              "      <td>Assoc-acdm</td>\n",
              "      <td>12.0</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Wife</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>1902</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&gt;=50k</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>44</td>\n",
              "      <td>Private</td>\n",
              "      <td>236746</td>\n",
              "      <td>Masters</td>\n",
              "      <td>14.0</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Exec-managerial</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>10520</td>\n",
              "      <td>0</td>\n",
              "      <td>45</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&gt;=50k</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>38</td>\n",
              "      <td>Private</td>\n",
              "      <td>96185</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>Black</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>32</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;50k</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>38</td>\n",
              "      <td>Self-emp-inc</td>\n",
              "      <td>112847</td>\n",
              "      <td>Prof-school</td>\n",
              "      <td>15.0</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Husband</td>\n",
              "      <td>Asian-Pac-Islander</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&gt;=50k</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>42</td>\n",
              "      <td>Self-emp-not-inc</td>\n",
              "      <td>82297</td>\n",
              "      <td>7th-8th</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Other-service</td>\n",
              "      <td>Wife</td>\n",
              "      <td>Black</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>50</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;50k</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   age          workclass  fnlwgt  ... hours-per-week  native-country salary\n",
              "0   49            Private  101320  ...             40   United-States  >=50k\n",
              "1   44            Private  236746  ...             45   United-States  >=50k\n",
              "2   38            Private   96185  ...             32   United-States   <50k\n",
              "3   38       Self-emp-inc  112847  ...             40   United-States  >=50k\n",
              "4   42   Self-emp-not-inc   82297  ...             50   United-States   <50k\n",
              "\n",
              "[5 rows x 15 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l2sRjqSlDqnF",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "procs = [FillMissing, Categorify, Normalize]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pvfpzOpnEcvO",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "valid_idx = range(len(df)-2000, len(df))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fPG5AHaTEe02",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "dep_var = 'salary'\n",
        "cat_names = ['workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race', 'sex', 'native-country']"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xlCWzRGCEkQW",
        "colab_type": "code",
        "outputId": "54da944a-32c5-491f-e3e8-50056c12ee1a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "data = TabularDataBunch.from_df(path, df, dep_var, valid_idx=valid_idx, procs=procs, cat_names=cat_names)\n",
        "print(data.train_ds.cont_names)  # `cont_names` defaults to: set(df)-set(cat_names)-{dep_var}"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['capital-gain', 'fnlwgt', 'hours-per-week', 'capital-loss', 'education-num', 'age']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N_WY-f94Eoed",
        "colab_type": "code",
        "outputId": "f544dec6-1d6e-4897-d85c-be27cde87979",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 212
        }
      },
      "source": [
        "(cat_x,cont_x),y = next(iter(data.train_dl))\n",
        "for o in (cat_x, cont_x, y): print(to_np(o[:5]))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[ 6 12  3  6  1  5  2 40  1]\n",
            " [ 5 10  3  2  1  5  2 40  1]\n",
            " [ 5 12  3 15  1  5  2 40  1]\n",
            " [ 1  6  7  1  2  5  1 40  1]\n",
            " [ 3 16  3  9  1  5  2 40  1]]\n",
            "[[ 1.871961  0.231205 -0.035789 -0.216787 -0.421569  0.613348]\n",
            " [ 2.54714   0.096588  0.772127 -0.216787  1.145728  2.736934]\n",
            " [-0.145922 -1.018481 -0.035789 -0.216787 -0.421569  0.173985]\n",
            " [-0.145922 -1.239652  0.287377 -0.216787 -2.380691 -0.631513]\n",
            " [-0.145922  0.920516 -0.035789 -0.216787 -0.029745  2.224345]]\n",
            "[1 1 0 0 0]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n4j2jOeEEsnW",
        "colab_type": "code",
        "outputId": "62802ba9-5b22-4a80-f0a1-83a857f5c4e7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 77
        }
      },
      "source": [
        "learn = tabular_learner(data, layers=[200,100], emb_szs={'native-country': 10}, metrics=accuracy)\n",
        "learn.fit_one_cycle(1, 1e-2)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>epoch</th>\n",
              "      <th>train_loss</th>\n",
              "      <th>valid_loss</th>\n",
              "      <th>accuracy</th>\n",
              "      <th>time</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>0.315658</td>\n",
              "      <td>0.320534</td>\n",
              "      <td>0.844500</td>\n",
              "      <td>00:06</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3N9vO2RxFuiW",
        "colab_type": "code",
        "outputId": "13043e54-be2a-4211-d0f0-2b8d925d873a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "learn.predict(df.iloc[2])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Category <50k, tensor(0), tensor([0.9861, 0.0139]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e4wzgV99GC22",
        "colab_type": "code",
        "outputId": "7ac6e1d1-e2be-458e-d743-00d8a9824c52",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "learn.predict(df.iloc[1])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Category >=50k, tensor(1), tensor([0.3900, 0.6100]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9dGtj802GFq2",
        "colab_type": "code",
        "outputId": "a09dda1d-ec7e-413d-80a6-18a59d244397",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 167
        }
      },
      "source": [
        "learn.predict(df.iloc[])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-1-e2874c51c196>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mlearn\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0miloc\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'learn' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R-MLkW-VGMIe",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6HUURUjh4PTA",
        "colab_type": "text"
      },
      "source": [
        "**The Following is using fastai Text**\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "FDVI6mu44eNM",
        "colab": {}
      },
      "source": [
        "import pathlib"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "XdCTNcxv4eNP",
        "colab": {}
      },
      "source": [
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "uFGg_Grw4eNS",
        "colab": {}
      },
      "source": [
        "from fastai.text import * "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "c30173af-c55a-4cd2-d92c-a8c1f8d4d8f7",
        "id": "urnh4JFw4eNV",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "source": [
        "path = untar_data(URLs.IMDB_SAMPLE)\n",
        "path"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading http://files.fast.ai/data/examples/imdb_sample\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              ""
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "PosixPath('/root/.fastai/data/imdb_sample')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "19c02553-d3ff-415f-9df6-c34f5118e925",
        "id": "VbNKWsv64eNZ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "df = pd.read_csv(path/'texts.csv')\n",
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>label</th>\n",
              "      <th>text</th>\n",
              "      <th>is_valid</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>negative</td>\n",
              "      <td>Un-bleeping-believable! Meg Ryan doesn't even ...</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>positive</td>\n",
              "      <td>This is a extremely well-made film. The acting...</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>negative</td>\n",
              "      <td>Every once in a long while a movie will come a...</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>positive</td>\n",
              "      <td>Name just says it all. I watched this movie wi...</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>negative</td>\n",
              "      <td>This movie succeeds at being one of the most u...</td>\n",
              "      <td>False</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      label                                               text  is_valid\n",
              "0  negative  Un-bleeping-believable! Meg Ryan doesn't even ...     False\n",
              "1  positive  This is a extremely well-made film. The acting...     False\n",
              "2  negative  Every once in a long while a movie will come a...     False\n",
              "3  positive  Name just says it all. I watched this movie wi...     False\n",
              "4  negative  This movie succeeds at being one of the most u...     False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "4b6fb363-29a2-41c3-d4b4-1ef813aa7351",
        "id": "x-0yQfZG4eNd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17
        }
      },
      "source": [
        "# Language model data\n",
        "data_lm = TextLMDataBunch.from_csv(path, 'texts.csv')\n",
        "# Classifier model data\n",
        "data_clas = TextClasDataBunch.from_csv(path, 'texts.csv', vocab=data_lm.train_ds.vocab, bs=32)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              ""
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              ""
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              ""
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              ""
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "kDgANhxF4eNg",
        "colab": {}
      },
      "source": [
        "data_lm.save('data_lm_export.pkl')\n",
        "data_clas.save('data_clas_export.pkl')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "1gF878Zv4eNj",
        "colab": {}
      },
      "source": [
        "data_lm = load_data(path, 'data_lm_export.pkl')\n",
        "data_clas = load_data(path, 'data_clas_export.pkl', bs=16)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "fb50c15d-1c7c-440d-8d90-3e32cda40b3f",
        "id": "1ty-dmWh4eNm",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 97
        }
      },
      "source": [
        "learn = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.5)\n",
        "learn.fit_one_cycle(1, 1e-2)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading https://s3.amazonaws.com/fast-ai-modelzoo/wt103-fwd\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              ""
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>epoch</th>\n",
              "      <th>train_loss</th>\n",
              "      <th>valid_loss</th>\n",
              "      <th>accuracy</th>\n",
              "      <th>time</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>4.329532</td>\n",
              "      <td>3.891484</td>\n",
              "      <td>0.285842</td>\n",
              "      <td>06:59</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "e74d2c60-89b1-4e14-8aa3-1d67050d9800",
        "id": "Ibb_Z4W24eNp",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 80
        }
      },
      "source": [
        "learn.unfreeze()\n",
        "learn.fit_one_cycle(1, 1e-3)\n"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>epoch</th>\n",
              "      <th>train_loss</th>\n",
              "      <th>valid_loss</th>\n",
              "      <th>accuracy</th>\n",
              "      <th>time</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>3.949292</td>\n",
              "      <td>3.820232</td>\n",
              "      <td>0.293240</td>\n",
              "      <td>10:52</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "874e3a87-c339-4f84-e301-bc9d356e750f",
        "id": "JlzsMjfR4eNs",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "learn.predict(\"This is a review about\", n_words=10)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'This is a review about New York City from records directed by'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "EsaIZHR_4eNu",
        "colab": {}
      },
      "source": [
        "learn.save_encoder('ft_enc')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "1BN8dZuu4eNw",
        "colab": {}
      },
      "source": [
        "learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.5)\n",
        "learn.load_encoder('ft_enc')\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "c6cf1bed-1c56-435d-caeb-4ed7e6d3dd1b",
        "id": "hbiwsQZj4eNy",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 289
        }
      },
      "source": [
        "data_clas.show_batch()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th>text</th>\n",
              "      <th>target</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>xxbos xxmaj xxunk xxmaj victor xxmaj xxunk : a xxmaj review \\n \\n  xxmaj you know , xxmaj xxunk xxmaj victor xxmaj xxunk is like sticking your hands into a big , xxunk xxunk of xxunk . xxmaj it 's warm and xxunk , but you 're not sure if it feels right . xxmaj try as i might , no matter how warm and xxunk xxmaj xxunk xxmaj</td>\n",
              "      <td>negative</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>xxbos xxup the xxup shop xxup around xxup the xxup corner is one of the sweetest and most feel - good romantic comedies ever made . xxmaj there 's just no getting around that , and it 's hard to actually put one 's feeling for this film into words . xxmaj it 's not one of those films that tries too hard , nor does it come up with</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>xxbos xxmaj now that xxmaj che(2008 ) has finished its relatively short xxmaj australian cinema run ( extremely limited xxunk screen in xxmaj sydney , after xxunk ) , i can xxunk join both xxunk of \" xxmaj at xxmaj the xxmaj movies \" in taking xxmaj steven xxmaj soderbergh to task . \\n \\n  xxmaj it 's usually satisfying to watch a film director change his style /</td>\n",
              "      <td>negative</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>xxbos xxmaj this film sat on my xxmaj tivo for weeks before i watched it . i dreaded a self - indulgent xxunk flick about relationships gone bad . i was wrong ; this was an xxunk xxunk into the screwed - up xxunk of xxmaj new xxmaj yorkers . \\n \\n  xxmaj the format is the same as xxmaj max xxmaj xxunk ' \" xxmaj la xxmaj ronde</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>xxbos xxmaj many neglect that this is n't just a classic due to the fact that it 's the first xxup 3d game , or even the first xxunk - up . xxmaj it 's also one of the first stealth games , one of the xxunk definitely the first ) truly claustrophobic games , and just a pretty well - xxunk gaming experience in general . xxmaj with graphics</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "e517fad5-1ee7-4e9b-ec01-9efafeb01d9f",
        "id": "t_3jP2MV4eN1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 80
        }
      },
      "source": [
        "learn.freeze_to(-2)\n",
        "learn.fit_one_cycle(1, slice(5e-3/2., 5e-3))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: left;\">\n",
              "      <th>epoch</th>\n",
              "      <th>train_loss</th>\n",
              "      <th>valid_loss</th>\n",
              "      <th>accuracy</th>\n",
              "      <th>time</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>0</td>\n",
              "      <td>0.578390</td>\n",
              "      <td>0.473341</td>\n",
              "      <td>0.810945</td>\n",
              "      <td>07:59</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "QEwIjIvC4eN4",
        "colab": {}
      },
      "source": [
        "learn.unfreeze()\n",
        "learn.fit_one_cycle(1, slice(2e-3/100, 2e-3))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "cc5259ac-7639-4f47-ae0d-b4b6de3ef932",
        "id": "ZRQWoue54eN9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "learn.predict(\"This was a great movie!\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Category positive, tensor(1), tensor([0.3372, 0.6628]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "83a1c815-a9bc-4686-d02a-cbf4e373b776",
        "id": "5ELfLzoO4eN_",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "learn.predict(\"That movie was not that good\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Category negative, tensor(0), tensor([0.7610, 0.2390]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "bc17f67f-311e-4607-c672-9a640a35d0da",
        "id": "jtEey9Uo4eOC",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "learn.predict(\"It was ok\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Category negative, tensor(0), tensor([0.5557, 0.4443]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "f3dec0ab-4ee0-43e7-f6d7-8b31334ee17e",
        "id": "2UXYXRpJ4eOI",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "learn.predict(\"That movie was epic!\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Category positive, tensor(1), tensor([0.4491, 0.5509]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "outputId": "16541773-7568-4820-9391-e4868c0f7892",
        "id": "hWIFnFwh4eON",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "learn.predict(\"Awesome movie!\")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(Category negative, tensor(0), tensor([0.5094, 0.4906]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "t6Evejk-4eOQ"
      },
      "source": [
        "This was my first model using fastai with NLP built on 1/22/2020 by Dom Steil\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "W2_RW-NK4eOR",
        "colab": {}
      },
      "source": [
        "learn.save('nlp-8')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B1AjDqYil7JJ",
        "colab_type": "text"
      },
      "source": [
        "Recurrent Neural Network (RNN) for Natural Language Processing using three libraries:\n",
        "\n",
        "\n",
        "\n",
        "*   torchtext\n",
        "*   spaCy\n",
        "*   pandas\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TEMScYKhmOO8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "tweetsDF = pd.read_csv(\"training.1600000.processed.noemoticon.csv\",\n",
        "                       engine=\"python\", header=None)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IeI2JfPAmgKc",
        "colab_type": "code",
        "outputId": "4375097c-cbbd-426b-ff7c-e23b3dc70d51",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 203
        }
      },
      "source": [
        "from torchtext import data\n",
        "\n",
        "\n",
        "LABEL = data.LabelField()\n",
        "STATESET = data.Field(tokenize='spacy', lower=true)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-10-05ebb2cf0d72>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mLABEL\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mLabelField\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0mSTATESET\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mField\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtokenize\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'spacy'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlower\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'true' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TdLL50CPnENl",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "fields = ['id', 'product_id', 'name', 'description', 'product_url', 'image_url', 'breadcrumbs', 'price', 'special_price', 'status', 'quantity', 'inventory_status', 'custom_color' ];"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ou3mGrO4nYVk",
        "colab_type": "text"
      },
      "source": [
        "Define the fields and product a list that maps them onto the list of rows that are in the CSV file produced from the Product Data Feed."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1wOolXbdndx9",
        "colab_type": "code",
        "outputId": "1c80de4d-90d7-4ad0-cde2-7bb6f841af30",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 238
        }
      },
      "source": [
        "statesetDataset = torchtext.data.TabularDataset(\n",
        "    path=\"./training-processed.csv\",\n",
        "    format=\"CSV\",\n",
        "    fields=fields,\n",
        "    skip_header=False\n",
        ")"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-5-18025bede6e8>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m statesetDataset = torchtext.data.TabularDataset(\n\u001b[0m\u001b[1;32m      2\u001b[0m     \u001b[0mpath\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"./training-processed.csv\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m     \u001b[0mformat\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"CSV\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m     \u001b[0mfields\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mfields\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[0mskip_header\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'torchtext' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MSdFqiJZn7ut",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VM1obIYLoM5X",
        "colab_type": "text"
      },
      "source": [
        "Split the training data into training, testing and validation sets"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FW9JENU3oR99",
        "colab_type": "code",
        "outputId": "72bed134-d212-475b-d5bf-e49f51b5db40",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 203
        }
      },
      "source": [
        "(train, test, valid) = statesetDataset.split(split_ratio=[0.8,0.1,0.1])\n",
        "\n",
        "(len(train),len(test),len(valid))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-8-ba6d7aa92aeb>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;34m(\u001b[0m\u001b[0mtrain\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtest\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalid\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstatesetDataset\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msplit_ratio\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0.8\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m0.1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m0.1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtrain\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtest\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvalid\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mNameError\u001b[0m: name 'statesetDataset' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IKQG_hasodMk",
        "colab_type": "code",
        "outputId": "acdb8646-994a-4f20-b337-53c1f31f590a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 185
        }
      },
      "source": [
        "vocab_size = 2000\n",
        "STATESET.build_vocab(train, max_size = vocab_size)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "error",
          "ename": "NameError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-9-1e248e529bfc>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mvocab_size\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m2000\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mSTATESET\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbuild_vocab\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtrain\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_size\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mvocab_size\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mNameError\u001b[0m: name 'STATESET' is not defined"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z4yFLM6AH9at",
        "colab_type": "code",
        "outputId": "63c3c99f-48ef-4adf-a6e1-2fdf8a3434b7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 132
        }
      },
      "source": [
        "train_iterator, valid_iterator, test_iterator = data.BucketIterator.splits(\n",
        "    (train, valid, test),\n",
        "    batch_size = 32,\n",
        "    device = device)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "error",
          "ename": "SyntaxError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-11-7daab19f7d24>\"\u001b[0;36m, line \u001b[0;32m5\u001b[0m\n\u001b[0;31m    )\u001b[0m\n\u001b[0m    ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FFcSMuBPIk3G",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}