This Jupyter notebook demonstrates text generation using a Long Short-Term Memory (LSTM) Recurrent Neural Network model trained on a corpus of Amazon product reviews.
The notebook loads a dataset of 5000 Amazon reviews for a Fire HD tablet. It performs the following steps:
-
Imports libraries like Pandas, Numpy, and Keras.
-
Loads the review dataset into a Pandas DataFrame.
-
Plots the number of reviews over time to visualize the data.
-
Tokenizes the text data into words and sentences using NLTK.
-
Loads a pre-trained Word2Vec embedding model from Google News data.
-
Creates training sequences and labels by extracting subsequences from each review.
-
Converts the words to indexes based on the vocabulary.
-
Creates an embedding matrix using Word2Vec vectors for each word.
-
Builds a Keras LSTM model with trained embedding layer, LSTM layer, dropout, and dense layer.
-
Compiles and trains the model on the review data.
-
Generates new text by feeding seed text into the trained model to predict next words.
The model architecture consists of:
- Embedding layer initialized with Word2Vec embeddings
- Masking layer
- LSTM layer with 50 memory units
- Dropout layer for regularization
- Fully connected output layer with softmax activation
The model is trained for 25 epochs with early stopping to prevent overfitting. Checkpointing is used to save the best model weights.
The notebook provides full code to load data, build, train and generate text from the LSTM model.
It can be easily adapted to any text corpus by:
- Changing the input data source
- Adjusting model hyperparameters
- Modifying text preprocessing
The resulting model can generate new realistic text after training on a large dataset.
The notebook implements text generation based on concepts from:
- Embedding words for input to RNN: https://towardsdatascience.com/word-embedding-for-sequence-modeling-part-1-e585c43994db
- LSTM for text generation: https://towardsdatascience.com/text-generation-with-lstm-recurse-networks-8565c2402f28
- Text generation using Keras LSTM: https://machinelearningmastery.com/text-generation-lstm-recurrent-neural-networks-python-keras/