Media Agent scrapes Twitter and Reddit submissions, summarizes them, and chats with them in an interactive terminal.

https://www.loom.com/share/f4954e7d34ef4b7b8491e2bf910e8521

• Scrapes tweets/submissions on your behalf either from a list of user accounts or a list of keywords.
• Embeds the tweets/submissions using OpenAI
• Indexes the embeddings (i.e. vectors) in ChromaDB
• Enriches the index with additional metadata
• Creates a summary of the tweets/submissions and provides potential questions to answer
• Opens a chat session on top of the tweets
• Saves the conversation with its metadata
• A rich terminal UI and logging features

• Langchain 🦜 to build and compose LLMs
• ChromaDB to store vectors (a.k.a embeddings) and query them to build conversational bots
• Tweepy to connect to your the Twitter API and extract Tweets and metadata
• Praw to connect to Reddit API
• Rich to build a cool terminal UX/UI
• Poetry to manage dependencies

• OpenAI (🔑 needed)
• Twitter (🔑 needed)
• Reddit (🔑 needed)

• Install dependencies with poetry

poetry install --with dev

• Add API credentials

Create .env file at the root of the project with the following keys:

OPENAI_API_KEY=<OPENAI KEY>
TWITTER_BEARER_TOKEN=<TWITTER BEARER TOKEN>
REDDIT_API_CLIENT_ID=<REDDIT_API_CLIENT_ID>
REDDIT_API_SECRET=<REDDIT_API_SECRET>
REDDIT_USER_AGENT=<REDDIT_USER_AGENT>

More info on these credentials here, here and here

• Run the app

make run-media-agent

This is an on-going project, so feel free to contribute:

Here's what I plan in the near future:

• Add more data sources: substack, press, LinkedIN
• support open-source LLMs (Falcon?)
• support Pinecone in addition to ChromaDB
• add instructions to deployment on a cloud infrastructure
• improve the prompts to have a more engaging/enriching conversations
• add actions to open URLs and fetch content from it