Skip to content

chaselikethebank/backend-server-cron-yt-data-fetch

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

19 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Express Playlist API with MongoDB Integration and Rate Limiting

This application serves as an API for managing a playlist. It fetches data from the YouTube API, saves it to MongoDB, and provides endpoints to retrieve the playlist data. Additionally, it implements rate limiting to prevent abuse of the API.

Technologies Used

  • Node.js
  • Express.js
  • MongoDB
  • Axios
  • CORS
  • dotenv
  • node-schedule
  • express-rate-limit

Setup

  1. Clone the repository.
  2. Install the dependencies using npm install.
  3. Create a .env file and add the following environment variables:
    • PORT: Port number for the Express server.
    • MONGODB_URI: MongoDB connection URI.
    • YOUTUBE_API_KEY: Your YouTube Data API v3 key.
  4. Run the application using npm start.

Functionality

  • Fetching and Saving Playlist Data: The application fetches playlist data from the YouTube API, processes it, and saves it to MongoDB. It checks for new items and inserts them into the database if they are not already present.

  • Scheduled Data Fetching: The application schedules a job to fetch playlist data periodically using the node-schedule package. By default, it runs every Tuesday after lunch.

  • Rate Limiting: The application implements rate limiting using the express-rate-limit middleware. It restricts the number of requests from a single IP address within a specified time window to prevent abuse and ensure fair usage of the API.

  • API Endpoints:

    • /fetch-youtube-playlist: Endpoint to manually trigger fetching and saving of playlist data.
    • /playlist: Endpoint to retrieve the playlist data from MongoDB.

Manual process for updating

playlist urls and names: loft: https://www.youtube.com/playlist?list=PLgE8B1SdCTDshXW3ET6UHQQzav65px_Qx woodforest: https://www.youtube.com/playlist?list=PLgE8B1SdCTDvBQalk1CLJQ6tLjSxsXwfw traditional: https://www.youtube.com/playlist?list=PLgE8B1SdCTDtykVWGL75lJHPhfnY5CqEG harvest: https://www.youtube.com/playlist?list=PLgE8B1SdCTDvc8Ogp6Lced5yCf7x9-JJB

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors