PastryVision is a computer vision project designed to help small pastry shops modernize their operations. By automatically detecting and tallying items in a customer's cart, the system enables a self-serve workflow that enhances efficiency while reducing costs. It eliminates the need for plastic packaging and barcode printing, promotes pastry freshness, and operates locally without requiring web hosting. With its simple and user-friendly setup, PastryVision empowers pastry shops to deliver a modern customer experience effortlessly.

1. Start/Stop the frontend client
2. Start/Stop the backend server

1. The frontend camera scans the pastry items on customer's tray.
2. The customer takes a snapshot of their tray.
3. The snapshot is sent to the backend to detect and identify the pastry items on the tray.
4. The backend automatically tallies the quantities and costs of the tray and sends this information to the frontend.
5. The customer can confirm the cost breakdown provided from the backend and proceed to pay or cancel their transaction.

• React
• Vite
• JavaScript
• TailwindCSS
• shadcn/ui
• FastAPI
• Python
• OpenCV
• Google MediaPipe
• TensorFlow
• NumPy

1. MediaPipe
2. React & ML Object Detection Example
3. Vite Configuration Guide
4. TailwindCSS Documentation
5. MediaPipe Object Detection Example
6. Training your own model using mediapipe
7. Bread dataset on roboflow
8. FastAPI - to build our backend
9. OpenCV - computer vision operations library
10. Python multipart parser
11. Mediapipe Examples GitHub Repo
12. BB Brain (example solution in Japan)
13. Viscovery

PastryVision was developed for the CVI620 NZA class at Seneca Polytechnic (Fall 2024), under Professor Savita Seharawat's guidance.

• Aryan Khurana
• Peter Wan
• Mimi Dang
• Jeremy Lee