Collecting Photo Stream in Node.js on Heroku
•
1 like•2,570 views
This document discusses collecting photos from Twitter streams in real-time using Node.js on Heroku. It describes tracking hashtags and photo URLs to filter tweets containing images, and extracting the photo URLs. The architecture involves using the Twitter Streaming API and socket.io for real-time streaming to browsers, with fallbacks for Heroku which does not yet support WebSockets. The goal is to present the photo stream to users in real-time.
Collecting Photo Stream in Node.js on Heroku
- 1. Collecting Photo Stream in Node.js on Heroku @stomita (ja) @shinichitomita (en)
- 2. Who am I ? • CEO at Mashmatrix, Inc. • JavaScript Developer • Formerly (almost forgot) • Identity (OpenID/SAML/SSO) @stomita • JavaEE • Oracle • Ex-Salesforce.com (2006/12-2007/12)
- 3. Objective • Collect and store photos tweeted in the world • Add metadata to photos manually by crowd • Category ? (food/people/landscape) • Venue ? (foursquare)
- 5. Photo Stream • Set tracking keywords to filter tweets with photo uploading services URL • e.g. tracks=lockerz,plixi,tweetphoto,twitpic,yfrog, photozou,twitgoo,instagr • Extract photo URLs by pre-given rule • e.g. http://twitpic.com/7wiwlw => http://twitpic.com/show/thumb/7wiwlw
- 6. Architecture Streaming JSON
- 8. Visualization • Present photo stream to users in real-time • Not our primary goal, but just interesting • Don’t think, FEEL.
- 10. socket.io • A Node.js library, enabling push messaging to browsers • Stream Transport Supported : • WebSocket/FlashSocket/XHR-Multipart/ XHR-polling/JSONP-polling • Used as fallback of SocketStream
- 11. Architecture Streaming socket.io
- 12. Similar Work - Twitcaps http://twitcaps.com/
- 13. Twitcaps - Not Real-time Delayed Streaming Ajax Polling
- 14. Issue • Heroku Node.js doesn’t support WebSocket yet (as of Dec 2011) • Fallbacks to XHR-polling • c.f. Joyent No.de SmartMachine
- 15. End