Applied machine learning at facebook a datacenter infrastructure perspective HPCA18
Applied machine learning at facebook a datacenter infrastructure perspective HPCA18AI-enhanced description This paper discusses the infrastructure that supports machine learning at Facebook, which serves 2.1 billion users through various applications like news feed ranking, ads, search, and language translation. Facebook employs a diverse set of machine learning methods, primarily using CPUs for i
facebookã®13億ユーザーを支ãˆã‚‹ãƒãƒ¼ãƒ‰ãƒãƒ©ãƒ³ã‚µãƒ¼ã®è©± - stanaka's blog
最近ã€SREãŒè©±é¡Œã§ã™ã。 tech.mercari.com www.wantedly.com ã¨ã„ã†ã“ã¨ã§SREã«ã¤ã„ã¦èª¿ã¹ã¦ãŸã‚‰ã€SREconãªã‚“ã¦ã‚‚ã®ãŒé–‹å‚¬ã•ã‚Œã¦ã„ãŸã®ã§ä¸ã‚’見ã¦ãŸã‚‰ã€ã€ŒBuilding a Billion User Load Balancerã€ã¨ã„ã†ã‚¿ã‚¤ãƒˆãƒ«ã§Facebookã®DNS〜LBã¾ã§ã®è©±ãŒã‚ã£ãŸã®ã§ã€ãã®ãƒ¡ãƒ¢ã§ã™ã€‚ Building a Billion User Load Balancer | USENIX tl;dr tinydns + IPVS 㧠Facebookè¦æ¨¡ã¯ã„ã‘ã‚‹ httpsã®æŽ¥ç¶šç¢ºç«‹ã¯ã‹ãªã‚Šé‡ã„(RTTã®4å€ = RTT 150msã¨ã™ã‚‹ã¨GETã¾ã§600ms)ã®ã§ã€å¤ªå¹³æ´‹è¶Šãˆã¨ã‹ã¯åŽ³ã—ã„ httpsを終端ã•ã›ã‚‹CDNã¨ã‹ã¯æ´»ç”¨ã®å¯èƒ½æ€§ã‚ã‚Šãㆠ(国内ã ã‘を考慮ã™ã‚‹ãªã‚‰å½±éŸ¿ã¯è»½å¾®ã‹ã‚‚) メモ L4 LB shiv (IPVS + pyt
Web performance: Cache efficiency exercise
Speed is a consideration for any website, whether it's for the local barbershop or Wikipedia, with its huge repository of knowledge. It's a feature that shouldn't be ignored. This is why caching is important — a great way to make websites faster is to save parts of them so they don't have to be calculated or downloaded again on the next visit. My team was recently having a discussion about the par
Introducing osquery
Maintaining real-time insight into the current state of your infrastructure is important. At Facebook, we’ve been working on a framework called osquery which attempts to approach the concept of low-level operating system monitoring a little differently. Osquery exposes an operating system as a high-performance relational database. This design allows you to write SQL-based queries efficiently and e
Facebook's software architecture
On distributed systems broadly defined and other curiosities. The opinions on this site are my own. I had summarized/discussed a couple papers (Haystack, Memcache caching) about Facebook's architecture before. Facebook uses simple architecture that gets things done. Papers from Facebook are refreshingly simple, and I like reading these papers. Two more Facebook papers appeared recently, and I brie
Scaling Memcache at Facebook | USENIX
by Michael Piatek Responsiveness is essential for web services. Speed drives user engagement, which drives revenue. To reduce response latency, modern web services are architected to serve as much as possible from in-memory caches. The structure is familiar: a database is split among servers with caches for scaling reads. Over time, caches tends to accumulate more responsibility in the storage sta
Instagram moves 20 billion images to Facebook servers - SiliconANGLE
Instagram announced last week that it’s picked up its billions of images stored in Amazon Web Services (AWS) and dumped them into Facebook’s own servers in one of the largest data migration operations ever undertaken. News of the move came from this interview with Facebook infrastructure engineer and Open Compute Foundation program developer Charlie Manese. Manese revealed that the massive migrati
Scaling the Facebook data warehouse to 300 PB
At Facebook, we have unique storage scalability challenges when it comes to our data warehouse. Our warehouse stores upwards of 300 PB of Hive data, with an incoming daily rate of about 600 TB. In the last year, the warehouse has seen a 3x growth in the amount of data stored. Given this growth trajectory, storage efficiency is and will continue to be a focus for our warehouse infrastructure. There
An Analysis of Facebook Photo Caching
An Analysis of Facebook Photo Caching Qi Huang∗, Ken Birman∗, Robbert van Renesse∗, Wyatt Lloyd†‡, Sanjeev Kumar‡, Harry C. Li‡ ∗Cornell University, †Princeton University, ‡Facebook Inc. Abstract This paper examines the workload of Facebook’s photo- serving stack and the effectiveness of the many layers of caching it employs. Facebook’s image-management infrastructure is complex and geographically
An analysis of Facebook photo caching
Every day people upload more than 350 million photos to Facebook (as of Dec 2012) and view many more in their News Feeds and on their friends’ Timelines. Facebook stores these photos on Haystack machines that are optimized to store photos. But there is also a deep and distributed photo-serving stack with many layers of caches that delivers photos to people so they can view them. We recently publis
Facebookã®ç¶™ç¶šçš„デプãƒã‚¤ãƒ¡ãƒ³ãƒˆ - ワザノム| wazanova
https://www.facebook.com/publications/514128035341603/ 1æ—¥500件ã€3,000ファイルã«åŠã¶æœ¬ç•ªã‚¢ãƒƒãƒ— フãƒãƒ³ãƒˆã‚¨ãƒ³ãƒ‰ã®ã‚³ãƒ¼ãƒ‰ã¯1050万行ã€å†…850万行ãŒPHP 開発エンジニア1,000åã¨ãƒªãƒªãƒ¼ã‚¹ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢3å QAやテスターã¯å˜åœ¨ã—ãªã„ 自分ã§ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã‚’é¸ã¶ & 自己責任ã®ã‚«ãƒ«ãƒãƒ£ãƒ¼ãŒå¼·ã„。 1/3ã®ãƒ•ã‚¡ã‚¤ãƒ«ãŒä¸€äººã®ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã€1/4ãŒäºŒäººã®ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã§ãƒ¡ãƒ³ãƒ†ã•ã‚Œã¦ã„る。 フãƒãƒ³ãƒˆã‚¨ãƒ³ãƒ‰ã®æœ¬ç•ªã‚³ãƒ¼ãƒ‰ãƒ™ãƒ¼ã‚¹ã¯ä¸€ã¤ã®ã‚‚ã®ã‚’共有 日常æ¥å‹™ã§ã¯ãƒãƒ¼ã‚«ãƒ«ã®gitを利用。本番アップå¯èƒ½ã«ãªã‚Œã°ã€ä¸å¤®ã®ãƒ¬ãƒã‚¸ãƒˆãƒªã«ãƒžãƒ¼ã‚¸ã—ã¦ã€ãã‚Œã‹ã‚‰Subversion（éŽåŽ»ã®çµŒç·¯ã§ä½¿ã£ã¦ã„る。）ã«ã‚³ãƒŸãƒƒãƒˆã™ã‚‹ åŒã˜ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ãŒã‚³ãƒ¼ãƒ‰ã‚’コミットã™ã‚‹é–“éš”ã¯ä¸å¤®å€¤ã§10時間 本番ã«ãƒ—ッシュã™ã‚‹å‰ã«ã€æ‹…当エンジニア自身ã§ã®ãƒ¦ãƒ‹ãƒƒãƒˆãƒ†ã‚¹ãƒˆã‚’終ãˆã€åŒåƒšã«ã‚ˆã‚‹ã‚³ãƒ¼ãƒ‰
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
The Evolution of Machine Learning Platforms at Facebook Webcast Recap
GitHub - facebook/sapling: A Scalable, User-Friendly Source Control System.
Zero Downtime Release: Disruption-free Load Balancing of a Multi-Billion User Website
OCPREG19 - Facebook Challenges and Opportunities of Architecting AI Systems at Facebook Datacenter
ã€ECCV 2018】Exploring the Limits of Weakly Supervised Pretraining
GitHub - facebook/wangle: Wangle is a framework providing a set of common client/server abstractions for building services in a consistent, modular, and composable way.
GitHub - facebook/mcrouter: Mcrouter is a memcached protocol router for scaling memcached deployments.
Scaling Userspace @ Facebook
F4 - Photo Storage at Facebook
{{#tags}}- {{label}}
{{/tags}}