Beringei: A high-performance time series storage engine
Operating large-scale, globally distributed services requires accurate monitoring of the health and performance of our systems to identify and diagnose problems as they arise. Facebook uses a time series database (TSDB) to track and store system measurements such as product stats (e.g., how many messages are being sent per minute), service stats (e.g., the rate of queries hitting the cache tier vs
FacebookãŒå…¬é–‹ã—ãŸ10億語を数分ã§å¦ç¿’ã™ã‚‹fastTextã§ä¸€ä½“何ãŒã§ãã‚‹ã®ã‹
fastTextã¨ã¯ä½•ãªã®ã‹ 自然言語処ç†ã®å¦ç¿’を高速化ã™ã‚‹ãƒ„ール ã“ã‚Œã¾ã§ï¼•æ—¥ã‹ã‹ã£ã¦ã„ãŸã‚¿ã‚¹ã‚¯ãŒãŸã£ãŸã®ï¼‘ï¼ç§’ã§çµ‚了 fastTextã§å–り組ã‚る３ã¤ã®ã“㨠fastTextã§å‡ºæ¥ã‚‹ï¼“ã¤ã®å…¨ä½“åƒ Facebookã¯ãƒ‹ãƒ¥ãƒ¼ã‚¹ãƒ•ã‚£ãƒ¼ãƒ‰ã‹ã‚‰é‡£ã‚Šè¦‹å‡ºã—を排除ã™ã‚‹ãŸã‚ã«fastTextã‚’ã¤ãã£ãŸï¼Ÿ リクルートテクノãƒã‚¸ãƒ¼ã‚ºã§ã¯ã€ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰ã«å¿œç”¨ サイãƒãƒ¼ã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆãŒå®Ÿç”¨åŒ–ã—ãŸAWAã§ã®ã‚¢ãƒ¼ãƒ†ã‚£ã‚¹ãƒˆãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰ Yahoo!ã¯ãƒ¬ã‚·ãƒ¼ãƒˆãƒ¡ãƒ¼ãƒ«ã®æ–‡ç« ã‹ã‚‰è£½å“をオススメã™ã‚‹ â—¯2Vecを考ãˆã‚Œã°æŽ¨è–¦ã«å¿œç”¨ã§ãã‚‹ fastTextを安全ã«ä½¿ã†ãŸã‚ã«å¿…è¦ãªç†è«– å˜èªžã‚’ベクトル表ç¾åŒ–ã™ã‚‹Word2Vec ベクトル表ç¾ã‚’構築ã™ã‚‹ã‚¢ãƒ¼ã‚テクãƒãƒ£ CBoW Skip-gram fastTextを使ã£ã¦ã¿ã‚ˆã† fastTextをインストールã™ã‚‹ å˜èªžã®ãƒ™ã‚¯ãƒˆãƒ«è¡¨ç¾ã‚’構築ã—よㆠTweetデータã®åŽé›† å˜èªžã®ãƒ™ã‚¯ãƒˆãƒ«è¡¨
MyRocks: A space- and write-optimized MySQL database
At Facebook we use MySQL to manage many petabytes of data, along with the InnoDB storage engine that serves social activities such as likes, comments, and shares. While InnoDB provides great performance and reliability for a variety of workloads, it has inefficiencies on space and write amplification when used with flash storage. A few years ago, we built RocksDB, an embeddable, persistent key-val
Cache-Control: immutable
About one year ago our friends at Facebook brought an interesting issue to the IETF HTTP Working Group - a lot (20%!) of their transactions for long lived resources (e.g. css, js) were resulting in 304 Not Modified. These are documents that have long explicit cache lifetimes of a year or more and are being revalidated well before they had even existed for that long. The problem remains unchanged.
Facebook's SSD findings: Failure, fatigue and the data center
Facebook's SSD findings: Failure, fatigue and the data center ​SSDs revolutionized data storage, even though we know little about how well they work. Now researchers at Facebook and Carnegie-Mellon share millions of hours of SSD experience Millions of SSDs are bought every year. It's easy to be impressed by fast boots and app starts. But what about 24/7 data center operations? What are the common
Facebookã€é™çš„解æžãƒ„ールInferをオープンソース化
Spring Bootã«ã‚ˆã‚‹APIãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰æ§‹ç¯‰å®Ÿè·µã‚¬ã‚¤ãƒ‰ 第2版 何åƒäººã‚‚ã®é–‹ç™ºè€…ãŒã€InfoQã®ãƒŸãƒ‹ãƒ–ック「Practical Guide to Building an API Back End with Spring Bootã€ã‹ã‚‰ã€Spring Bootを使ã£ãŸREST API構築ã®åŸºç¤Žã‚’å¦ã‚“ã 。ã“ã®æœ¬ã§ã¯ã€å‡ºç‰ˆæ™‚ã«æ–°ã—ãリリースã•ã‚ŒãŸãƒãƒ¼ã‚¸ãƒ§ãƒ³ã§ã‚ã‚‹ Spring Boot 2 を使用ã—ã¦ã„る。ã—ã‹ã—ã€Spring Boot3ãŒæœ€è¿‘リリースã•ã‚Œã€é‡è¦ãªå¤‰...
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
RocksDB is an embedded key-value store written in C++ and optimized for fast storage environments like flash or RAM. It uses a log-structured merge tree to store data by writing new data sequentially to an in-memory log and memtable, periodically flushing the memtable to disk in sorted SSTables. It reads from the memtable and SSTables, and performs background compaction to merge SSTables and remov
ã“ã®ãƒšãƒ¼ã‚¸ã‚’見るã«ã¯ã€ãƒã‚°ã‚¤ãƒ³ã¾ãŸã¯ç™»éŒ²ã—ã¦ãã ã•ã„
ã“ã®ãƒ–ラウザã§ã¯Facebookを利用ã§ãã¾ã›ã‚“Facebookã®åˆ©ç”¨ã‚’続ã‘ã‚‹ã«ã¯ã€ä»¥ä¸‹ã®ãƒ–ラウザã®ã„ãšã‚Œã‹ã‚’入手ã—ã¦ãã ã•ã„。詳ã—ãã¯ã“ã¡ã‚‰
FacebookãŒæ–°ã—ã„NoSQLã§ã‚ã‚‹Apolloを発表
Spring Bootã«ã‚ˆã‚‹APIãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰æ§‹ç¯‰å®Ÿè·µã‚¬ã‚¤ãƒ‰ 第2版 何åƒäººã‚‚ã®é–‹ç™ºè€…ãŒã€InfoQã®ãƒŸãƒ‹ãƒ–ック「Practical Guide to Building an API Back End with Spring Bootã€ã‹ã‚‰ã€Spring Bootを使ã£ãŸREST API構築ã®åŸºç¤Žã‚’å¦ã‚“ã 。ã“ã®æœ¬ã§ã¯ã€å‡ºç‰ˆæ™‚ã«æ–°ã—ãリリースã•ã‚ŒãŸãƒãƒ¼ã‚¸ãƒ§ãƒ³ã§ã‚ã‚‹ Spring Boot 2 を使用ã—ã¦ã„る。ã—ã‹ã—ã€Spring Boot3ãŒæœ€è¿‘リリースã•ã‚Œã€é‡è¦ãªå¤‰...
Saving capacity with HDFS RAID | Engineering Blog | Facebook Code | Facebook
As we continue to evolve our data infrastructure, we’re constantly looking for ways to maximize the utility and efficiency of our systems. One technology we’ve deployed is HDFS RAID, an implementation of Erasure Codes in HDFS to reduce the replication factor of data in HDFS. We finished putting this into production last year and wanted to share the lessons we learned along the way and how we incre
Scaling the Facebook data warehouse to 300 PB
At Facebook, we have unique storage scalability challenges when it comes to our data warehouse. Our warehouse stores upwards of 300 PB of Hive data, with an incoming daily rate of about 600 TB. In the last year, the warehouse has seen a 3x growth in the amount of data stored. Given this growth trajectory, storage efficiency is and will continue to be a focus for our warehouse infrastructure. There
ITæ¥ç•Œï¼ˆã‚¤ãƒ³ãƒ•ãƒ©ã‚ˆã‚Šï¼‰ã®ã¿ã‚“ãªï¼ã‚¢ãƒ‹ãƒ¡ã®BDを大人買ã„ã—ã¦ã‚‹å ´åˆã˜ã‚ƒãªã„ãž - kuenishi's blog
飲ã¿ä¼šã§è©±ã—ã¦ã„ãŸã‚‰ã€ã“ã®å±é™ºã•ã«æ°—ã¥ã„ã¦ã„る人ã‚ã¾ã‚Šã„ãªã‹ã£ãŸã‚ˆã†ãªã®ã§ã€‚ Google Driveã€æ¿€å€¤ä¸‹ã’―1TBãŒæœˆé¡49.9ドルã‹ã‚‰ãªã‚“ã¨æœˆé¡9.99ドル㫠アマゾン ウェブ サービスãŒAmazon Glacierを発表 Facebook uses 10,000 Blu-ray discs to store 'cold' data ãã®è¡æ’ƒçš„å‹•ç”» ã“ã†ã‚„ã£ã¦ãƒ‹ãƒ¥ãƒ¼ã‚¹ã‚’並ã¹ã‚‹ã¨è¡æ˜Žãªè«¸å›ã¯æ°—ã¥ãã ã‚ã†ãŒã€ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã®å˜ä¾¡ãŒã»ã¼åŒã˜ã 。Facebookã®è¨˜äº‹ã¯ã‚³ãƒ³ã‚·ãƒ¥ãƒ¼ãƒžå‘ã‘ã§ã¯ãªã„ãŒã€å˜ä¾¡ã¯ãã®ãƒ¬ãƒ™ãƒ«ã ã‚ã†ã¨æƒ³åƒã•ã‚Œã‚‹*1。 Glacierã¯ç™»å ´ã—ãŸå½“åˆã¯ã€Œã™ã‚テープドライブã¾ã§ã‚¯ãƒ©ã‚¦ãƒ‰åŒ–ã‹ï¼Ÿã€ã¨ã„ã†æ†¶æ¸¬ãŒé£›ã³äº¤ã£ãŸãŒã€ã€Œãƒ†ãƒ¼ãƒ—ã§ã¯ãªã„らã—ã„ãžã€ã¨ã„ã†æ†¶æ¸¬ã‚‚ç™»å ´ã—ã¦å®Ÿéš›ã®ã¨ã“ã‚ã¯åˆ†ã‹ã‚‰ãªã„。GMailã¯ãƒãƒƒã‚¯ã‚¢ãƒƒãƒ—ã«ãƒ‡ãƒ¼ãƒ—ドライブを使ã£ã¦ã„ãŸã‚‰ã—ã„ãŒä»Šã§ã‚‚ãã†ãªã®ã‹ã¯åˆ†ã‹ã‚‰ãªã„。
Using Cassandra for RTB systems
3. Real Time Bidding (RTB) â— Real-time bidding is a dynamic auction process where each impression is a bid for in (near) real time versus a static auction â— Kenshoo is engaged In Facebook Exchange (FBX) â— In FBX, each bid has a life-time of 120ms. All transactions have to complete within that period, and the winning ad is presented to the user. â— Kenshoo employs ad re-targeting, where search engin
Watchman·Facebook製ã®ãƒ•ã‚¡ã‚¤ãƒ«/フォルダ監視ツール MOONGIFT
ファイルã®å¤‰æ›´ã‚’監視ã—ã¦ä½•ã‚‰ã‹ã®å‡¦ç†ã‚’è¡Œã†ã¨è¨€ã£ãŸã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã¯é™çš„サイトã®æµè¡Œã‚‚ã‚ã£ã¦å¹¾ã¤ã‹å‡ºã¦ãã¦ã„ã¾ã™ã€‚軽é‡ãªã‚‚ã®ã€ãƒ—ラグインã§æ‹¡å¼µã§ãã‚‹ã‚‚ã®ãªã©è‰²ã€…ã§ã™ãŒã€ä»Šå›žã¯Facebookã«ã‚ˆã‚Šé–‹ç™ºã•ã‚Œã¦ã„ã‚‹Watchmanを紹介ã—ã¾ã™ã€‚ 特徴ã¨ã—ã¦ã¯ãƒ•ã‚¡ã‚¤ãƒ«ã®æ‹¡å¼µåã‚‚å•ã‚ãšã€æŒ‡å®šã—ãŸãƒ•ã‚¡ã‚¤ãƒ«ï¼ˆãƒ‘ターン指定å¯ï¼‰ãŒå¤‰æ›´ã•ã‚ŒãŸã‚‰ãれをトリガーã«ä½•ã‚‰ã‹ã®å‡¦ç†ã‚’è¡Œã†ã¨è¨€ã£ãŸã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã«ãªã‚‹ã‚ˆã†ã§ã™ã€‚ã•ã£ãã見ã¦ã„ãã¾ã—ょã†ã€‚ 最も簡å˜ãªä½¿ã„æ–¹ã¨ã—ã¦ã€CSSを変更ã—ãŸã‚‰ãƒŸãƒ‹ãƒ•ã‚¡ã‚¤ã™ã‚‹å ´åˆã¯æ¬¡ã®ã‚ˆã†ã«ãªã‚Šã¾ã™ã€‚ $ watchman -- trigger ~/src buildme '*.css' -- minify-css ã•ã‚‰ã«è¤‡æ•°ã®ãƒ‡ã‚£ãƒ¬ã‚¯ãƒˆãƒªã€ãƒ•ã‚¡ã‚¤ãƒ«ã‚’監視ã§ãã¾ã™ã€‚ $ watchman watch ~/www ã“ã®å ´åˆã€~/wwwを監視対象ã«åŠ ãˆã¾ã™ã€‚ã•ã‚‰ã«åˆ¥ãªãƒ‡ã‚£ãƒ¬ã‚¯ãƒˆãƒªã‚‚監視対象ã«åŠ ãˆ
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
GitHub - facebookarchive/fbtftp: fbtftp is Facebook's implementation of a dynamic TFTP server framework.
http://www.vldb.org/pvldb/vol8/p1816-teller.pdf
https://rocksdb.org/blog/323/the-1st-rocksdb-local-meetup-held-on-march-27-2014/
Scaling Userspace @ Facebook
https://qiita.com/kenokabe/items/9c650ec8bcb1418c596d
https://www.xtendsys.net/blog/post-1
{{#tags}}- {{label}}
{{/tags}}