LINE Storage: Storing billions of rows in Sharded-Redis and HBase per Month « NAVER Engineers' Blog
Hi, I’m Shunsuke Nakamura (@sunsuk7tp). Just half a year ago, I completed the Computer Science Master’s program in Tokyo Tech and joined to NHN Japan as a member of LINE server team. My ambition is to hack distributed processing and storage systems and develop the next generation’s architecture. In the LINE server team, I’m in charge of development and operation of the advanced storage system whi
èªè¨¼ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã¸ã®HBase/Hadoopã®é©ç”¨
3.1. テーブルスã‚ーマ HBaseã®ãƒ†ãƒ¼ãƒ–ルスã‚ーマã«ã¯ãƒãƒ¼ï¼ˆRow）ã¨ã‚«ãƒ©ãƒ ファミリー（Column Family）ãŒã‚ã‚Šã¾ã™ã€‚ãƒãƒ¼ã¯ä¸€æ„性制約をæŒã¤ã®ã§ã€ã‚ーã¨ã—ã¦ä½¿ãˆã¾ã™ã€‚ãã—ã¦ï¼‘ã¤ã¾ãŸã¯è¤‡æ•°ã‚るカラムファミリーã¯ã€ãã‚Œãžã‚Œã®ä¸ã§ã‚«ãƒ©ãƒ ã‚’ä¿æŒã—ã¦ãŠã‚Šã€ãã®ã‚«ãƒ©ãƒ ã¯ãƒ†ãƒ¼ãƒ–ルã®ä½œæˆå¾Œã‚‚å‹•çš„ã«å¢—ã‚„ã™ã“ã¨ãŒã§ãã¾ã™ã€‚ HBaseã®ãƒ†ãƒ¼ãƒ–ルã«ãŠã‘ã‚‹ãƒãƒ¼ã¯è¡Œå˜ä½ã®æ“作ãŒã‚¢ãƒˆãƒŸãƒƒã‚¯ã«è¡Œãˆã‚‹ã“ã¨ã‚’ä¿è¨¼ã—ã¦ã„ã¾ã™ã€‚よã£ã¦ã€HBaseã®ãƒ†ãƒ¼ãƒ–ルをèªè¨¼ãƒ†ãƒ¼ãƒ–ルã¨ã—ã¦ä½¿ç”¨ã™ã‚‹å ´åˆã¯ã€ãƒãƒ¼ã®å€¤ã‚’ユーザーIDã¨ã—ã¦ã€ã‚«ãƒ©ãƒ ファミリーã®ä¸ã®ã‚«ãƒ©ãƒ をユーザー属性ã¨ã—ã¦ä½¿ç”¨ã™ã‚‹ã“ã¨ã§ã€ä¸Šè¿°ã®ï¼ˆA）ãŠã‚ˆã³ï¼ˆB）ã®è¦ä»¶ã‚’満ãŸã™ã“ã¨ãŒã§ãã¾ã™ã€‚図3ã«èªè¨¼ãƒ†ãƒ¼ãƒ–ルスã‚ーマã®æ¦‚念を示ã—ã¾ã™ï¼Š2。 [*2] タイムスタンプ（Time Stamp）ã«é–¢ã—ã¦ã¯ã€æœ¬ç¨¿ã§ã¯å‰²æ„›ã—ã¾ã™ã€‚詳細ã¯The Apache HBase Boo
FacebookãŒæ–°ã‚µãƒ¼ãƒ“スã®åŸºç›¤ã«ã—ãŸã®ã¯ã€MySQLã§ã‚‚Cassandraã§ã‚‚ãªãã€HBaseã ã£ãŸ
FacebookãŒ15æ—¥ã«ç™ºè¡¨ã—ãŸæ–°ã—ã„サービス「Facebook Messagesã€ã¯ã€ãƒãƒ£ãƒƒãƒˆã‚„ã¤ã¶ã‚„ãã€ãã—ã¦é›»åメールãªã©ã€è‡ªåˆ†å®›ã®ãƒ†ã‚ストやメッセージをã™ã¹ã¦1ã¤ã®ã‚¤ãƒ³ãƒœãƒƒã‚¯ã‚¹ã§ç®¡ç†ã§ãã‚‹ã¨ç™ºè¡¨ã•ã‚Œã¾ã—ãŸã€‚ åŒç¤¾ãŒ15カ月ã‹ã‘ã¦é–‹ç™ºã—ã¦ããŸã“ã®æ–°ã‚µãƒ¼ãƒ“スã®ãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã¯ã€ã“ã‚Œã¾ã§åŒç¤¾ãŒå¤§è¦æ¨¡é‹ç”¨ã—ã¦ããŸMySQLã§ã‚‚ã€åŒç¤¾ãŒé–‹ç™ºã—ãŸNoSQLデータベースã®Cassandraã§ã‚‚ãªãã€ã‚°ãƒ¼ã‚°ãƒ«ã®BigTableをモデルã¨ã—ã¦ã‚ªãƒ¼ãƒ—ンソースã§é–‹ç™ºã•ã‚ŒãŸåˆ†æ•£ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã€ŒHBaseã€ã§ã—ãŸã€‚ Facebookã®ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã€Kannan Muthukkaruppanæ°ãŒFacebookã«ãƒã‚¹ãƒˆã—ãŸè¨˜äº‹ã€ŒThe Underlying Technology of Messagesã€ã§ã€ãã®æŠ€è¡“的背景ãŒç´¹ä»‹ã•ã‚Œã¦ã„ã¾ã™ã€‚ MySQLã¨CassandraãŒè½é¸ã—ãŸç†ç”± H
HBaseã¨ã¯ã©ã‚“ãªNoSQLデータベースãªã®ã‹ï¼Ÿ 日本語ã§èªã‚ã‚‹æƒ…å ±ã‚’é›†ã‚ã¦ã¿ãŸ
FacebookãŒæ–°ã—ã„サービス「Messagesã€ã®åŸºç›¤ã¨ã—ã¦ã€NoSQLデータベースã®ã€ŒHBaseã€ã‚’é¸æŠžã—ãŸã“ã¨ã‚’ã€å…ˆæ—¥ã®è¨˜äº‹ã€ŒFacebookãŒæ–°ã‚µãƒ¼ãƒ“スã®åŸºç›¤ã«ã—ãŸã®ã¯ã€MySQLã§ã‚‚Cassandraã§ã‚‚ãªãã€HBaseã ã£ãŸã€ã§ç´¹ä»‹ã—ã¾ã—ãŸã€‚ HBaseã¯ã€Facebookã«ã‚ˆã‚‹ã¨æ¬¡ã®ã‚ˆã†ãªç‰¹å¾´ã‚’å‚™ãˆã¦ã„ã‚‹ã¨èª¬æ˜Žã•ã‚Œã¦ã¾ã™ã€‚ è² è·ã«å¯¾ã—ã¦éžå¸¸ã«é«˜ã„スケーラビリティã¨æ€§èƒ½ã‚’ç™ºæ® CassandraよりもシンプルãªConsistency Model（一貫性モデル）を備ãˆã¦ã„ã‚‹ 自動ãƒãƒ¼ãƒ‰ãƒãƒ©ãƒ³ã‚¹ã€ãƒ•ã‚§ã‚¤ãƒ«ã‚ªãƒ¼ãƒãƒ¼ã€åœ§ç¸®æ©Ÿèƒ½ サーãƒãƒ¼ã”ã¨ã«æ•°å個ã®ã‚·ãƒ£ãƒ¼ãƒ‰ã‚’割り当ã¦å¯èƒ½ã€ãªã©ãªã© ã“ã®HBaseã¯ã©ã®ã‚ˆã†ãªãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ãªã®ã§ã—ょã†ã‹ï¼Ÿ æƒ…å ±ã‚’é›†ã‚ã¦ã¿ã¾ã—ãŸã€‚ HBase入門ã®ãƒ—レゼンテーション 最åˆã«ç´¹ä»‹ã™ã‚‹ã®ã¯ã€ŒHBaseエãƒãƒ³ã‚¸ã‚§ãƒªã‚¹ãƒˆã€Tatsuya Kawanoæ°ã®ãƒ—レゼン
Apache HBase 入門 (第１回)
The document discusses big data and Hadoop concepts. It covers Hadoop operations like put, get, scan, filter, delete as well as join and group by. It also discusses the different types of data access patterns like random write, sequential read, sequential write and random read. The document focuses on big data, Hadoop operations, and data access patterns.Read less
"Hbase at Facebook" ã«è¡Œã£ã¦ã㟠- ãŸã”ã‚‚ã‚Šã™ãƒ¡ãƒ¢
å称表記ãŒæºã‚Œã¦ã¦å¾®å¦™ã ã‘ã© Hbase at FaceBook on Zusaar ã“ã®ã‚¤ãƒ™ãƒ³ãƒˆã«è¡Œã£ã¦ããŸã€‚Facebookã®äººã¯ "HBase Tokyo meetup" ã¨èªè˜ã—ã¦ã„ãŸã‚ˆã†ã 。 内容ã®ã¾ã¨ã‚ã¯ã‚„らãªã„ã®ã§ã€ä»¥ä¸‹ã®å„ページãªã©ã‚’ã”覧ã«ãªã‚‹ã¨è‰¯ã„ã®ã§ã¯ãªã„ã§ã—ょã†ã‹ã€‚ Tokyo HBase Meetup - Realtime Big Data at Facebook with Hadoop and HB… Hbase at FaceBookã®ã¾ã¨ã‚ - Togetterã¾ã¨ã‚ FacebookãŒHBaseを大è¦æ¨¡ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ 処ç†ã«åˆ©ç”¨ã—ã¦ã„ã‚‹ç†ç”±ï¼ˆå‰ç·¨ï¼‰ ï¼ Publickey FacebookãŒHBaseを大è¦æ¨¡ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ 処ç†ã«åˆ©ç”¨ã—ã¦ã„ã‚‹ç†ç”±ï¼ˆå¾Œç·¨ï¼‰ ï¼ Publickey セッションã®å†…容ã¨è‡ªåˆ†ãŒè€ƒãˆãŸã“ã¨ã¨äººã¨ã—ゃã¹ã£ãŸã“ã¨ã‚’ã„ã£ã—ょããŸã«ã“ã“ã«æ›¸ã„ã¦ãŠã。
FacebookãŒHBaseを大è¦æ¨¡ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ 処ç†ã«åˆ©ç”¨ã—ã¦ã„ã‚‹ç†ç”±ï¼ˆå‰ç·¨ï¼‰
Facebookã¯å¤§è¦æ¨¡ãªãƒ‡ãƒ¼ã‚¿å‡¦ç†ã®åŸºç›¤ã¨ã—ã¦HBaseを利用ã—ã¦ã„ã¾ã™ã€‚ãªãœFacebookã¯HBaseを用ã„ã¦ã„ã‚‹ã®ã‹ã€ã©ã®ã‚ˆã†ã«åˆ©ç”¨ã—ã¦ã„ã‚‹ã®ã§ã—ょã†ã‹ï¼Ÿ 7月1æ—¥ã«éƒ½å†…ã§è¡Œã‚ã‚ŒãŸå‹‰å¼·ä¼šã§ã€Facebookã®ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã§ã‚るジョナサン・グレイ（Jonathan Gray）æ°ã«ã‚ˆã‚‹è§£èª¬ãŒè¡Œã‚ã‚Œã¾ã—ãŸã€‚ 解説ã¯ã»ã¼ã‚¹ãƒ©ã‚¤ãƒ‰ã®å†…容ãã®ã¾ã¾ã§ã—ãŸã€‚当日使ã‚ã‚ŒãŸæ—¥æœ¬èªžè¨³ã•ã‚ŒãŸã‚¹ãƒ©ã‚¤ãƒ‰ãŒå…¬é–‹ã•ã‚Œã¦ã„ã‚‹ã®ã§ã€ãƒã‚¤ãƒ³ãƒˆã¨ãªã‚‹ãƒšãƒ¼ã‚¸ã‚’紹介ã—ã¾ã—ょã†ã€‚ Realtime Apache Hadoop at Facebook ãªãœãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ データã®åˆ†æžã«ã€Hadoop/HBaseを使ã†ã®ã‹ï¼Ÿ MySQLã¯å®‰å®šã—ã¦ã„ã‚‹ãŒã€åˆ†æ•£ã‚·ã‚¹ãƒ†ãƒ ã¨ã—ã¦è¨è¨ˆã•ã‚Œã¦ãŠã‚‰ãšã€ã‚µã‚¤ã‚ºã«ã‚‚上é™ãŒã‚る。一方ã€Hadoopã¯ã‚¹ã‚±ãƒ¼ãƒ©ãƒ–ルã ãŒãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ãŒé›£ã—ãã€ãƒ©ãƒ³ãƒ€ãƒ ãªæ›¸ãè¾¼ã¿ã‚„èªã¿è¾¼ã¿ã«å‘ã„ã¦ã„ãªã„。 Faceb
Hbase at FaceBookã®ã¾ã¨ã‚
Tatsuya Kawano @tatsuya6502 今晩ã®ã€ŒHBase at Facebookã€è¬›æ¼”ã®ã‚¹ãƒ©ã‚¤ãƒ‰è‹±æ—¥ç¿»è¨³ãŒå®Œäº†ï¼ã‚¢ãƒ‹ãƒ¡ãƒ¼ã‚·ãƒ§ãƒ³ãŒæ²¢å±±å…¥ã£ã¦ã¾ã™ã。HBaseã®åŸºæœ¬å‹•ä½œãŒã‚ã‹ã‚Šã‚„ã™ã„ã€‚ã‚¤ãƒ™ãƒ³ãƒˆæƒ…å ±ï¼š http://t.co/jMyNWbG #hbasefacebook 2011-07-01 06:36:43
Hadoopã€hBaseã§æ§‹ç¯‰ã™ã‚‹å¤§è¦æ¨¡åˆ†æ•£ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ
CodeZine編集部ã§ã¯ã€ç¾å ´ã§æ´»èºã™ã‚‹ãƒ‡ãƒ™ãƒãƒƒãƒ‘ーをスターã«ã™ã‚‹ãŸã‚ã®ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã€ŒDevelopers Summitã€ã‚„ã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®ç”Ÿãã–ã¾ã‚’ブーストã™ã‚‹ãŸã‚ã®ã‚¤ãƒ™ãƒ³ãƒˆã€ŒDevelopers Boostã€ãªã©ã€ã•ã¾ã–ã¾ãªã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã‚’ä¼ç”»ãƒ»é‹å–¶ã—ã¦ã„ã¾ã™ã€‚
HBase - HADOOP2 - Apache Software Foundation
HBase: Bigtable-like structured storage for Hadoop HDFS HBase has an awesome community resource in our online book. It's maintained and looked after with diligence. In contrast, the below Wiki content has not been updated since 2012. For that reason, the Wiki has been deprecated (see HBASE-14481). Please update your links. You can find historical pages (which have been updated to give the new link
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
{{#tags}}- {{label}}
{{/tags}}