デブサミ2014-Stormã§å®Ÿç¾ã™ã‚‹ãƒ“ッグデータã®ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ 処ç†ãƒ—ラットフォーム～ストリームデータ処ç†ã‹ã‚‰æ©Ÿæ¢°å¦ç¿’ã¾ã§ï½ž
2. 自己紹介 { “æ°å†“所属†“Twitter†“Facebook†“分野†: : : : : â€éˆ´æœ¨ 貴典†“Acroquest Technology Co., Ltd.†“@takanorig†“https://www.facebook.com/takanorig†[“SEPGâ€, “Testingâ€, “Managementâ€, â€Frameworkâ€, â€Platformâ€] } Copyright © Acroquest Technology Co., Ltd. All rights reserved. 2
日々進化ã™ã‚‹Hadoop。ã“ã‚Œã¾ã§ã®ãŠã•ã‚‰ã„ã¨æœ€è¿‘ã®å‹•å‘（å‰ç·¨ï¼‰
大è¦æ¨¡ãªåˆ†æ•£å‡¦ç†ãƒ•ãƒ¬ãƒ¼ãƒ ワークã¨ã—ã¦HadoopãŒç™»å ´ã—ãŸã“ã¨ã«ã‚ˆã‚Šã€ãƒ“ッグデータã®ãƒ–ームやã€å¤§è¦æ¨¡ãªã‚½ãƒ¼ã‚·ãƒ£ãƒ«ã‚²ãƒ¼ãƒ ã§ã®ãƒã‚°è§£æžã«ã‚ˆã‚‹æ”¹å–„ã€ã‚³ãƒžãƒ¼ã‚¹ã‚µã‚¤ãƒˆã§ã®æ©Ÿæ¢°å¦ç¿’ã«ã‚ˆã‚‹ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ãªã©ã€å¤šãã®å¤‰åŒ–ãŒå¼•ãèµ·ã“ã•ã‚Œã¦ãã¾ã—ãŸã€‚ ãã—ã¦ãã®Hadoop自体もã€æ—¥ã€…進化ã—続ã‘ã¦ã„ã¾ã™ã€‚ Hadoopã¨ã¯ã©ã†ã„ã†ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã§ã‚ã‚Šã€ã„ã¾ã©ã®ã‚ˆã†ãªçŠ¶æ³ã«ãªã£ã¦ã„ã‚‹ã®ã‹ã€‚NTTデータã®æ¿±é‡Žè³¢ä¸€æœ—æ°ãŒã€å…ˆæ—¥è¡Œã‚ã‚ŒãŸç¬¬2回 NHNテクノãƒã‚¸ãƒ¼ã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã§è¡Œã£ãŸã‚»ãƒƒã‚·ãƒ§ãƒ³ã€Œæ—¥ã€…進化ã™ã‚‹Hadoopã®ã€Žã„ã¾ã€ã€ã§åˆ†ã‹ã‚Šã‚„ã™ã解説ã—ã¦ã„ã¾ã™ã€‚ ã“ã®è¨˜äº‹ã§ã¯ãã®ã‚»ãƒƒã‚·ãƒ§ãƒ³ã®å†…容をダイジェストã§ç´¹ä»‹ã—ã¾ã—ょã†ã€‚ 日々進化ã™ã‚‹Hadoopã®ã€Œã„ã¾ã€ NTTデータ 基盤システム事æ¥éƒ¨ 濱野賢一朗æ°ã€‚ NTTデータã¨ã„ã†ã¨ã“ã‚ã§ä»•äº‹ã‚’ã—ã¦ã„ã¾ã™ã€‚NTTデータ自体ã¯ã‚‚ã†5å¹´ãらã„Hadoopã‚’ã‚„ã£ã¦ã¾ã—ã¦ã€ã
Cloudera Blog
Enterprise IT leaders across industries are tasked with preparing their organizations for the technologies of the future – which is no simple task. With the use of AI exploding, Cloudera, in partnership with Researchscape, surveyed 600 IT leaders who work at companies with over 1,000 employees in the U.S., EMEA and APAC regions. The survey, […] Read blog post
Apache Hadoop 1.0.0㌠Kerberos èªè¨¼, Apache HBaseã€HDFSã¸ã®RESTful APIをサãƒãƒ¼ãƒˆ
Spring Bootã«ã‚ˆã‚‹APIãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰æ§‹ç¯‰å®Ÿè·µã‚¬ã‚¤ãƒ‰ 第2版 何åƒäººã‚‚ã®é–‹ç™ºè€…ãŒã€InfoQã®ãƒŸãƒ‹ãƒ–ック「Practical Guide to Building an API Back End with Spring Bootã€ã‹ã‚‰ã€Spring Bootを使ã£ãŸREST API構築ã®åŸºç¤Žã‚’å¦ã‚“ã 。ã“ã®æœ¬ã§ã¯ã€å‡ºç‰ˆæ™‚ã«æ–°ã—ãリリースã•ã‚ŒãŸãƒãƒ¼ã‚¸ãƒ§ãƒ³ã§ã‚ã‚‹ Spring Boot 2 を使用ã—ã¦ã„る。ã—ã‹ã—ã€Spring Boot3ãŒæœ€è¿‘リリースã•ã‚Œã€é‡è¦ãªå¤‰...
インフラエンジニアã®ãŸã‚ã®Hadoopæƒ…å ±ã€€Gangliaãã®ï¼‘：So-net Developer Blog：So-netブãƒã‚°
ã“ã‚Œã¾ã§ã¯Hadoopã®æ»æ´»ç›£è¦–ãŒç›®çš„ã®Nagiosè¨å®šã‚’見ã¦ãã¾ã—ãŸãŒã€Hadoopã¯JMX を通ã—ã¦ãƒ¡ãƒ¢ãƒªä½¿ç”¨çŠ¶æ³ã€ã‚¸ãƒ§ãƒ–（MapReduce)ã®é€²è¡ŒçŠ¶æ³ã‚’詳細ã«è¦‹ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ ã“ã®JMXã®å€¤ã‚’ã€Gangliaã¨ã„ã†ã‚°ãƒ©ãƒ•è¡¨ç¤ºã®ç›£è¦–ツールを使ã£ã¦å¯è¦–化ã—ã¾ã—ょã†ã€‚ ジョブã®æŠ•å…¥çŠ¶æ³ã«åˆã‚ã›ã¦ã€ãƒ¡ãƒ¢ãƒªã‚„CPUã®åˆ©ç”¨çŽ‡ãŒå„ノード毎ã«ã‚°ãƒ©ãƒ•åŒ–ã•ã‚Œã¦ 見ãˆã‚‹ã®ã§ã€ãƒœãƒˆãƒ«ãƒãƒƒã‚¯ã®ç™ºè¦‹ã«å½¹ç«‹ã¤ã¯ãšã§ã™ã€‚ Gangliaã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã¯çµ‚ã‚ã£ã¦ã„ã‚‹ã‚‚ã®ã¨ã—ã¦ãƒ»ãƒ»ãƒ»ã¨ã—ãŸã„ã¨ã“ã‚ã§ã™ãŒ Nagiosã¨é•ã£ã¦ã€Hadoopã®å„ノードå´å…¨ã¦ã«ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã—ãªã‘ã‚Œã°ã„ã‘ãªã„ã®ã§ ã¡ã‚‡ã£ã¨é¢å€’。 rpmパッケージを作ã£ã¦ã€å„ノードã«é…布ã—ã¡ã‚ƒã„ã¾ã—ょã†ã€‚ ã¾ãšã¯ã€Ganglia監視サーãƒå´ã‚’作æˆã€‚ rpm化ã«ã¯checkinstallを使ã„ã¾ã™ã€‚ã‚らã‹ã˜ã‚インストールã—ã¦ãŠãã¾ã™ã€‚ ビルドã«å¿…è¦ãªã‚‚ã®ã‚’
GangliaMetrics - HADOOP2 - Apache Software Foundation
Edit the file hadoop-metrics.properties, if it is not already present in your config directory. The file should contain at least the following lines: dfs.class=org.apache.hadoop.metrics.ganglia.GangliaContext dfs.period=10 dfs.servers=<at:var at:name="GANGLIA" />:8649 mapred.class=org.apache.hadoop.metrics.ganglia.GangliaContext mapred.period=10 mapred.servers=<at:var at:name="GANGLIA" />:8649 jvm
HBase Javaメモ(Hishidama's HBase Java Memo)
S-JIS[2010-02-20/2012-04-28] å¤‰æ›´å±¥æ´ HBaseã«Javaã‹ã‚‰ã‚¢ã‚¯ã‚»ã‚¹ HBaseã«Javaã‹ã‚‰ã‚¢ã‚¯ã‚»ã‚¹ã™ã‚‹ä¾‹ã€‚ ライブラリー&åˆæœŸè¨å®š&基本サンプル HBase0.20 [2010-02-20] HBase0.89 [2010-07-11] コーディング HTable [2010-07-11] データæ“作 [/2010-07-15] テーブルæ“作 [2010-02-27] トランザクション [/2010-05-17] 一æ„制約付ã登録 [2010-07-13]
分散データベース「HBaseã€ã®å®‰å®šé‹ç”¨ã‚’目指ã—㦠- Preferred Networks Research & Development
1年経ã£ã¦iPhone4ã®é›»æ± ãŒãƒ˜ã‚¿ã£ã¦ããŸã€å¤ªç”°ã§ã™ã€‚ 指数関数的ã«ã‚¨ãƒ³ãƒˆãƒªæ•°ãŒå°‘ãªããªã£ã¦ããŸæœ¬ãƒ–ãƒã‚°ã§ã™ãŒwã€æ™¯æ°—付ã‘ã®ãŸã‚ã«ã‚¨ãƒ³ãƒˆãƒªã‚’投稿ã—ãŸã„ã¨æ€ã„ã¾ã™ï¼æœ¬æ—¥ã¯HBaseã«ã¤ã„ã¦ã§ã™ã€‚ Linux 㨠Hadoop 㨠HBase 㨠ZooKeeper ã«è©³ã—ã„ã‚ãªãŸï¼ã‚ãªãŸãŒã‚¿ãƒ¼ã‚²ãƒƒãƒˆã§ã™ã€‚ HBaseã¨ã¯? HBaseã¨ã¯ã€HDFS (Hadoop Distributed File System)上ã«æ§‹ç¯‰ã•ã‚ŒãŸåˆ†æ•£ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã§ã™ã€‚大é‡ã®éžå¸¸ã«ç´°ã‹ã„データをリアルタイムã«èªã¿æ›¸ã出æ¥ã‚‹ã®ãŒç‰¹å¾´ã§ã™ã€‚最近ã§ã¯Facebook Messageã®åŸºç›¤æŠ€è¡“ã¨ã—ã¦ä½¿ç”¨ã•ã‚ŒãŸäº‹ã§æ³¨ç›®ã‚’集ã‚ã¦ã„ã¾ã™ã€‚ HBaseå…¬å¼ã‚µã‚¤ãƒˆ Apache HBase ブック ä¿å˜ã•ã‚ŒãŸãƒ‡ãƒ¼ã‚¿ã¯HDFS上ã«ä¿å˜ã•ã‚Œã€HDFSã®ä»•çµ„ã¿ã«ã‚ˆã£ã¦ãƒ¬ãƒ—リケーションã•ã‚Œã‚‹ãŸã‚安全ã«ãƒ‡ãƒ¼ã‚¿ã‚’ä¿æŒã™ã‚‹ã“ã¨ãŒå‡ºæ¥ã¾ã™ã€‚ ã¾
HBase Shellメモ(Hishidama's HBase0.89 Shell Memo)
S-JIS[2010-07-07/2010-07-12] å¤‰æ›´å±¥æ´ HBase Shell（ãƒãƒ¼ã‚¸ãƒ§ãƒ³0.89） HBase Shellã¯ã€HBaseã‚’æ“作ã™ã‚‹å¯¾è©±åž‹ãƒ„ール。 （Oracleã®SQL*Plusã€JavaDBã®ijã€PostgreSQLã®psqlã®ã‚ˆã†ãªã‚‚ã®ï¼‰ HBase0.89ã§ã€HBase0.20ã‹ã‚‰æ©Ÿèƒ½ãŒå¢—å¼·ã•ã‚ŒãŸã€‚ HBase Shellã®èµ·å‹• （Windowsã®å ´åˆã€Cygwin（bash）ã‹ã‚‰ï¼‰ä»¥ä¸‹ã®ã‚³ãƒžãƒ³ãƒ‰ã§HBase Sehllã‚’èµ·å‹•ã™ã‚‹ã€‚ $ /usr/local/hbase-0.89/bin/hbase shell --helpを付ã‘ã‚‹ã¨ã‚³ãƒžãƒ³ãƒ‰ã®ã‚ªãƒ—ションãŒè¡¨ç¤ºã•ã‚Œã‚‹ã€‚ $ /usr/local/hbase-0.89/bin/hbase shell --help HBase Shell command-line options: format Forma
Liberty On Hbase 20091113
4. Liberty – æ—¢å˜ã‚µãƒ¼ãƒ“スã‹ã‚‰æ–°ãŸãªã‚µãƒ¼ãƒ“スを ウェブ ä½æ‰€éŒ² メール 検索 æ—¢å˜ã‚µãƒ¼ãƒ“ス Contact List Web Logging Mail Data Location Log Libertyスケーラブルインデクシングサービス(Liberty Scalable Indexing Service) Document Abstraction Service Indexing Sub-System Index Access Subsystem Java / C++ API æ–°è¦ã‚µãƒ¼ãƒ“ス アクセス グループ è§£æž ã‚¦ã‚§ã‚¢ CONFIDENTIAL © 2009 Telanium Technology Incorporated 4 5. Liberty – BDB（SleepyCat）を分散é…ç½®ã™ã‚‹ ï® æ±Žç”¨ã‚µãƒ¼ãƒã‚’æ°´å¹³é…ç½®ã™ã‚‹ã“㨠ã§ã‚¹ã‚±ãƒ¼ãƒ©ãƒ“リティを実ç¾ã™ã‚‹ Mail Bo
GitHub - hbase-trx/hbase-transactional-tableindexed: Transactional and indexing extensions for hbase
This project is a transactional and indexing extension for hbase. Current status: Now working with HBase 0.90. Working Features: * Ability to create manage and query with pre-defined table indexes. * Ability to perform multiple HBase operations within serialized and atomic JTA transactions. Known limitations and issues: https://github.com/hbase-trx/hbase-transactional-tableindexed/issues Installat
Cloudera Blog
We are excited to announce the acquisition of Octopai, a leading data lineage and catalog platform that provides data discovery and governance for enterprises to enhance their data-driven decision making. Cloudera’s mission since its inception has been to empower organizations to transform all their data to deliver trusted, valuable, and predictive insights. With AI and […] Read blog post
Cloudera Blog
In an era where artificial intelligence (AI) is reshaping enterprises across the globe—be it in healthcare, finance, or manufacturing—it’s hard to overstate the transformation that AI has had on businesses, regardless of industry or size. At Cloudera, we recognize the urgent need for bold steps to harness this potential and dramatically accelerate the time to […] Read blog post
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
http://storm-project.net/
Charming Python: Functional programming in Python, Part 3
Charming Python: Functional programming in Python, Part 3
Charming Python: Functional programming in Python, Part 3
http://www.ryangreenhall.com/2010/10/monitoring-hadoop-clusters-using-ganglia/
Google Code Archive - Long-term storage for Google Code Project Hosting.
NoSQL with Hadoop and HBase
{{#tags}}- {{label}}
{{/tags}}