GitHub - datawrangling/trendingtopics: Rails app for tracking trends in server logs - powered by the Cloudera Hadoop Distribution on EC2You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Cascading
Please note that all new project news and releases have moved to https://cascading.wensel.net The Cascading Ecosystem is a collection of applications, languages, and APIs for developing data-intensive applications. At the ecosystem core is Cascading, a Java API for defining complex data flows and integrating those flows with back-end systems, and a query planner for mapping and executing logical f
優良ä¼æ¥ã¯ãªãœHadoopã«èµ°ã‚‹ã®ã‹
ã¡ãªã¿ã«ã€ã“ã®åˆ†æžã®ãŸã‚ã«å¿…è¦ã¨ã•ã‚Œã‚‹MapReduceã®ã‚³ãƒ¼ãƒ‰ã§ã‚ã‚‹ãŒã€ãã®ã‚µã‚¤ã‚ºã¯ã‚ãšã‹20ステップã ã¨ã„ã†ã€‚Yahoo!ã®ãƒ—レゼンテーターã§ã‚ã‚‹ã€ã‚¨ãƒªãƒƒã‚¯ãƒ»ãƒãƒ«ãƒ‡ã‚·ãƒ¥ãƒã‚¤ãƒ©ãƒ¼æ°ã«ã‚ˆã‚‹ã¨ã€ãŸã¨ãˆçµŒé¨“ã®æµ…ã„エンジニアã§ã‚ã£ã¦ã‚‚ã€MapReduceã«ã‚ˆã‚‹ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã¯å¯èƒ½ã§ã‚ã‚‹ã¨ã•ã‚Œã‚‹ã€‚ ã¾ãŸã€VISAã®ã‚¸ãƒ§ãƒ¼ãƒ»ã‚«ãƒ‹ãƒ³ã‚¬ãƒ æ°ã‹ã‚‰ã‚‚ã€è²´é‡ãªãƒ‡ãƒ¼ã‚¿ãŒæä¾›ã•ã‚Œã¦ã„ãŸã®ã§ä»¥ä¸‹ã«ç´¹ä»‹ã™ã‚‹ã€‚åŒç¤¾ã§ã¯ã€1æ—¥ã«1億トランザクションãŒç™ºç”Ÿã™ã‚‹ãŸã‚ã€2å¹´é–“ã§700å„„å¼·ã®ãƒˆãƒ©ãƒ³ã‚¶ã‚¯ã‚·ãƒ§ãƒ³ãƒã‚°ãŒè“„ç©ã•ã‚Œã€ãã®ãƒ‡ãƒ¼ã‚¿é‡ã¯36テラãƒã‚¤ãƒˆã«è‡³ã‚‹ã¨ã„ã†ã€‚ã“ã†ã—ãŸã‚¹ã‚±ãƒ¼ãƒ«ã®ãƒ‡ãƒ¼ã‚¿ã‚’ã€å¾“æ¥ã®RDBを用ã„ã¦åˆ†æžã™ã‚‹ã«ã¯ã€ç´„1カ月ã®æ™‚é–“ãŒå¿…è¦ã¨ã•ã‚Œã¦ããŸãŒã€Hadoopを用ã„ã‚‹ã“ã¨ã§13分ã«çŸç¸®ã•ã‚ŒãŸã¨ã„ã†ã€‚ ã“ã‚Œã¾ã§ã¯ã€Yahoo!ã«ã—ã‚VISAã«ã—ã‚ã€è†¨å¤§ãªãƒ‡ãƒ¼ã‚¿ã‚’RDBã«æŠ¼ã—込むã»ã‹ã«æ–¹æ³•ã¯ãªãã€ãã®åˆ†æžã«æ•°å日をè¦ã™ã‚‹
Disco MapReduce
Disco is a lightweight, open-source framework for distributed computing based on the MapReduce paradigm. Disco is powerful and easy to use, thanks to Python. Disco distributes and replicates your data, and schedules your jobs efficiently. Disco even includes the tools you need to index billions of data points and query them in real-time. Disco was born in Nokia Research Center in 2008 to solve rea
Silicon Soul » HadoopDBã®ã‚¢ãƒ¼ã‚テクãƒãƒ£
â– HadoopDBã®ã‚¢ãƒ¼ã‚テクãƒãƒ£ã«ã¤ã„㦠HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads. Azza Abouzeid, Kamil Bajda-Pawlikowski, Daniel J. Abadi, Avi Silberschatz, Alex Rasin. In Proceedings of VLDB, 2009. より〠HadoopDBã®ã‚¢ãƒ¼ã‚テクãƒãƒ£ã«é–¢ã™ã‚‹ç« ã‹ã‚‰ã€Hadoopã«è¿½åŠ ã•ã‚ŒãŸï¼”ã¤ã®ã‚³ãƒ³ãƒãƒ¼ãƒãƒ³ãƒˆã«ã¤ã„ã¦é †ã«èªã‚“ã§ãã¾ã™ã€‚ â–¼Database Connector Database Connectorã¯ã€ã‚¯ãƒ©ã‚¹ã‚¿ã®å„ノードã«ã‚る個別ã®ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã¨TaskTrackerã®é–“ã®ã‚¤ãƒ³ã‚¿ãƒ•ã‚§ãƒ¼ã‚¹ã§ã€ Hadoopã®InputFo
Hadoop＋Hive検証環境を構築ã—ã¦ã¿ã‚‹
Hadoop＋Hive検証環境を構築ã—ã¦ã¿ã‚‹ï¼šHive――RDB使ã„ã®ãŸã‚ã®Hadoopガイド（å‰ç·¨ï¼‰ï¼ˆ1/3 ページ） Hadoop Hiveã¯Hadoop上ã§SQLライクãªã‚¯ã‚¨ãƒªæ“作ãŒå¯èƒ½ãªDWHå‘ã‘ã®ãƒ—ãƒãƒ€ã‚¯ãƒˆã§ã™ã€‚SQLã«è¿‘ã„æ“作ãŒå¯èƒ½ãªãŸã‚ã€HBaseよりもデータベースã«æ…£ã‚Œè¦ªã—ã‚“ã ã¿ãªã•ã‚“ã«ã¯ä½¿ã„å‹æ‰‹ãŒã„ã„ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“。本稿ã§ã¯ã“ã®Hiveã®ä½¿ã„æ–¹ã¨ãƒ¬ãƒ“ューを行ã£ã¦ã„ãã¾ã™ã€‚
Hadoopã®è§£æžè³‡æ–™
オープンソース分散システム「Hadoopã€ã«é–¢ã™ã‚‹è§£æžè³‡æ–™ã‚’公開ã•ã›ã¦é ‚ã„ã¦ãŠã‚Šã¾ã™ã€‚ã“ã®èª¿æŸ»ã¯NTTãƒ¬ã‚¾ãƒŠãƒ³ãƒˆæ ªå¼ä¼šç¤¾æ§˜ã¨å…±åŒã§è¡Œã„ã¾ã—ãŸ(プレスリリース)。 Hadoop解æžè³‡æ–™(PDF), 最終更新: 2008/08/25, 公開: 2008/08/25 Hadoopã®å®Ÿéš›ã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«æ–¹æ³•ãªã©ã«ã¤ãã¾ã—ã¦ã¯ã€å¼Šç¤¾å¤ªç”°ã«ã‚ˆã‚‹ä»¥ä¸‹ã®è¨˜äº‹ã‚‚ã”å‚考下ã•ã„。 Hadoopã€hBaseã§æ§‹ç¯‰ã™ã‚‹å¤§è¦æ¨¡åˆ†æ•£ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ Hadoopã®ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã¨ã‚µãƒ³ãƒ—ルプãƒã‚°ãƒ©ãƒ ã®å®Ÿè¡Œ 複数マシンã¸Hadoopをインストールã™ã‚‹
MapReduce - naoyaã®ã¯ã¦ãªãƒ€ã‚¤ã‚¢ãƒªãƒ¼
"MapReduce" 㯠Google ã®ãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰ã§åˆ©ç”¨ã•ã‚Œã¦ã„る並列計算システムã§ã™ã€‚検索エンジンã®ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ä½œæˆã‚’ã¯ã˜ã‚ã¨ã™ã‚‹ã€å¤§è¦æ¨¡ãªå…¥åŠ›ãƒ‡ãƒ¼ã‚¿ã«å¯¾ã™ã‚‹ãƒãƒƒãƒå‡¦ç†ã‚’想定ã—ã¦ä½œã‚‰ã‚ŒãŸã‚·ã‚¹ãƒ†ãƒ ã§ã™ã€‚ MapReduce ã®é¢ç™½ã„ã¨ã“ã‚ã¯ã€map() 㨠reduce() ã¨ã„ã†äºŒã¤ã®é–¢æ•°ã®çµ„ã¿åˆã‚ã›ã‚’定義ã™ã‚‹ã ã‘ã§ã€å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ã«å¯¾ã™ã‚‹æ§˜ã€…ãªè¨ˆç®—å•é¡Œã‚’解決ã™ã‚‹ã“ã¨ãŒã§ãる点ã§ã™ã€‚ MapReduce ã®è¨ˆç®—モデル map() ã«ã¯ãã®è¨ˆç®—å•é¡Œã®ãƒ‡ãƒ¼ã‚¿ã¨ã—ã¦ã® key-value ペアãŒæ¬¡ã€…ã«æ¸¡ã£ã¦ãã¾ã™ã€‚map() ã§ã¯ key-value 値ã®ãƒšã‚¢ã‚’ç•°ãªã‚‹è¤‡æ•°ã® key-value ペアã«å¤‰æ›ã—ã¾ã™ã€‚reduce() ã«ã¯ã€map() ã§ä½œã£ãŸ key-value ペアをåŒä¸€ã® key ã§æŸããŸã‚‚ã®ãŒé †ç•ªã«æ¸¡ã£ã¦ãã¾ã™ã€‚ãã® key-values ペアを任æ„ã®å½¢å¼ã«å¤‰æ›ã™ã‚‹ã“ã¨
Apache Hadoop
Apache Hadoop The Apache® Hadoop® project develops open-source software for reliable, scalable, distributed computing. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation an
En Yüksek Kur Paykasa Bozdurma
IBM TechXchange Community Join the Community and get 30% off the TechXchange Conference. Join / Log in Where is my content? If you’re looking for developerWorks content or a Support forum and ended up here, don't panic! You are in the right place. The content you're looking for. This page will help you find the content you are looking for, get answers to your questions, and find a new community to
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
mrtoolkit - Google Code
{{#tags}}- {{label}}
{{/tags}}