Apache Spark and the Typesafe Reactive Platform: A Match Made in Heaven | @lightbend
I started my first Hadoop-based, Big Data projects four years ago. As an experienced Enterprise and Internet developer with a love of Scala, it was frustrating to find that writing Hadoop jobs was hard to do in the tedious, low-level MapReduce API. The powerful, functional programming operations we know and love in Scala’s collections were absent. The object orientation of Java at the time (before
Falcon - Data Management Platform on Hadoop (Beyond ETL)
Hadoop and its ecosystem of products have made storing and processing massive amounts of data common place. This has enabled numerous businesses to gain valuable foresights that they never could have in the past. While it is easy to leverage Hadoop for crunching large volumes of data, organizing data, managing life cycle of data and processing data is fairly involved. This is solved adequately wel
A Reference Architecture for ETL 2.0
More and more organizations are moving their ETL workloads to a Hadoop based ELT grid architecture. Hadoop`s inherit capabilities, especially it`s ability to do late binding addresses some of the key challenges with traditional ETL platforms. In this presentation, attendees will learn the key factors, considerations and lessons around ETL for Hadoop. Areas such as pros and cons for different extra
毎秒10億件以上ã®ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ イベントを処ç†ã™ã‚‹DataTorrent 1.0
Spring Bootã«ã‚ˆã‚‹APIãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰æ§‹ç¯‰å®Ÿè·µã‚¬ã‚¤ãƒ‰ 第2版 何åƒäººã‚‚ã®é–‹ç™ºè€…ãŒã€InfoQã®ãƒŸãƒ‹ãƒ–ック「Practical Guide to Building an API Back End with Spring Bootã€ã‹ã‚‰ã€Spring Bootを使ã£ãŸREST API構築ã®åŸºç¤Žã‚’å¦ã‚“ã 。ã“ã®æœ¬ã§ã¯ã€å‡ºç‰ˆæ™‚ã«æ–°ã—ãリリースã•ã‚ŒãŸãƒãƒ¼ã‚¸ãƒ§ãƒ³ã§ã‚ã‚‹ Spring Boot 2 を使用ã—ã¦ã„る。ã—ã‹ã—ã€Spring Boot3ãŒæœ€è¿‘リリースã•ã‚Œã€é‡è¦ãªå¤‰...
Hadoop YARNã¨Apache Mesosã®é•ã„ã£ã¦ä½•ï¼Ÿ - 夢ã¨ã‚¬ãƒ©ã‚¯ã‚¿ã®é›†ç©å ´
ã“ã‚“ã«ã¡ã¯ã€‚ CDH上ã§SparkãŒã‚µãƒãƒ¼ãƒˆã•ã‚Œã‚‹ã¨ã„ã†ç™ºè¡¨ã‚‚ã‚ã‚Šã€ãƒ‹ãƒƒãƒãªé ˜åŸŸã‚’ã¡ã‚‡ã“ã¡ã‚‡ã“調ã¹ã¦ã„ãŸã¯ãšãŒã€ ã„ããªã‚ŠSparkãŒãƒ¡ã‚¸ãƒ£ãƒ¼ãªã‚¹ãƒ†ãƒ¼ã‚¸ã«é£›ã³å‡ºã™ã®ã‹ãªã・・ã¨æ¥½ã—ã¿ã«ã—ã¦ã„る今日ã“ã®é ƒã§ã™ã€‚ ãŸã ã€CDH上ã§ã®Sparkã¯ãƒªã‚½ãƒ¼ã‚¹ãƒžãƒãƒ¼ã‚¸ãƒ£ã¨ã—ã¦Hadoop YARNを使ã†æ¨¡æ§˜ã€‚ Apache Mesosã¯ä½¿ã‚ã‚Œãªã„よã†ã§ã™ã€‚ ã¨ã¯ã„ãˆã€ãれ以å‰ã®å•é¡Œã¨ã—ã¦ã€Hadoop YARNã¨Apache Mesosã®é•ã„ãŒãã‚‚ãもよãã‚ã‹ã£ã¦ã„ãªã„ã¨ã„ã†ç¾çŠ¶ãŒã‚ã‚Šã¾ã™ã€‚ ãã®ãŸã‚ã€ã„ã„æƒ…å ±ãŒç„¡ã„ã‹ãªã・・・ã¨èª¿ã¹ã¦ã„ãŸã¨ã“ã‚ã€ä¸åº¦ä»¥ä¸‹ã®ã‚¹ãƒ¬ãƒƒãƒ‰ãŒè¦‹ã¤ã‹ã‚Šã¾ã—ãŸã€‚ How does YARN compare to Mesos? http://www.quora.com/How-does-YARN-compare-to-Mesos ã¨ã‚Šã‚ãˆãšã€ã“ã®ã‚¹ãƒ¬ãƒƒãƒ‰ã«æ›¸ã‹ã‚Œã¦ã„る「差分ã€ã‚’訳ã—
オープンソースã®SQL-in-Hadoopソリューション:我々ã¯ã„ã¾ã©ã“ã«ï¼Ÿ
Spring Bootã«ã‚ˆã‚‹APIãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰æ§‹ç¯‰å®Ÿè·µã‚¬ã‚¤ãƒ‰ 第2版 何åƒäººã‚‚ã®é–‹ç™ºè€…ãŒã€InfoQã®ãƒŸãƒ‹ãƒ–ック「Practical Guide to Building an API Back End with Spring Bootã€ã‹ã‚‰ã€Spring Bootを使ã£ãŸREST API構築ã®åŸºç¤Žã‚’å¦ã‚“ã 。ã“ã®æœ¬ã§ã¯ã€å‡ºç‰ˆæ™‚ã«æ–°ã—ãリリースã•ã‚ŒãŸãƒãƒ¼ã‚¸ãƒ§ãƒ³ã§ã‚ã‚‹ Spring Boot 2 を使用ã—ã¦ã„る。ã—ã‹ã—ã€Spring Boot3ãŒæœ€è¿‘リリースã•ã‚Œã€é‡è¦ãªå¤‰...
Hadoopã¯èª°ãŒé–‹ç™ºã—ã¦ã„ã‚‹ã®ã‹ï¼ˆ2013年） - ã‚ã—ãŸã€ãã£ã¨ ã‹ã‚る。。。
ã“ã®æŠ•ç¨¿ã¯ @hamaken ã«ã‚ˆã‚‹Hadoop Advent Calendar 2013ã®18日目ã®è¨˜äº‹ã§ã™ã€‚ ã“ã®ã‚¨ãƒ³ãƒˆãƒªã§ã¯ã€Hadoop ãŒèª°ã®æ‰‹ã«ã‚ˆã£ã¦é–‹ç™ºã•ã‚Œã¦ã„ã‚‹ã®ã‹ãƒ»ãƒ»ãƒ»ã‚’紹介ã—ã¾ã™ã€‚ ã“ã‚Œã¾ã§ã« Hortonworks ã‚„ Cloudera ã«ã‚ˆã£ã¦ Hadoop ã«è²¢çŒ®ã—ãŸã‚³ãƒ¼ãƒ‰é‡ãªã©ã®é›†è¨ˆãŒç´¹ä»‹ã•ã‚Œã¦ãã¾ã—ãŸ*1。ã—ã‹ã—ã€2011年を最後ã«æ–°ã—ã„データãŒå‡ºã¦ãã¦ã„ãªã„ã®ã§ã€2013å¹´ã¯ã©ã†ã ã£ãŸã®ã‹ãªãƒ¼ã€ã¨æ€ã„集計ã—ã¦ã¿ã¾ã—ãŸã€‚ 集計作æ¥ã¯ @ajis_ka ã•ã‚“ã«å”力ã—ã¦ã‚‚らã„ã¾ã—ãŸã€‚ã‚ã‚ŠãŒã¨ã†ã”ã•ã„ã¾ã™ï¼ ■集計方法 皆ã•ã‚“ã€ã”å˜ã˜ã®ã¨ãŠã‚Š Apache Hadoop ã®é–‹ç™ºã¯ JIRA ã‚’ä¸å¿ƒã«é€²ã‚られã¦ã„ã¾ã™*2。新機能もã€ãƒã‚°ä¿®æ£ã‚‚ã€JIRA ã® Issue ã¨ã—ã¦ç™»éŒ²ãƒ»ç®¡ç†ã•ã‚Œã‚‹ã®ãŒåŽŸå‰‡ã§ã™ã€‚ 今回ã®é›†è¨ˆã§ã¯ã€Hadoopã®ä¸»è¦ã‚³ãƒ³ãƒãƒ¼ãƒãƒ³ãƒˆï¼ˆHADOO
Hadoop活用をæˆåŠŸã«å°Žãéµã¨ã¯ï¼Ÿ--本気ã§å–り組むä¼æ¥ã»ã©ã€Œå•†ç”¨è£½å“ã€ã‚’é¸ã¶ç†ç”±
2013å¹´ã®ITæ¥ç•Œã€æµè¡Œèªžå¤§è³žã®æœ‰åŠ›å€™è£œã¨ã—ã¦é–“é•ã„ãªã—ã®ãƒ“ッグデータ。ãã“ã‹ã‚‰çš†ã•ã‚“ã¯ä½•ã‚’連想ã™ã‚‹ã ã‚ã†ã‹ã€‚ ã‚„ã¯ã‚Šãƒ“ッグデータã¨ã„ãˆã°Hadoopã€Hadoopã¨ã„ãˆã°ã‚ªãƒ¼ãƒ—ンソースã€ãã—ã¦ã‚ªãƒ¼ãƒ—ンソースã¨ã„ãˆã°ã€Œç„¡å„Ÿã€--。ã“ã†æ„Ÿã˜ã‚‹äººã¯å¤šã„ã¯ãšã 。ãã†ã§ã‚ã‚Œã°ã€å•†ç”¨ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã®ãƒ™ãƒ³ãƒ€ãƒ¼ã«å¯¾ä¾¡ã‚’支払ã£ã¦Hadoopを使ã†ã€ã¾ã—ã¦ç§ãŒå±žã™ã‚‹IBMã«ç›¸è«‡ã™ã‚‹ã€ãªã©ã¨ã„ã†ç™ºæƒ³ã«ã¯ã€ãªã‹ãªã‹çµã³ã¤ã‹ãªã„ã®ã§ã¯ãªã„ã ã‚ã†ã‹ã€‚ ã—ã‹ã—ã€Hadoopを本気ã§ä½¿ãŠã†ã¨ã™ã‚‹ä¼æ¥ã®å¤šãã¯ã€å•†ç”¨ãƒ™ãƒ³ãƒ€ãƒ¼ãŒæä¾›ã™ã‚‹æœ‰å„Ÿè£½å“を採用ã—ã¦ã„ã‚‹ã€ã¨ã„ã†ã®ã‚‚ã¾ãŸäº‹å®Ÿãªã®ã 。Gartner Magic Quadrant BI and Analytics 2013ã«ã‚ˆã‚Œã°ã€IBMã¯ãƒ“ッグデータã®åˆ†é‡Žã«ãŠã„ã¦æ¥ç•Œã®ãƒªãƒ¼ãƒ€ãƒ¼ã¨ã—ã¦ãƒã‚¸ã‚·ãƒ§ãƒ³ã‚’確立ã—ã¦ã„ã‚‹ã¨ã•ã‚Œã¦ãŠã‚Šã€Hadoopを活用ã—よã†ã¨ã™ã‚‹ä¼æ¥ã‚’支æ´ã™ã‚‹è£½å“ã¨ã‚µãƒ¼
Design, Scale and Performance of MapR's Distribution for Hadoop
Details the first ever Exabyte-scale system that can hold a Trillion large files. Describes MapR's Distributed NameNode (tm) architecture, and how it scales very easily and seamlessly. Shows map-reduce performance across a variety of benchmarks like dfsio, pig-mix, nnbench, terasort and YCSB.Read less
IBMã€Hadoop専用機ãªã©ã€ŒPureData Systemã€è£½å“群を強化
日本アイ・ビー・エム（日本IBM）ã¯10月15æ—¥ã€ãƒ“ッグデータ処ç†å‘ã‘ã®çµ±åˆãƒ—ラットフォーム「IBM Pure Data Systemã€ã®ãƒ©ã‚¤ãƒ³ã‚¢ãƒƒãƒ—拡充を発表ã—ãŸã€‚æ–°ãŸã«Hadoopアプライアンス（専用機）ã€ãŠã‚ˆã³ã‚¨ãƒ³ãƒˆãƒªãƒ¼ãƒ¢ãƒ‡ãƒ«ã®ãƒ‡ãƒ¼ã‚¿ã‚¦ã‚§ã‚¢ãƒã‚¦ã‚¹ï¼ˆDWHï¼‰ã‚¢ãƒ—ãƒ©ã‚¤ã‚¢ãƒ³ã‚¹ã‚’è¿½åŠ ã™ã‚‹ã€‚ IBMã§ã¯ã€ãƒãƒ¼ãƒ‰ã‚¦ã‚§ã‚¢ã¨ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ã®ã‚³ãƒ³ãƒãƒ¼ãƒãƒ³ãƒˆã‚’çµ±åˆã—ã€ã‚·ã‚¹ãƒ†ãƒ 構築やé‹ç”¨ã®çŸ¥è¦‹ã‚’「パターンã€ã¨ã—ã¦æä¾›ã™ã‚‹ã“ã¨ã§è‡ªå‹•çš„ã«æœ€é©ãªè³‡æºã‚’構築ã™ã‚‹ã€Œã‚¨ã‚スパート・インテグレーテッド・システムã€ã¨ã—ã¦ã€ŒIBM PureSystemsã€ãƒ•ã‚¡ãƒŸãƒªãƒ¼ã‚’æä¾›ã—ã¦ã„る。PureData Systemã¯ã€ãã®ã†ã¡ã®ãƒ“ッグデータ処ç†å‘ã‘プラットフォーム製å“群ã«å½“ãŸã‚‹ã€‚ 今回新ãŸã«è¿½åŠ ã•ã‚ŒãŸã€ŒIBM PureData System for Hadoop H1001ã€ã¯ã€Hadoop基盤ソフトウェア「IBM InfoSp
アクセンãƒãƒ¥ã‚¢æ–°å’・第二新å’採用
<section id="hero_custom" class="hero_custom"> <video aria-hidden="true" role="presentation" class="hero_custom__background hero_custom__background--video" webkit-playsinline="true" playsinline="true" loop="true" autoplay="true" muted="true"> <source src="/content/dam/system-files/acom/custom-code/change-more/media/custom-rad-hero-bg-video.mp4" type="video/mp4" /> </video> <a class="hero_custom__c
Notes of upgrading from CDH4.1 to CDH4.4 | 外é“父ã®åŒ
ã¤ã„ã¤ã„ CDH4.1 ã‹ã‚‰ CDH4.4 ã«ã‚¢ãƒƒãƒ—グレードã—ã¦ã—ã¾ã„ã¾ã—ãŸã®ã§ã€æ‰‹é †ã‚’çœã„ã¦æ³¨æ„点ãªã©ã‚’記ã—ã¦ãŠãã¾ã™ã€‚ 機能的ã«ã¯ What’s New in CDH4.4.0 ã¾ã§è¦‹ã¦ã‚‚メリットよりリスクä¸å®‰ã®æ–¹ãŒé«˜ã„ã®ã§ã™ãŒã€Hadoop新担当者ã®é‹ç”¨é›éŒ¬ã¨ã„ã†åç›®ã§ã‚´ãƒªãƒƒã¨ã‚„ã£ã¦ã‚‚らã£ã¦ã€ç§ã¯ãã®å¾Œã‚ã§ç…½ã£ã¦ã¾ã—ãŸã€‚ æ‰‹é †ã«ã¤ã„㦠How to upgrade from CDH4.0 to CDH4.1 for Debian | 外é“父ã®åŒ ã¨æµã‚Œã¯åŒã˜ã§ã€ ジョブをæ¢ã‚㦠Hadoopã‚’æ¢ã‚㦠アップグレードã—㦠Hiveメタストアを更新ã—㦠å†é–‹ã—ã¦å‹•ä½œç¢ºèªã™ã‚‹ï¼ˆãƒã‚°ä¿å˜ï¼ã‚¸ãƒ§ãƒ–） ã ã‘ãªã®ã§é›£ã—ã„ã“ã¨ã¯ç‰¹ã«ã‚ã‚Šã¾ã›ã‚“。 QJM HAも特ã«ãªã«ã‚‚ãªã‹ã£ãŸã§ã™ã€‚ ãªã®ã§ã€ç´°ã‹ã„メモã ã‘書ã„ã¦ãŠãã¾ã™ã€‚ NameNode WARNãƒã‚° 挙動ã«æ”¯éšœã¯ãªã„ã®ã§ã™ãŒã€èªã¿æ›¸ã両方ã¨ã‚‚ã«
MapR アーã‚テクãƒãƒ£æ¦‚è¦ - MapR CTO Meetup 2013/11/12
3. 3 ‘09 ‘11 07 06 MapReduce 論文を発表 MR 論文もã¨ã« Hadoop を開発 Hadoop ã‚’ 利用開始 Hadoop ã‚’ 利用開始 Hadoop ã‚’ 利用開始 NYダウ㌠14,300 ã‹ã‚‰ 6,800 ã«æ€¥è½ MapR è¨ç«‹ ‘13 ‘12 é«˜ä¿¡é ¼æ€§Hadoopを発表 ã¨ã®ãƒ‘ートナー 数々ã®ä¸–ç•Œ 記録を更新 MapR ã®æ´å² 2500 ノード ã®æœ€å¤§ã® 商用クラスタ MapR M7 世界最速NoSQL 4. 4 MapR DistribuCon for Apache Hadoop åˆ†æž ã‚ªãƒšãƒ¬ãƒ¼ã‚·ãƒ§ãƒ³ OLTP Management ãƒãƒƒãƒ インタラクティブ リアルタイムData Platform NoSQL Hardening, Testing, Integrating, Innovating, Contributing as part of the
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
SASã€Hadoop対応ã®ç”»æœŸçš„ãªãƒ“ッグデータ・アナリティクス・ソフトウェアを発表
CDH 5ã§ä¼æ¥ã«Hadoopã®ã‚¤ãƒ³ãƒ•ãƒ©ã‚’～米Cloudera CTO
ã€ä»®æƒ³åŒ–é“å ´ã€‘ 並列分散処ç†ã®ãƒ•ãƒ¬ãƒ¼ãƒ ワークã¨ãªã‚‹Hadoop 2.0
marblog
IBM Documentation
https://qiita.com/advent-calendar/2013/hadoop/participants
Attunity Solutions for Data Analysis (Hadoop EMR) on Amazon S3 and EMR Clusters on EC2 | Attunity CloudBeam
{{#tags}}- {{label}}
{{/tags}}