[B! hadoop] dannã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

dann id:dann

hadoopã«é–¢ã™ã‚‹dannã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (156)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Preferred Networks ã«ãŠã‘ã‚‹Hadoop - Preferred Networks Research & Development
Preferred Networks ï¼ˆä»¥ä¸‹PFNï¼‰ã§ã¯ã€ã€Œç¾å®Ÿä¸–ç•Œã‚’è¨ˆç®—å¯èƒ½ã«ã™ã‚‹ã€ã€Œå…¨ã¦ã®ã²ã¨ã«ãƒãƒœãƒƒãƒˆã‚’ã€ã¨ã„ã†ç›®æ¨™ã®ã‚‚ã¨ã€æ©Ÿæ¢°å¦ç¿’ã‚’å§‹ã‚ã¨ã—ãŸã‚ã‚‰ã‚†ã‚‹è¨ˆç®—æŠ€è¡“ã‚’ç”¨ã„ã¦ç ”ç©¶é–‹ç™ºã«æ—¥ã€…å–ã‚Šçµ„ã‚“ã§ã„ã¾ã™ã€‚ãã®éŽç¨‹ã§ã¯å¿…ãšã¨ã„ã£ã¦ã„ã„ã»ã©ã€ãƒ‡ãƒ¼ã‚¿ã®ä¿å˜ã‚„èªã¿å‡ºã—ãŒå¿…è¦ã«ãªã‚Šã¾ã™ã€‚ã“ã“ã§ã¯ã€æˆ‘ã€…ãŒã©ã®ã‚ˆã†ã«ãƒ‡ãƒ¼ã‚¿ç®¡ç†ã‚’ã—ã¦ã„ã‚‹ã‹ã€ã¾ãŸã€ãã®éŽç¨‹ã§ã©ã®ã‚ˆã†ã«Hadoopã‚’åˆ©ç”¨ã—ã¦ã„ã‚‹ã‹ã«ã¤ã„ã¦ç´¹ä»‹ã—ãŸã„ã¨æ€ã„ã¾ã™ã€‚ å†™çœŸ: PFNã‚«ãƒ©ãƒ¼ãƒªãƒ³ã‚°ã®ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã‚µãƒ¼ãƒãƒ¼ Hadoopå°Žå…¥ã®çµŒç·¯ HadoopãŒå¤šãåˆ©ç”¨ã•ã‚Œã¦ã„ã‚‹ã‚ˆã†ãªãƒã‚°åˆ†æžã‚„ã€ã‚¨ãƒ³ã‚¿ãƒ¼ãƒ—ãƒ©ã‚¤ã‚ºå‘ã‘ã®ETLå‡¦ç†ã‚„ãƒ‡ãƒ¼ã‚¿ã‚µã‚¤ã‚¨ãƒ³ã‚¹ã«æ¯”ã¹ã‚‹ã¨ã€æ·±å±¤å¦ç¿’ã®åˆ†é‡Žã§ã‚ˆãåˆ©ç”¨ã•ã‚Œã¦ã„ã‚‹ã‚ˆã†ãªè¦æ¨¡ã®ãƒ‡ãƒ¼ã‚¿ã¯æ¯”è¼ƒçš„å°ã•ã„ã§ã™ã€‚ã‚ˆãç”»åƒèªè˜ã®ãƒ™ãƒ³ãƒãƒžãƒ¼ã‚¯ã¨ã•ã‚Œã‚‹ImageNetã®2012å¹´ã®ã‚³ãƒ³ãƒšã§åˆ©ç”¨ã•ã‚ŒãŸãƒ‡ãƒ¼ã‚¿ã‚»ãƒƒãƒˆã¯200GBç¨‹åº¦ã§ã™ã€‚ã“ã‚Œã¯ãã‚‚ã
dann 2020/06/30
pfn

storage

hadoop
ãƒªãƒ³ã‚¯
ç¬¬1å›žã€€ãªãœã€Hadoopã¯ã©ã®ã‚ˆã†ã«å‹•ãã®ã‹ã€ã‚’å¦ã¶ã®ã‹ | gihyo.jp
ã¯ã˜ã‚ã« ãƒ“ãƒƒã‚°ãƒ‡ãƒ¼ã‚¿è§£æžã®ãŸã‚ã®ã‚·ã‚¹ãƒ†ãƒ åŸºç›¤ã¨ã—ã¦ã€Hadoopã‚’ã¯ã˜ã‚ã¨ã™ã‚‹ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã®ãƒ‡ãƒ¼ã‚¿å‡¦ç†ã‚½ãƒ•ãƒˆã‚¦ã‚§ã‚¢ï¼ˆãƒ‡ãƒ¼ã‚¿å‡¦ç†ç³»ï¼‰ãŒåºƒãåˆ©ç”¨ã•ã‚Œã¤ã¤ã‚ã‚Šã¾ã™ãŒã€å½“è©²ãƒ‡ãƒ¼ã‚¿å‡¦ç†ç³»ã‚’ã™ã§ã«åˆ©ç”¨ã—ã¦ã„ã‚‹ã€ã‚‚ã—ãã¯åˆ©ç”¨ã®æ¤œè¨Žã‚’ã—ã¦ã„ã‚‹èªè€…ã®æ–¹ã€…ã®ä¸ã«ã¯ã€ãŸã¨ãˆã°ä»¥ä¸‹ã®ã‚ˆã†ãªå•é¡Œã‚’æŠ±ãˆã¦ã„ã‚‹æ–¹ãŒå°‘ãªã‹ã‚‰ãšã„ã‚‰ã£ã—ã‚ƒã‚‹ã®ã§ã¯ãªã„ã§ã—ã‚‡ã†ã‹ã€‚ ãƒ‡ãƒ¼ã‚¿å‡¦ç†ç³»ã®ä½¿ã„æ–¹ã¯ãªã‚“ã¨ãªãã‚ã‹ã‚‹ãŒã€ãã®å†…éƒ¨ã‚’ã‚ã¾ã‚Šç†è§£ã§ãã¦ã„ãªã„ã€‚ã¾ãŸã¯ã€å†…éƒ¨ã®å‹•ä½œåŽŸç†ãŒã‚ˆãã‚ã‹ã‚‰ãªã„ã®ã§ã€æœ¬æ ¼çš„ã«ä½¿ã†æ°—ã«ãªã‚‰ãªã„ã€‚ åŒæ§˜ã®ç›®çš„ã‚’é”æˆã™ã‚‹è¤‡æ•°ã®ãƒ‡ãƒ¼ã‚¿å‡¦ç†ç³»ã«ãŠã„ã¦ã€ã©ã‚Œã‚’ä½¿ã£ã¦è‰¯ã„ã‹ãŒã‚ˆãã‚ã‹ã‚‰ãªã„ã€‚ã¾ãŸã¯ã€é©åˆ‡ã«ä½¿ã„åˆ†ã‘ã‚‰ã‚Œã¦ã„ãªã„æ°—ãŒã™ã‚‹ã€‚ãŸã¨ãˆã°ã€ã©ã®ã‚ˆã†ãªå ´åˆã«Hadoopã‚’ç”¨ã„ã¦ã€ã©ã®ã‚ˆã†ãªå ´åˆã«åŒé¡žã®ãƒ‡ãƒ¼ã‚¿å‡¦ç†ç³»ã§ã‚ã‚‹Impalaã‚„Sparkã‚’ç”¨ã„ã‚Œã°è‰¯ã„ã‹ãŒâ€œâ æ˜Žç¢ºã«â â€ã‚ã‹ã‚‰ãªã„ã€‚ ã“ã®ã‚ˆã†ãªå•é¡Œã‚’è§£æ±ºã™ã‚‹ã«ã¯ã€
dann 2015/04/01
hadoop
ãƒªãƒ³ã‚¯
JavaOne 2013 Presentation: Next Generation Hadoop: It's Not Just Batch!
Hadoop is rapidly becoming the kernel for distributed computing. Hadoop is known as the de facto tool for anything related to batch processing, but the â€¦
dann 2013/10/05
hadoop
ãƒªãƒ³ã‚¯
Resources | Cloudera
Cloudera acquires Octop ai's platform to enhance metadata management capabilities Read the press release
dann 2013/08/07
hadoop
ãƒªãƒ³ã‚¯
Sqoopãƒ¡ãƒ¢(Hishidama's Apache Sqoop Memo)
æ¦‚è¦ ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ« [/2012-06-19] ä½¿ç”¨ä¾‹ MySQL [/2012-06-01] PostgreSQL [/2012-06-01] Oracle [/2012-06-22] ä½¿ç”¨ä¸Šã®æ³¨æ„ [2012-05-12] Sqoopã®å†…éƒ¨ï¼ˆã‚½ãƒ¼ã‚¹ï¼‰ [/2012-07-07] æ¦‚è¦ Sqoopã¯ã€RDBã¨HDFSé–“ã§ãƒ‡ãƒ¼ã‚¿è»¢é€ã‚’è¡Œã†ãƒ„ãƒ¼ãƒ«ã€‚ 2012/4/2ã«Apacheã®incubatorãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã‹ã‚‰ãƒˆãƒƒãƒ—ãƒ¬ãƒ™ãƒ«ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã«æ˜‡æ ¼ã—ãŸã€‚ ãƒ‡ãƒ¼ã‚¿è»¢é€ã¯Mapã‚¿ã‚¹ã‚¯ã®ã¿ã®MapReduceã§ä¸¦åˆ—ã—ã¦åˆ†æ•£å®Ÿè¡Œã•ã‚Œã‚‹ã€‚ RDBã¸ã®ã‚¢ã‚¯ã‚»ã‚¹ã¯JDBCçµŒç”±ã€‚ ãƒ†ãƒ¼ãƒ–ãƒ«å®šç¾©ã‚‚RDBã‹ã‚‰èªã¿è¾¼ã‚“ã§ãã‚Œã‚‹ã€‚ import RDBã®ãƒ†ãƒ¼ãƒ–ãƒ«ã‚’èªã¿è¾¼ã‚“ã§HDFSä¸Šã®ãƒ•ã‚¡ã‚¤ãƒ«ã«å‡ºåŠ›ã™ã‚‹ã€‚ å·®åˆ†ã ã‘å–ã‚Šè¾¼ã‚€äº‹ã‚‚å‡ºæ¥ã‚‹ï¼ˆmergeï¼‰ã€‚ãŸã ã—å‰Šé™¤ã¯å‡ºæ¥ãªã„ã€‚ export HDFSä¸Šã®ãƒ•ã‚¡ã‚¤ãƒ«ã‚’èªã¿è¾¼ã‚“ã§R
dann 2013/01/26
sqoop

hadoop
ãƒªãƒ³ã‚¯
http://www.nttdocomo.co.jp/binary/pdf/corporate/technology/rd/technical_journal/new/vol20_3_024jp.pdf
dann 2012/12/10
hadoop
ãƒªãƒ³ã‚¯
Distributed Image Processing Using Hadoop MapReduce Framework | PDF | File Format | Convolution
Distributed Image Processing using Hadoop MapReduce framework. Demonstrates how framework can be extended to work with image data for distributed image processing.
dann 2012/12/06
hadoop
ãƒªãƒ³ã‚¯
http://adass2010.cfa.harvard.edu/ADASS2010/incl/presentations/O02_4.pdf
dann 2012/12/06
hadoop
ãƒªãƒ³ã‚¯
http://allthingshadoop.com/2012/07/10/hadoop-distribution-bake-off-my-experience-with-cloudera-and-mapr/
dann 2012/07/11
hadoop
ãƒªãƒ³ã‚¯
Flexible In-Situ Indexing for Hadoop via Elephant Twin
This document discusses flexible indexing in Hadoop. It describes how Twitter uses Elephant-Twin, an open source library they developed, to create indexes at the block level or record level in Hadoop. Elephant-Twin allows minimal changes to jobs/scripts, indexes data without copying it, supports post-factum indexing, and indexes can be used to efficiently retrieve relevant data through an IndexedI
dann 2012/06/24
hadoop
ãƒªãƒ³ã‚¯
Hadoop and Vertica: The Data Analytics Platform at Twitter
Hadoop Summit 2012 - Hadoop and Vertica: The Data Analytics Platform at Twitter The document discusses Twitter's data analytics platform, including Hadoop and Vertica. It outlines Twitter's data flow, which ingests 400 million tweets daily into HDFS, then uses various tools like Crane, Oink, and Rasvelg to run jobs on the main Hadoop cluster before loading analytics into Vertica and MySQL for web
dann 2012/06/18
twitter

hadoop
ãƒªãƒ³ã‚¯
Hands on MapR -- Viadea
The document discusses MapR cluster management using the MapR CLI. It provides examples of starting and stopping a MapR cluster, managing nodes, volumes, mirrors and schedules. Specific examples include creating volumes, linking mirrors to volumes, syncing mirrors, moving volumes and nodes to different topologies, and creating schedules to automate tasks.Read less
dann 2012/05/06
mapr

hadoop
ãƒªãƒ³ã‚¯
GitHub - hanborq/hadoop: A Hanborq optimized Hadoop Distribution, especially with high performance of MapReduce. It's the core part of HDH (Hanborq Distribution with Hadoop for Big Data Engineering).
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
dann 2012/04/20
hadoop
ãƒªãƒ³ã‚¯
Hanborq Optimizations on Hadoop MapReduce
2. Motivations â€¢ MapReduce is a proved and successful data processing framework. It can be implemented enough efficient and flexible. â€“ MapReduce: Simplified Data Processing on Large Clusters â€“ MapReduce: A Flexible Data Processing Tool â€¢ Hadoop is the most popular Open Source implementation of MapReduce, but itâ€™s not so good implemented. â€“ Long latency. â€“ Inefficiency with low performance. â€“ Not
dann 2012/04/20
sendfile, zero-copy

hadoop
ãƒªãƒ³ã‚¯
Introducing Spring Hadoop | SpringSource Team Blog
Categories 2.0 (29) 2.5/2.1 (18) 3.0 (37) 3.1 (15) AMQP (7) Android (4) AOP (19) Apache Tomcat (7) Books (3) Builds (15) Bundlor (1) Business (26) Cloud Computing (46) Cloud Foundry (18) Containers (11) Data Access (38) dm Server (64) Enterprise Integration (17) Google (3) Green Beans (6) Groovy/Grails (47) Hyperic HQ (6) Hyperic IQ (1) IOC Container (31) Java (68) JMS (6) JMX (4) JSF (2) Manageme
dann 2012/03/05
hadoop

spring
ãƒªãƒ³ã‚¯
MapReduceã®ãƒ‘ã‚¿ãƒ¼ãƒ³ã€ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã€ãã—ã¦ãƒ¦ãƒ¼ã‚¹ã‚±ãƒ¼ã‚¹ - ãã—ã ã®Hatena
Ilya Katsovæ°ã«ã‚ˆã‚‹ã€ŒMapReduce Patterns, Algorithms, and Use Casesã€ã®ç¿»è¨³ http://highlyscala ble.wordpress.com/2012/02/01/mapreduce-patterns/ (ä¸‹æ›¸ãã«å…¥ã‚Œã¦æŽ¨æ•²ã™ã‚‹ã¤ã‚‚ã‚ŠãŒã€ãªã‚“ã‹å…¬é–‹ã•ã‚Œã¦ã—ã¾ã£ã¦ã„ãŸã®ã§ã€ã‚ã¨ã§ã„ã‚ã„ã‚ä¿®æ£ã™ã‚‹ã¨æ€ã„ã¾ã™) February 1, 2012 ã“ã®è¨˜äº‹ã§ã¯ã€Webã‚„ç§‘å¦è«–æ–‡ã§è¦‹ã‚‰ã‚Œã‚‹ç•°ãªã‚‹ãƒ†ã‚¯ãƒ‹ãƒƒã‚¯ã®ä½“ç³»çš„ãªè¦–ç‚¹ã‚’ä¸Žãˆã‚‹ãŸã‚ã«ã€æ•°ã€…ã®MapReduceãƒ‘ã‚¿ãƒ¼ãƒ³ã¨ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã‚’ã¾ã¨ã‚ãŸã€‚ ã„ãã¤ã‹ã®å®Ÿç”¨çš„ãªã‚±ãƒ¼ã‚¹ã‚¹ã‚¿ãƒ‡ã‚£ã‚‚æä¾›ã—ã¦ã„ã‚‹ã€‚ ã™ã¹ã¦ã®èª¬æ˜Žã¨ã‚³ãƒ¼ãƒ‰ã‚¹ãƒ‹ãƒšãƒƒãƒˆã§ã¯ã€Mapperã€Reducerã€Combinerã€Partitionaerã€ã‚½ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ã«ãŠã„ã¦Hadoopã®æ¨™æº–çš„ãªMapReduceãƒ¢ãƒ‡ãƒ«ã‚’åˆ©ç”¨ã—ã¾ã™ã€‚ã“ã®ãƒ•ãƒ¬ãƒ¼
dann 2012/02/27
mapreduce

hadoop
ãƒªãƒ³ã‚¯
Hadoop World 2011: Hadoop Troubleshooting 101 - Kate Ting - Cloudera
Attend this session and walk away armed with solutions to the most common customer probl ems. Learn proactive configuration tweaks and best practices to keep your cluster free of fetch failures, job tracker hangs, and the like.Read less
dann 2011/11/12
hadoop
ãƒªãƒ³ã‚¯
ã€Žãƒ¢ãƒã‚²ãƒ¼ã®å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°åŸºç›¤ã«ãŠã‘ã‚‹Hadoopæ´»ç”¨ã€ï¼Hadoop Conference Japan 2011ï¼ #hcj2011
ã€Žãƒ¢ãƒã‚²ãƒ¼ã®å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°åŸºç›¤ã«ãŠã‘ã‚‹Hadoopæ´»ç”¨ã€ï¼Hadoop Conference Japan 2011ï¼ #hcj2011 2011/02/22 [ç™»å£‡å¾Œã‚¨ãƒ³ãƒˆãƒª] ï¼š" ã€Œãƒ¢ãƒã‚²ãƒ¼ã®å¤§è¦æ¨¡ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ã‚°åŸºç›¤ã«ãŠã‘ã‚‹Hadoopæ´»ç”¨ã€ï¼Hadoop Conference Japan 2011 #hcj2011 ã§ç™»å£‡ã—ã¦ãã¾ã—ãŸ " http://d.hatena.ne.jp/hamadakoichi/20110222/p1Read less
dann 2011/10/28
hadoop
ãƒªãƒ³ã‚¯
Mellanox HP_Database Acceleration
dann 2011/10/03
oracle

hadoop

infiniband

rdma

performance
ãƒªãƒ³ã‚¯
http://infra-engineer.com/hadoop/hadoop-conference-japan-2011-fall%E3%81%A7%E4%BD%BF%E7%94%A8%E3%81%95%E3%82%8C%E3%81%9F%E8%B3%87%E6%96%99%E3%82%84%E3%81%A4%E3%81%B6%E3%82%84%E3%81%8D-hcj11f/
dann 2011/09/26
hadoop
ãƒªãƒ³ã‚¯
1 2 3 4 5 6 7 8 æ¬¡ã®ãƒšãƒ¼ã‚¸