[B! mapreduce] uokadaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

uokada id:uokada

mapreduceã«é–¢ã™ã‚‹uokadaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (10)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Pig, Hiveãªã©8ç¨®é¡žã®MapReduceè¨€èªžã«ã¤ã„ã¦ã®æ¯”è¼ƒ - nokunoã®æ—¥è¨˜
ä»¥ä¸‹ã®è¨˜äº‹ã§ã¯Pig, Hiveãªã©8ç¨®é¡žã®è¨€èªžã§MapReduceã®ä¸€ç•ªå˜ç´”ãªä¾‹ã€WordCountã‚’è¨˜è¿°ã—ãŸå ´åˆã®ã‚¹ã‚¯ãƒªãƒ—ãƒˆã®æ¯”è¼ƒã‚’è¡Œã£ã¦ã„ã¾ã™ã€‚ã¨ã¦ã‚‚èˆˆå‘³æ·±ã‹ã£ãŸã®ã§ç´¹ä»‹ã—ãŸã„ã¨æ€ã„ã¾ã™ã€‚Pigs, Bees, and Elephants: A Comparison of Eight MapReduce Languages Â« Dataspora ãªãŠã€ä»Šå›žã¯RSSè³¼èªã—ã¦ã„ã‚‹DataSporaã®ãƒ–ãƒã‚°ã‹ã‚‰ç™ºè¦‹ã—ã¾ã—ãŸãŒã€ã‚¨ãƒ³ãƒˆãƒªã‚’å¯„ç¨¿ã—ãŸã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®å€‹äººãƒ–ãƒã‚°ã‚‚å¤§å¤‰èˆˆå‘³æ·±ã„ã®ã§ã‚ªã‚¹ã‚¹ãƒ¡ã§ã™ã€‚ ã¯ã˜ã‚ã«MapReduceã‚’ç¾Žã—ãåŠ¹çŽ‡çš„ã«æ›¸ããŸã‚ã«ã€ç§ã¯æ§˜ã€…ãªè¨€èªžã‚’æ¯”è¼ƒã—ã¾ã—ãŸã€‚æžœãŸã—ã¦ãã®å‹è€…ã¯ï¼ï¼Ÿç§ã®å€‹äººãƒ–ãƒã‚°ã§ã¯çµ±è¨ˆã‚„ã‚°ãƒ©ãƒ•ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã‚’MapReduceã§è¨˜è¿°ã™ã‚‹æ–¹æ³•ã‚’ç´¹ä»‹ã—ã€æ“¬ä¼¼ã‚³ãƒ¼ãƒ‰ã«ã‚ˆã‚‹å®Ÿè£…ã‚’ç¤ºã—ã¾ã—ãŸã€‚æ“¬ä¼¼ã‚³ãƒ¼ãƒ‰ã«ã¯2ã¤ã®å•é¡Œç‚¹ãŒã‚ã‚Šã¾ã™ï¼šèª°ã‚‚ãŒãã®å‘½ä»¤ã‚’ç†è§£ã§ãã‚‹ã¨
uokada 2015/04/25
mapreduce

hadoop

hive
ãƒªãƒ³ã‚¯
niw.at â€” ãŸã®ã—ã„ Scalding å…¥é–€
TL; DR: git clone https://github.com/niw/scalding_examples.git ã–ã£ã¨ã‚°ã‚°ã£ãŸæ„Ÿã˜ã€Scalding ã«ã¤ã„ã¦ã¯ã¾ã ã‚ã¾ã‚Šæ—¥æœ¬èªžæƒ…å ±ãªã•ãã†ãªã®ã§ã€ã“ã†åºƒã‚ãŸã‚Šã™ã‚‹ç›®çš„ã‚‚è¾¼ã‚ã¦ã¡ã‚‡ã£ã¨ã¾ã¨ã‚ã¦ãŠã“ã†ã¨æ€ã„ã¾ã™ã€‚ Scalding ã¨ã¯ Scalding ã¨ã¯ã€Scala ã«ç›¸å½“å…¥ã‚Œè¾¼ã‚“ã§ã‚‹ Twitter ã§ä½¿ã‚ã‚Œã¦ã„ã‚‹ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã§ã€Hadoop ã® MapReduce ã‚’ Scala ã§ç°¡å˜ã«æ›¸ã‘ã‚‹ã‚‚ã®ã§ã™ã€‚ä¸èº«ã¯ Cascading ã‚’ãƒ©ãƒƒãƒ—ã—ãŸã‚‚ã®ã«ãªã£ã¦ã„ã¾ã™ã€‚ Scala ã«ç›¸å½“å…¥ã‚Œè¾¼ã‚“ã§ã‚‹ Twitter ã§ã¯ã—ã‹ã—ãªãŒã‚‰ Pig ãŒé »ç¹ã«ä½¿ã‚ã‚Œã¦ã„ã‚‹ã®ã§ã™ãŒã‚ã‚‹æ–¹é¢ã§ã¯ Scalding ãŒä½¿ã‚ã‚Œã¦ãŠã‚Šã¾ã—ã¦ã€Pig ã¨è‚©ã‚’ä¸¦ã¹ã‚‰ã‚Œã‚‹ãã‚‰ã„ã«ä½¿ãˆã¦ã€ã•ã‚‰ã« Scala ã®ç§˜ã‚ãŸåŠ›ã‚‚ä½¿ãˆã‚‹ã‚“ã ãœâ€¦! ã¨ã€å°‘ãªãã¨ã‚‚
uokada 2013/09/01
scala

hadoop

mapreduce
ãƒªãƒ³ã‚¯
Googleã®ä¸¦åˆ—ãƒã‚°è§£æžå‘ã‘è¨€èªžã€ŒSawzallã€ãŒå…¬é–‹ã•ã‚ŒãŸã®ã§ä½¿ã£ã¦ã¿ãŸ | Preferred Research Blog
æœ€è¿‘å…‰éººã«ãƒãƒžã£ã¦ã„ã‚‹å¤ªç”°ã§ã™ã€‚ ã‚°ãƒ¼ã‚°ãƒ«ã€åˆ†æ•£å‡¦ç†ã®ãŸã‚ã«ãƒ‡ã‚¶ã‚¤ãƒ³ã•ã‚ŒãŸè¨€èªžã€ŒSawzallã€ã‚’ã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã§å…¬é–‹ ? Publickeyã§ç´¹ä»‹ã•ã‚Œã¦ã„ã‚‹ã€ä¸¦åˆ—ãƒã‚°è§£æžå‘ã‘è¨€èªžã€ŒSawzallã€ã‚’è©¦ã—ã¦ã¿ã¾ã—ãŸã€‚å‹•ã‹ã—æ–¹ã®ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆãŒå°‘ãªã‹ã£ãŸã®ã§ã€ç´¹ä»‹ã‚¨ãƒ³ãƒˆãƒªã‚’æ›¸ã„ã¦ã¿ã¾ã™ã€‚ ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆãƒšãƒ¼ã‚¸ ãƒ‰ã‚ãƒ¥ãƒ¡ãƒ³ãƒˆ Sawzallã«ã¤ã„ã¦ã¯ã€5å¹´å‰ã«è«–æ–‡ãŒç™ºè¡¨ã•ã‚Œã¦ãŠã‚Šä¸€éƒ¨æ¦‚è¦ã‚’çŸ¥ã‚‹ã“ã¨ã¯å‡ºæ¥ã¾ã—ãŸãŒã€å…ˆæ—¥å®Ÿè£…ãŒã‚ªãƒ¼ãƒ—ãƒ³ã‚½ãƒ¼ã‚¹ã§å…¬é–‹ã•ã‚Œã¾ã—ãŸã€‚è«–æ–‡ã®ç¬¬ä¸€è‘—è€…ã¯UNIXã‚„Plan9ã®é–‹ç™ºè€…ã§çŸ¥ã‚‰ã‚Œã‚‹Rob Pikeæ°ã§ã™ã€‚ Interpreting the Data: Parallel Analysis with Sawzall MapReduceã®OSSå®Ÿè£…ã¨ã—ã¦ã€ŒHadoopã€ãŒè‰¯ãçŸ¥ã‚‰ã‚Œã¦ã„ã¾ã™ãŒã€Hadoopå‘ã‘ã®è¨€èªžã¨ã—ã¦ã¯Hiveã‚„Pigç‰ãŒæœ‰åã§ã™ã€‚ Hive: MapRed
uokada 2013/02/18
mapreduce

hadoop
ãƒªãƒ³ã‚¯
ã‚¢ã‚¯ã‚»ã‚¹è§£æžã‚·ã‚¹ãƒ†ãƒ ã®è£å´ ï¼ˆå…¬é–‹ç”¨ï¼‰
2. ç›®æ¬¡ 1. è‡ªå·±ç´¹ä»‹ 2. ãƒ¦ãƒ¼ã‚¶ãƒ¼ãƒãƒ¼ã‚«ãƒ«ã¨ã‚¢ã‚¯ã‚»ã‚¹è§£æž 3. ã‚¢ã‚¯ã‚»ã‚¹è§£æžã®åŸºç¤Ž 4. ã‚¢ã‚¯ã‚»ã‚¹è§£æžã‚·ã‚¹ãƒ†ãƒ ã®è£å´ - ãƒªã‚¢ãƒ«ã‚¿ã‚¤ãƒ é›†è¨ˆ - å°ãƒãƒƒãƒé›†è¨ˆ - å¤§è¦æ¨¡ãƒãƒƒãƒé›†è¨ˆ ã‚¹ãƒ©ã‚¤ãƒ‰49æžš 2 12å¹´9æœˆ23æ—¥æ—¥æ›œæ—¥ 3. è‡ªå·±ç´¹ä»‹ â€¢ ä¸‰ä¸Šä¿Šè¼” â€¢ æ ªå¼ä¼šç¤¾ãƒ¦ãƒ¼ã‚¶ãƒ¼ãƒãƒ¼ã‚«ãƒ« â€¢ ä»Šå¹´ï¼“æœˆã«ç‘æ³¢å¤§å¦é™¢ã‚’å’æ¥ â€¢ å¤§å¦ã§ã¯åˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ ã®ç ”ç©¶ â€¢ å¦ç”Ÿã®æ™‚ã¯Cookpadã®ãƒ‡ãƒ¼ã‚¿ãƒžã‚¤ãƒ‹ãƒ³ ã‚°éƒ¨é–€ã§2ãƒ¶æœˆã»ã©ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ³ 3 12å¹´9æœˆ23æ—¥æ—¥æ›œæ—¥ 4. éŽåŽ»ã®ç ”ç©¶ã€ç™ºè¡¨ â€¢ ç ”ç©¶ãƒ–ãƒã‚°ï¼šhttp://shun0102.net/ â€¢ ç™ºè¡¨ï¼šã€Œåˆ†æ•£ãƒ•ã‚¡ã‚¤ãƒ«ã‚·ã‚¹ãƒ†ãƒ Gfarm ä¸Šã§ã®Hadoop MapReduceã€ãªã© - http://www.slideshare.net/shun0102/ 4 12å¹´9æœˆ23æ—¥æ—¥æ›œæ—¥
uokada 2012/09/23
hadoop

MapReduce
ãƒªãƒ³ã‚¯
liris.orgÂ -Â ã“ã®ã‚¦ã‚§ãƒ–ã‚µã‚¤ãƒˆã¯è²©å£²ç”¨ã§ã™ï¼Â -Â liris ãƒªã‚½ãƒ¼ã‚¹ãŠã‚ˆã³æƒ…å ±
This webpage was generated by the domain owner using Sedo Domain Parking. Disclaimer: Sedo maintains no relationship with third party advertisers. Reference to any specific service or trade mark is not controlled by Sedo nor does it constitute or imply its association, endorsement or recommendation.
uokada 2012/06/12
python

programming

mapreduce

hadoop
ãƒªãƒ³ã‚¯
English (US)
Did someone say â€¦ cookies? Twitter and its partners use cookies to provide you with a better, safer and faster service and to support our business. Some cookies are necessary to use our services, improve our services, and make sure they work properly. Show more about your choices.
uokada 2012/03/08
twitter

scala

framework

hadoop

mapreduce
ãƒªãƒ³ã‚¯
ç¿»è¨³ - Pythonã®multiprocessingãƒ¢ã‚¸ãƒ¥ãƒ¼ãƒ«ã‚’ä½¿ã£ã¦MapReduceã‚’ã‚¤ãƒ³ãƒ—ãƒªã™ã‚‹ â€” TRIVIAL TECHNOLOGIES 2.0
ã¿ã‚“ãªã®IoT/ã¿ã‚“ãªã®Pythonã®è‘—è€…ã€‚äºŒåçŽ‰è¿‘ã160å¹³ç±³ã®åºä»˜ãä¸€æˆ¸å»ºã¦ã«å«/æ¯å/å¨˜/ã‚ã‚“ã“ã¨æš®ã‚‰ã—ã¦ã¾ã™ã€‚æœˆé–“1000ä¸‡PV/150ä¸‡UUã®Webã‚µãƒ¼ãƒ“ã‚¹é‹å–¶ä¸ã€‚ å…è²¬äº‹é … ãƒ—ãƒ©ã‚¤ãƒã‚·ãƒ¼ãƒãƒªã‚·ãƒ¼ PyMOTWã‚’ãƒ¡ãƒ³ãƒ†ã—ã¦ã‚‹Doug Hallmannã•ã‚“ãŒPython 2.6(ãŠã‚ˆã³3.0)ã‹ã‚‰å°Žå…¥ã•ã‚ŒãŸï¼Œåˆ†æ•£ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ã‚’å¼·åŠ›ã«ã‚µãƒãƒ¼ãƒˆã™ã‚‹multiprocessingãƒ¢ã‚¸ãƒ¥ãƒ¼ãƒ«ã‚’ä½¿ã£ã¦MapReduceã‚’å®Ÿè£…ã—ã¦ã„ã‚‹ã®ã‚’ç™ºè¦‹ã—ãŸã€‚MapReduceã¯GoolgeãŒä½¿ã£ã¦ã„ã‚‹ä¸¦åˆ—å‡¦ç†ã‚·ã‚¹ãƒ†ãƒ ã€‚multiprocessingãƒ¢ã‚¸ãƒ¥ãƒ¼ãƒ«ã®ã‚µãƒ³ãƒ—ãƒ«ã¨ã—ã¦ã‚‚ï¼ŒPythonã‚’ä½¿ã£ãŸMapReduceã®å®Ÿè£…ã¨ã—ã¦ã‚‚é¢ç™½ã„ã—ï¼Œå‹‰å¼·ã«ãªã‚‹ã¨æ€ã†ã®ã§ã‚¨ãƒ³ãƒˆãƒªã‚’ç¿»è¨³ã—ã¦ã¿ãŸã€‚Pythonã‚’è¦šãˆã‚‹ã“ã¨ã®åˆ©ç‚¹ã¯ã„ãã¤ã‹ã‚ã‚‹ã‘ã©ï¼Œã“ã†ã„ã†ä¸Šè³ªãªãƒªã‚½ãƒ¼ã‚¹ã«ãƒªãƒ¼ãƒã™ã‚‹æ¨©åˆ©ã‚’å¾—ã‚‰ã‚Œã‚‹ï¼Œã®ã‚‚å¤§ããªåˆ©ç‚¹ã®ä¸€
uokada 2012/03/05
mapreduce

multiprocessing

programming

python
ãƒªãƒ³ã‚¯
MapReduceä»¥å¤–ã®åˆ†æ•£å‡¦ç†åŸºç›¤BSP, Piccolo, Sparkã®ç´¹ä»‹ - Preferred Networks Research & Development
ã©ã†ã‚‚ï¼Œå®Ÿã¯ä»Šå¹´ã‹ã‚‰é–‹ç™ºãƒãƒ¼ãƒ ã«joinã—ã¦ã„ãŸä¸å·ã§ã™ï¼Žå¯æ„›ã„çŠ¬ã®å†™çœŸãŒãªã‹ã£ãŸã®ã§ï¼Œå¯æ„›ã„ãƒžã‚¹ã‚³ãƒƒãƒˆã®ç”»åƒã‚’è²¼ã£ã¦ãŠãã¾ã™ï¼Ž æœ€è¿‘MapReduceã¨ã‹ãã®å®Ÿè£…ã§ã‚ã‚‹Hadoopã¨ã‹ã‚’ã‚ˆãèžãã‚ˆã†ã«ãªã‚Šã¾ã—ãŸï¼Žã“ã‚Œã¯ã¤ã¾ã‚Šï¼Œãã‚Œã ã‘å¤§é‡ã®ãƒ‡ãƒ¼ã‚¿ã‚’ãªã‚“ã¨ã‹å‡¦ç†ã—ãŸã„ã¨ã„ã†è¦æœ›ãŒã‚ã‚‹ã‹ã‚‰ã ã¨æ€ã„ã¾ã™ï¼Žã—ã‹ã—å½“ãŸã‚Šå‰ã§ã™ãŒï¼ŒMapReduceã¯éŠ€ã®å¼¾ä¸¸ã§ã¯ã‚ã‚Šã¾ã›ã‚“ï¼Ž ã¨ã„ã†ã“ã¨ã§ï¼Œæœ€è¿‘æ°—ã«ãªã£ã¦ã„ã‚‹MapReduceã¨ã¯é•ã£ãŸã‚¢ãƒ—ãƒãƒ¼ãƒã‚’å–ã£ã¦ã„ã‚‹åˆ†æ•£å‡¦ç†åŸºç›¤ã«ã¤ã„ã¦ï¼Œç¤¾å†…ã®TechTalkã§è©±ã—ãŸå†…å®¹ã‚’ç°¡å˜ã«ã¾ã¨ã‚ã¦ç´¹ä»‹ã—ãŸã„ã¨æ€ã„ã¾ã™ï¼Ž Bulk Sychronous Parallel ã“ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ è‡ªä½“ã¯1990å¹´ã«èª•ç”Ÿã—ãŸã‚‚ã®ã§ã™ï¼Žé•·ã„ã®ã§BSPã¨æ›¸ãã¾ã™ï¼Žã•ã¦ï¼Œã‚°ãƒ©ãƒ•ã‹ã‚‰æœ€çŸçµŒè·¯ã‚’æ±‚ã‚ã‚‹æ™‚ï¼ŒMapReduceã¯ä½¿ãˆã‚‹ã§ã—ã‚‡ã†ã‹ï¼Ÿã“ã®ã‚ˆã†ãªè«–æ–‡ãŒå‡ºã‚‹ãã‚‰ã„ã§ã™ã‹ã‚‰å‡ºæ¥ãªã„ã“ã¨ã¯ã‚
uokada 2011/10/18
mapreduce

algorithm

Hadoop

bsp

Piccolo

Hama
ãƒªãƒ³ã‚¯
Scala on Hadoop: Hadoop Conference - stanaka's blog
å…ˆæ—¥ã€Hadoop Conferenceã§Scala on Hadoopã¨ã„ã†ã‚¿ã‚¤ãƒˆãƒ«ã§ç™ºè¡¨ã—ã¦ãã¾ã—ãŸã€‚ã‚¹ãƒ©ã‚¤ãƒ‰ã‚’ä»¥ä¸‹ã«ç½®ã„ã¦ãŠãã¾ã™ã€‚ Scala on HadoopView more presentations from Shinji Tanaka. ãƒ€ã‚¤ã‚¸ã‚§ã‚¹ãƒˆã¨ã—ã¦ã€Scalaã‚’Hadoopã§å‹•ã‹ã™ãŸã‚ã®æ–¹æ³•ã‚’æ›¸ã„ã¦ãŠãã¾ã™ã€‚ ã¾ãšã€Hadoopä¸Šã§Scalaã‚’å®Ÿè¡Œã•ã›ã‚‹ãŸã‚ã«ã¯ã€Javaã¨Scalaã‚’æŽ¥ç¶šã™ã‚‹ãƒ©ã‚¤ãƒ–ãƒ©ãƒªãŒå¿…è¦ã¨ãªã‚Šã¾ã™ã€‚ã“ã“ã§ã¯ã€SHadoop( http://code.google.com/p/jweslley/source/browse/#svn/trunk/scala/shadoop )ã‚’ä½¿ç”¨ã—ã¾ã™ã€‚SHadoopã¯ã€åž‹å¤‰æ›ã‚’è¡Œã†ã‚·ãƒ³ãƒ—ãƒ«ãªãƒ©ã‚¤ãƒ–ãƒ©ãƒªã§ã™ã€‚ ã‚ˆãã‚ã‚‹WordCountã®ã‚µãƒ³ãƒ—ãƒ«ã€WordCount.scala (http://blog.jo
uokada 2011/10/07
hadoop

scala

ã‚ã¨ã§èªã‚€

mapreduce

programming

hatena
ãƒªãƒ³ã‚¯
Hadoopå…¥é–€ã¨ã‚¯ãƒ©ã‚¦ãƒ‰åˆ©ç”¨
This document discusses Hadoop and related techno logies. It introduces Hadoop, its components MapReduce and HDFS, and how they work together. It also briefly mentions related Apache projects like Mahout and how companies like Amazon, Yahoo and Facebook use Hadoop in their systems. Finally, it covers Amazon's Elastic MapReduce service, which allows running Hadoop jobs in the cloud.Read less
uokada 2011/03/31
hadoop

mapreduce

HDFS
ãƒªãƒ³ã‚¯
1