Cloudera Blog
In an era where artificial intelligence (AI) is reshaping enterprises across the globe—be it in healthcare, finance, or manufacturing—it’s hard to overstate the transformation that AI has had on businesses, regardless of industry or size. At Cloudera, we recognize the urgent need for bold steps to harness this potential and dramatically accelerate the time to […] Read blog post
The Anatomy of Hadoop I/O Pipeline (Hadoop and Distributed Computing at Yahoo!)
Introduction In a typical Hadoop MapReduce job, input files are read from HDFS. Data are usually compressed to reduce the file sizes. After decompression, serialized bytes are transformed into Java objects before being passed to a user-defined map() function. Conversely, output records are serialized, compressed, and eventually pushed back to HDFS. This seemingly simple, two-way process is in fact
X-RIME: Hadoop based large scale social network analysis
X-RIME: Hadoop based large scale social network analysis Motivation Today's Internet-based social network sites possess huge user communities. They hold large amount of data about their users and want to generate core competency from the data. A key enabler for this is a cost efficient solution for social data management and social network analysis (SNA). Such a solution faces a few challenges. Th
Brown University - Data Management Group » A Comparison of Approaches to Large-Scale Data Analysis: MapReduce vs. DBMS Benchmarks
Overview The following information is meant to provide documentation on how others can recreate the benchmark trials used in our SIGMOD 2009 paper. Our experiments were conducted on a 100-node cluster at the University of Wisconsin-Madison; each node had a single 2.40 GHz Intel Core 2 Duo processor running 64-bit Red Hat Enterprise Linux 5 (kernel version 2.6.18) with 4GB RAM and two 250GB SATA-I
Cascading
Please note that all new project news and releases have moved to https://cascading.wensel.net The Cascading Ecosystem is a collection of applications, languages, and APIs for developing data-intensive applications. At the ecosystem core is Cascading, a Java API for defining complex data flows and integrating those flows with back-end systems, and a query planner for mapping and executing logical f
Apache Avroâ„¢ 1.8.2 Documentation
Apache Avroâ„¢ is a data serialization system. Avro provides: Rich data structures. A compact, fast, binary data format. A container file, to store persistent data. Remote procedure call (RPC). Simple integration with dynamic languages. Code generation is not required to read or write data files nor to use or implement RPC protocols. Code generation as an optional optimization, only worth implementi
Hive/LanguageManual/Joins - Hadoop Wiki
Join Syntax Hive supports the following syntax for joining tables: join_table: table_reference JOIN table_factor [join_condition] | table_reference {LEFT|RIGHT|FULL} [OUTER] JOIN table_reference join_condition table_reference: table_factor | join_table table_factor: tbl_name [alias] | table_subquery alias | ( table_references ) join_condition: ON equality_expression ( AND equality_expression )* e
HadoopDB Project
HadoopDB An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads. HadoopDB is: A hybrid of DBMS and MapReduce technologies that targets analytical workloads Designed to run on a shared-nothing cluster of commodity machines, or in the cloud An attempt to fill the gap in the market for a free and open source parallel DBMS Much more scalable than currently available parall
Cloudera Blog
We are excited to announce the acquisition of Octopai, a leading data lineage and catalog platform that provides data discovery and governance for enterprises to enhance their data-driven decision making. Cloudera’s mission since its inception has been to empower organizations to transform all their data to deliver trusted, valuable, and predictive insights. With AI and […] Read blog post
Cloudera Blog
It’s hard to believe it’s been 15 years since the global financial crisis of 2007/2008. While this might be a blast from the past we’d rather leave in the proverbial rear-view mirror, in March of 2023 we were back to the future with the collapse of Silicon Valley Bank (SVB), the largest US bank to […] Read blog post
Hadoopã®ç´¹ä»‹
3. è¬è¾ž 以下ã®ãƒ›ãƒ¼ãƒ ページをå‚照＆資料を貸ã—ã¦ã„ãŸã ãã¾ã—㟠http://hadoop.apache.org http://developer.yahoo.net/blogs/hadoop/2008/02/yahoo-worlds-largest-production-hadoop.html http://www.techcrunch.com/2008/01/09/google-processing-20000-terabytes-a-day-and-growing/ http://codezine.jp/article/detail/2448?p=1 http://www.slideshare.net/kakuda/hadoop/ 4. outline what is Hadoop? Hadoop distributed file system Hadoop mapreduce H
Rubyã§Hadoopをラップã€åˆ†æ•£å‡¦ç†ãƒ„ールã‚ットãŒç™»å ´ ï¼ ï¼ IT
2009/05/12 米新èžç¤¾å¤§æ‰‹ã®ãƒ‹ãƒ¥ãƒ¼ãƒ¨ãƒ¼ã‚¯ãƒ»ã‚¿ã‚¤ãƒ ズã¯5月11æ—¥ã€Rubyã«ã‚ˆã‚‹å¤§è¦æ¨¡åˆ†æ•£å‡¦ç†ã®ãƒ„ールã‚ット「Map/Reduce Toolkitã€ï¼ˆMRToolkit）をGPLv3ã®ä¸‹ã«ã‚ªãƒ¼ãƒ—ンソースã§å…¬é–‹ã—ãŸã¨ç™ºè¡¨ã—ãŸã€‚MRToolkitã¯ã€ã™ã§ã«ç¨¼åƒã—ã¦ã„るクラスタ上ã®Hadoopã¨åˆã‚ã›ã¦ä½¿ã†ã“ã¨ã§Rubyã§å®¹æ˜“ã«Map/Reduce処ç†ã‚’記述ã™ã‚‹ã“ã¨ãŒã§ãる一種ã®ãƒ©ãƒƒãƒ‘ー。処ç†è‡ªä½“ã¯HadoopãŒè¡Œã†ã€‚ã™ã§ã«Hadoopを使ã£ã¦ã„るユーザーã§ã‚ã‚Œã°ã€ä¸å°è¦æ¨¡ã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã«å¯¾ã—ã¦ã€ã™ãã«MRToolkitã‚’é©ç”¨å¯èƒ½ã¨ã—ã¦ã„る。 デフォルトã§æœ‰ç”¨ãªMapã€Reduceã®å‡¦ç†ãƒ¢ã‚¸ãƒ¥ãƒ¼ãƒ«ãŒå«ã¾ã‚Œã¦ã„ã¦ã€æ•°è¡Œã®Rubyスクリプトを書ãã ã‘ã§ã€ä¾‹ãˆã°è†¨å¤§ãªApacheã®ãƒã‚°ã‹ã‚‰IPアドレス別ã®é–²è¦§å±¥æ´ã‚’ã¾ã¨ã‚ã‚‹ã¨ã„ã£ãŸå‡¦ç†ãŒå¯èƒ½ã¨ã„ã†ã€‚独自ã«Mapã‚„Reduceã®å‡¦ç†ã‚’定義ã™ã‚‹ã“ã¨ã‚‚
Hadoopã§ã€ã‹ã‚“ãŸã‚“åˆ†æ•£å‡¦ç† (Yahoo! JAPAN Tech Blog)
ãƒ¤ãƒ•ãƒ¼æ ªå¼ä¼šç¤¾ã¯ã€2023å¹´10月1æ—¥ã«LINEãƒ¤ãƒ•ãƒ¼æ ªå¼ä¼šç¤¾ã«ãªã‚Šã¾ã—ãŸã€‚LINEãƒ¤ãƒ•ãƒ¼æ ªå¼ä¼šç¤¾ã®æ–°ã—ã„ブãƒã‚°ã¯ã“ã¡ã‚‰ã§ã™ã€‚LINEヤフー Tech Blog ã“ã‚“ã«ã¡ã¯ã€åœ°åŸŸã‚µãƒ¼ãƒ“ス事æ¥éƒ¨ã®å‰ç”°ä¸€æ˜Ÿã§ã™ã€‚ 今回ã¯ã€Hadoopã«ã¤ã„ã¦ã€Yahoo! JAPANã§ã®å®Ÿéš›ã®ä½¿ç”¨ä¾‹ã‚’交ãˆãªãŒã‚‰æ›¸ããŸã„ã¨æ€ã„ã¾ã™ã€‚Hadoopã¨ã¯ã€å¤§é‡ã®ãƒ‡ãƒ¼ã‚¿ã‚’手軽ã«è¤‡æ•°ã®ãƒžã‚·ãƒ³ã«åˆ†æ•£ã—ã¦å‡¦ç†ã§ãるオープンソースã®ãƒ—ラットフォームã§ã™ã€‚ 複数ã®ãƒžã‚·ãƒ³ã¸ã®åˆ†æ•£å‡¦ç†ã¯ã€ãƒ—ãƒã‚»ã‚¹é–“通信やã€éšœå®³æ™‚ã¸ã®å¯¾å¿œãªã©ã‚’考ãˆãªã‘ã‚Œã°ãªã‚‰ãšã€ãƒ—ãƒã‚°ãƒ©ãƒžã«ã¨ã£ã¦æ•·å±…ãŒé«˜ã„ã‚‚ã®ã§ã™ãŒã€ Hadoopã¯ãã†ã„ã£ãŸé¢å€’ãã•ã„分散処ç†ã‚’一手ã«å¼•ãå—ã‘ã¦ãã‚Œã¾ã™ã€‚ 1å°ã§ã¯å‡¦ç†ã«ã‹ãªã‚Šæ™‚é–“ãŒã‹ã‹ã‚‹ã‚ˆã†ãªå¤§é‡ã®ãƒ‡ãƒ¼ã‚¿ã‚‚ã€è¤‡æ•°ãƒžã‚·ãƒ³ã«åˆ†æ•£ã•ã›ã‚‹ã“ã¨ã§ã€é©šãã¹ãスピードã§å‡¦ç†ã‚’è¡Œã†ã“ã¨ãŒã§ãã¾ã™ã€‚ 例ãˆã°ã€ä»Šã¾ã§1å°ã§ã‚„ã£ã¦ã„ãŸã€ã‚ã‚‹ãƒã‚°é›†è¨ˆå‡¦ç†
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
hbaseexplorer
xadoop
Google Code Archive - Long-term storage for Google Code Project Hosting.
mrtoolkit - Google Code
Home - Open Cirrus
http://www.cloudera.com/hadoop-training-mapreduce-hdfs
Open Source & Open Standards | Cloudera
{{#tags}}- {{label}}
{{/tags}}