Incubating Apache Hivemall
Hivemall is an open source machine learning library built as a collection of Hive UDFs. It provides over 100 machine learning algorithms and functions for tasks like feature engineering, evaluation, and recommendation. Hivemall entered the Apache Incubator in 2016 and the first Apache release (v0.5.0) is upcoming. It supports platforms like Hive, Spark, and Pig for scalable parallel processing.Rea
History of Event Collector in Treasure Data
The Event Collector system was one of the legacy systems at Treasure Data. Over time it faced several performance and scalability problems as usage increased. Engineers addressed these problems through optimizations like increasing socket backlogs, caching parsers, running processes in parallel, and moving deduplication to a separate thread to avoid blocking the input pipeline. These changes helpe
Planet-scale Data Ingestion Pipeline: Bigdam
Bigdam is a planet-scale data ingestion pipeline designed for large-scale data ingestion. It addresses issues with the traditional pipeline such as imperfectqueue throughput limitations, latency in queries from event collectors, difficulty maintaining event collector code, many small temporary and imported files. The redesigned pipeline includes Bigdam-Gateway for HTTP endpoints, Bigdam-Pool for d
User Defined Partitioning on PlazmaDB
User defined partitioning is a new partitioning strategy in Treasure Data that allows users to specify which column to use for partitioning, in addition to the default "time" column. This provides more flexible partitioning that better fits customer data platform workloads. The user can define partitioning rules through Presto or Hive to improve query performance by enabling colocated joins and fi
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
処ç†ã‚’実行ä¸ã§ã™
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
Journey of Migrating Millions of Queries on The Cloud
{{#tags}}- {{label}}
{{/tags}}