Breaking the SQL Barrier: Google BigQuery User-Defined Functions- Google Developers Blog
Share Facebook Twitter LinkedIn Mail Posted by, Thomas Park, Senior Software Engineer, Google BigQuery Many types of computations can be difficult or impossible to express in SQL. Loops, complex conditionals, and non-trivial string parsing or transformations are all common examples. What can you do when you need to perform these operations but your data lives in a SQL-based Big data tool? Is it po
ãªãœç§ãŸã¡ã¯Sumo Logicã‚’æ¨ã¦ã¦BigQueryã‚’é¸ã‚“ã ã®ã‹ - tech.guitarrapc.cóm
ãƒã‚°åˆ†æžã‚µãƒ¼ãƒ“スã¯ã‚¢ãƒ—リケーションã®ã‚¤ãƒ³ãƒ•ãƒ©ã§ã‚ã‚Šã€ã‚µãƒ¼ãƒ“ス開発/é‹ç”¨ã®ä¸ã§é‡è¦ãªä½ç½®ã‚’å ã‚ã¾ã™ã€‚グラニã§ã¯ã€ä»Šå¹´ã«å…¥ã£ã¦åˆ©ç”¨ã—ã¦ã„ã‚‹ãƒã‚°åˆ†æžã‚µãƒ¼ãƒ“スを〠Sumo Logic ã‹ã‚‰ã€€Google BigQuery ã«å®Œå…¨ç§»è¡Œã—ã¾ã—ãŸã€ 本記事ã¯ã€ã‚°ãƒ©ãƒ‹ã§è°è«–ã•ã‚ŒãŸã€Œãƒã‚°åˆ†æžã‚µãƒ¼ãƒ“スã¨ã—ã¦ã® SumoLogic 㨠BigQueryã€ã®ã¾ã¨ã‚を推敲ã€è»¢è¼‰ã—ãŸã‚‚ã®ã§ã™ã€‚ã“ã‚Œã‹ã‚‰ãƒã‚°åˆ†æžã‚µãƒ¼ãƒ“スを検討ã•ã‚Œã‚‹æ–¹ã€…ã«ã¨ã£ã¦ã€è°è«–ã®å†…容ãŒå°‘ã—ã§ã‚‚å‚考ã«ãªã‚‹ã“ã¨ã‚’願ã„公開ã—ã¾ã™ã€‚ アジェンダ ã¾ãšã¯æ–‡è„ˆã‚’æ•´ãˆã‚‹ãŸã‚ã«ã‚¢ã‚¸ã‚§ãƒ³ãƒ€ã‹ã‚‰ã€‚ アジェンダ 日常的ãªã‚¢ãƒ—リケーション監視フãƒãƒ¼ APM ã¨ã—ã¦ç›¤çŸ³ãª New Relic ãƒã‚°åˆ†æžã‚µãƒ¼ãƒ“スã«ã‚ˆã‚‹ã‚¢ãƒ‰ãƒ›ãƒƒã‚¯ãªãƒã‚°åˆ†æž ãƒã‚°åˆ†æžã‚µãƒ¼ãƒ“スã«æ±‚ã‚ã‚‹ã“㨠Sumo Logic ã®åˆ©ç”¨ã¨èª²é¡Œ Sumo Logic ã®åˆ©ç‚¹ Sumo Logicã§ç™ºç”Ÿã—ãŸèª²é¡Œ ãƒã‚°åŽ
BigQuery JavaScript User-Defined Functions by THOMAS PARK and FELIPE HOFFA at Big Data Spain 2014
BigQuery JavaScript User-Defined Functions by THOMAS PARK and FELIPE HOFFA at Big Data Spain 2014 Performing ETL on big data can be slow, expensive and painful - but it doesn't have to be! In this session, we'll take an in-depth look at several real-world examples of computations that don't fit well with the SQL language model and how to solve them with user-defined functions in Google BigQuery. S
Dataproc | Google Cloud
Dataproc is a fully managed and highly scalable service for running Apache Hadoop, Apache Spark, Apache Flink, Presto, and 30+ open source tools and frameworks. Use Dataproc for data lake modernization, ETL, and secure data science, at scale, integrated with Google Cloud, at a fraction of the cost. Flexible: Use serverless, or manage clusters on Google Compute and Kubernetes. Deploy a Google-recom
Google BigQueryã«ã¤ã„㦠紹介ã¨æŽ¨æ¸¬
4. What s Next • 著者ã®æ–¹ãŒå¤§å¤‰è¦ªåˆ‡ã€ã‹ã¤BigQuery æ„›ãŒæ„Ÿã˜ã‚‰ã‚Œã¾ã™ • ç§ï¼šã€Œã“ã“ã€å‹•ã‹ãªã„ã‚“ã ã‘ã©ã€ Tigani：「ã‚ã€æ›¸ã„ãŸæ™‚ã«ã¯ãれ実装 ã•ã‚Œã‚‹äºˆå®šã ã£ãŸã‚“ã ã‚。çµå±€ãƒœãƒ„ã« ãªã£ã¦å®Ÿè£…ã•ã‚Œãªã‹ã£ãŸã‚“ã§ã€å‰Šã£ã¦ ãã‚ŒãŸã‚‰ã„ã„よ〠ç§ï¼šã€Œã•ã„ã§ã™ã‹ã€ • ãŸã¶ã‚“3月発売!!
Google BigQuery ã®è©± #yapcasia // Speaker Deck
フãƒãƒ³ãƒˆã‚¨ãƒ³ãƒ‰ã®ãƒ‘ラダイムをå‚考ã«ãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰é–‹ç™ºã‚’å†è€ƒã™ã‚‹ / TypeScript ã«ã‚ˆã‚‹ GraphQL ãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰é–‹ç™º
MPP on Hadoop, Redshift, BigQuery - Go ahead!
Twitterã§ã€Œæ—©ã今æµè¡Œã®MPPã®å¤§ã¾ã‹ãªä½¿ã„æ–¹ã®é•ã„書ã‘よï¼ã€ã¨ã„ã†ãƒ—レッシャーãŒåŠç«¯ãªã„ã®ã§ã¦ãã¨ã†ã«æ›¸ãã¾ã™ï¼Žã“ã®è¨˜äº‹ã¯ä¿ºã®çµŒé¨“ã¨å‹‰å¼·ä¼šãªã©ã§ãƒ¦ãƒ¼ã‚¶ã‹ã‚‰èžã„ãŸè©±ã‚’ã‚‚ã¨ã«æ›¸ã„ã¦ã„ã‚‹ã®ã§ï¼Œã™ã¹ã¦ãŒä¿ºã®çµŒé¨“ã§ã¯ã‚ã‚Šã¾ã›ã‚“(特ã«BigQuery)．å„社ã®SAã®äººã¨ã‹ã«èžã‘ã°ï¼Œã‚‚ã£ã¨è‰¯ã„アプãƒãƒ¼ãƒã¨ã‹è©³ç´°ã‚’æ•™ãˆã¦ãれるã‹ã‚‚ã—ã‚Œã¾ã›ã‚“． オンプレミスã®å•†ç”¨MPPã¯ä½¿ã£ãŸã“ã¨ãªã„ã®ã§ãƒŽãƒ¼ã‚³ãƒ¡ãƒ³ãƒˆã§ã™ï¼Ž MPP on Hadoopã§PrestoãŒãƒ¡ã‚¤ãƒ³ãªã®ã¯ä»Šä¸€ç•ªä½¿ã£ã¦ã„ã‚‹ã‹ã‚‰ã§ï¼ŒImpalaãªã©ä»–ã®MPP on Hadoopçš„ãªã‚‚ã®ã‚‚ä¼¼ãŸã‚ˆã†ãªæ„Ÿã˜ã‹ãªã¨æ€ã£ã¦ã„ã¾ã™ï¼Ž ã‚‚ã¡ã‚ん実装ã®é•ã„ãªã©ãŒã‚ã‚‹ã®ã§ï¼Œãã®è¾ºã¯é©å®œè‡ªåˆ†ã§è£œé–“ã—ã¦ãã ã•ã„． å‰æ アプリケーションを開発ã—ã¦ã„ã¦ï¼Œãã®ãŸã‚ã®è§£æžåŸºç›¤ã‚’一ã‹ã‚‰ä½œã‚‹ï¼Ž ç°¡å˜ãªã¾ã¨ã‚ データを貯ã‚る所ãŒä½œã‚Œã‚‹ã®ã§ã‚ã‚Œã°ï¼Œãã“ã«ç›´æŽ¥ã‚¯ã‚¨ãƒªã‚’投ã’られるPre
BigQueryã«Twitterã®ã‚¿ã‚¤ãƒ ラインを入れる [Ruby]
Twitter ã®ã‚¿ã‚¤ãƒ ラインをä¿å˜ã—ã¦ãŠãã¨ãªã«ã‹ã¨ä¾¿åˆ©ãªã®ã§ã€è‰²ã€…ã¨ä¿å˜å½¢å¼ã‚’変ãˆãªãŒã‚‰ 4 å¹´ãらã„記録ã—続ã‘ã¦ã„る。ツイートã®ä¿å˜ãŒä¾¿åˆ©ã™ãŽã‚‹ã®ã§ã€ãƒ„イセーブã¨ã„ã†ã‚µãƒ¼ãƒ“ス化ã¾ã§ã—ãŸã€‚ã‹ã¤ã¦ã¯ãƒ†ã‚ストã§ã€MongoDB ã‚„ MySQL ã¨ã‹ Groonga ã¨ã‹ã„ã‚ã„ã‚ã‚„ã£ã¦ããŸã€‚ã©ã‚Œã‚‚å•é¡Œãªã„ã‚“ã ã‘ã©ã€å¢—ãˆç¶šã‘ã‚‹ãƒã‚°ãƒ‡ãƒ¼ã‚¿ã‚’ä¿å˜ã™ã‚‹å ´æ‰€ã¨ã—ã¦è€ƒãˆã‚‹ã¨ BigQuery ãŒç¾ä»£ã«ãƒžãƒƒãƒã—ã¦ã‚‹ã‚ˆã†ãªã®ã§ãã¡ã‚‰ã«ç§»è¡Œã—ãŸã€‚ BigQuery ã« TL ã‚’ä¿å˜ã™ã‚‹ã¨ã§ãã‚‹ã“㨠TL ã®å…¨ã¦ã®ãƒ‡ãƒ¼ã‚¿ã‚’フルスã‚ャンã§ãる。ã“ã‚Œã¯ã‹ãªã‚Šä¾¿åˆ©ã§ã€ä»Šå›žã“ã®ãƒ–ãƒã‚°è¨˜äº‹ã‚’書ãã«ã‚ãŸã£ã¦ã‚‚ ‘BigQuery’ ã‚’ TL ã‹ã‚‰æ¤œç´¢ã™ã‚Œã°ã€ä¿¡é ¼ã§ãるフォãƒãƒ¼ã‚¤ãƒ³ã‚°ã®äººã€…ã®å£°ã‚’見るã“ã¨ãŒã§ããŸã€‚ã“ã‚Œã«ã‚ˆã‚Šã€ŒæŸ CA 社ã§ã¯ 5000 å°ã® MongoDB クラスタ㧠BigQuery ã«å¯¾æŠ—ã—ã¦ã„ã‚‹ã€ã¨ã„ã†
![BigQueryã«Twitterã®ã‚¿ã‚¤ãƒ ラインを入れる [Ruby]](https://cdn-ak-scissors.b.st-hatena.com/image/square/eb99bf7e3074ef0fe5df0ca25a644e696c62ad78/height=288;version=1;width=512/https%3A%2F%2Fs3-ap-northeast-1.amazonaws.com%2Fkksg%2Fblog%2F53b4ac19c38aa5513f47ea75.png)
Googleã®è™Žã®å「BigQueryã€ã‚’FluentdユーザーãŒä½¿ã‚ãªã„ç†ç”±ãŒãªããªã£ãŸç†ç”± #gcpja - Qiita
From Fluentd Meetupã«è¡Œã£ã¦ãã¾ã—㟠ã“れをèªã‚“ã 時ã€BigQueryã®æ¤œç´¢ã‚¹ãƒ”ードã«ã¤ã„ã¦ã¡ã‚‡ã£ã¨è£œè¶³ã—ãŸããªã£ãŸã€‚確ã‹ã«Fluentd Meetupã®ãƒ‡ãƒ¢ã§ã¯9億件を7秒程度ã§æ¤œç´¢ã—ã¦ã„ãŸãŒã€BigQueryã®çœŸã®å®ŸåŠ›ã¯ã“れより1〜2ケタ上ã ã‹ã‚‰ã 。ã¡ã‚‡ã£ã¨æ‰‹å…ƒã§å°‘ã—大ãã‚ã®ãƒ†ãƒ¼ãƒ–ルã§è©¦ã—ã¦ã¿ãŸã‚‰ã€120å„„è¡Œã®æ£è¦è¡¨ç¾ãƒžãƒƒãƒä»˜ã集計ãŒ5秒ã§å®Œäº†ã—ãŸã€‚è«–ã‚ˆã‚Šè¨¼æ‹ ã§ã€ãƒ‡ãƒ¢ãƒ“デオ（1分16秒）を作ã£ã¦ã¿ãŸï¼š From The Speed of Google BigQuery ã“ã‚Œã¯é€Ÿã™ãŽã‚‹ã€‚何ã‹ã®ã‚¤ãƒ³ãƒã‚ã§ã‚る（最åˆã«ãƒ‡ãƒ¢ã‚’見ãŸæ™‚ãã†æ€ã£ãŸï¼‰ã€‚æ£è¦è¡¨ç¾ã‚’ã„ã‚ã„ã‚変ãˆã¦ã¿ã¦ã‚‚スピードã¯å¤‰ã‚らãªã„。ã¤ã¾ã‚Šã€ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ã‚’事å‰æ§‹ç¯‰ã§ããªã„クエリã«å¯¾ã—ã¦ã“ã®ã‚¹ãƒ”ードãªã®ã§ã‚る。 ä¾¡æ ¼ã‚‚å®‰ã„。ã•ã™ãŒã«120å„„è¡Œã®ã‚¯ã‚¨ãƒªã¯1回ã§200円もã‹ã‹ã£ã¦æ°—軽ã«å®Ÿè¡Œã§ããªã•ãã†ã§ã‚ã‚‹ãŒã€1.2å„„
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
{{#tags}}- {{label}}
{{/tags}}