[B! databricks] manboubirdã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

manboubird id:manboubird

databricksã«é–¢ã™ã‚‹manboubirdã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (40)

${{author_name}}$

{{author_name}} {{created}}

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

{{#following_bookmarks}}

${{author_name}}$

{{author_name}} {{created}}

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

{{/following_bookmarks}}

{{/is_wiped}}

æ–°èˆˆ PostgreSQL ã‚µãƒ¼ãƒ“ã‚¹ä¼æ¥ã®è²·åŽåŠ‡ã®èƒŒæ™¯ï¼šSnowflake ã¨ Databricks ãŒæãæœªæ¥ã«æ€ã„ã‚’é¦³ã›ã¦ã¿ãŸ
æ–°èˆˆ PostgreSQL ã‚µãƒ¼ãƒ“ã‚¹ä¼æ¥ã®è²·åŽåŠ‡ã®èƒŒæ™¯ï¼šSnowflake ã¨ Databricks ãŒæãæœªæ¥ã«æ€ã„ã‚’é¦³ã›ã¦ã¿ãŸ æ—¥æœ¬ã® PostgreSQL ã‚³ãƒŸãƒ¥ãƒ‹ãƒ†ã‚£ãŒä»Šæœˆã€å¤§ããªç››ã‚Šä¸ŠãŒã‚Šã‚’è¦‹ã›ã¦ã„ã¾ã™ã€‚ãã®ç†ç”±ã¯ã€ãƒ‡ãƒ¼ã‚¿ãƒ»AI æ¥ç•Œã®äºŒå¤§å·¨é ã§ã‚ã‚‹ Snowflake ã¨ Databricks ãŒã€ç«‹ã¦ç¶šã‘ã« PostgreSQL ã®ãƒžãƒãƒ¼ã‚¸ãƒ‰ã‚µãƒ¼ãƒ“ã‚¹ãƒ—ãƒãƒã‚¤ãƒ€ã®æ–°èˆˆä¼æ¥ã‚’è²·åŽã—ã€ä¸¡ç¤¾ãŒ PostgreSQL ã®ãƒžãƒãƒ¼ã‚¸ãƒ‰ã‚µãƒ¼ãƒ“ã‚¹æä¾›ã‚’é–‹å§‹ã™ã‚‹ã¨ç™ºè¡¨ã—ãŸãŸã‚ã§ã™ã€‚ã“ã®å‹•ãã¯ã€ PostgreSQL ãŒã“ã‚Œã¾ã§ä»¥ä¸Šã«æœ€ã‚‚é‡è¦ãªãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã®ä¸€ã¤ã¨ã—ã¦ã®åœ°ä½ã‚’ç¢ºç«‹ã—ã€PostgreSQL ãƒ¦ãƒ¼ã‚¶ãƒ¼ã®ã‚¹ã‚ãƒ«ãŒæ´»ã‹ã›ã‚‹å ´ãŒå¤§ããåºƒãŒã£ã¦ã„ã‚‹ã“ã¨ã‚’ç¤ºå”†ã—ã¦ã„ã¾ã™ã€‚ ãªãœã€ã“ã®è²·åŽåŠ‡ãŒç«‹ã¦ç¶šã‘ã«èµ·ããŸã®ã§ã—ã‚‡ã†ã‹ã€‚è€ƒãˆã‚‰ã‚Œã‚‹ä¸»ãªç†ç”±ã¯ä»¥ä¸‹ã® 3 ç‚¹ã§ã™ã€‚ 1. ã‚¢ãƒŠãƒªãƒ†ã‚£ã‚¯ã‚¹ãƒ—ãƒ©ãƒƒãƒˆãƒ•ã‚©ãƒ¼
manboubird 2025/06/16
databricks

snowflake

postgres
ãƒªãƒ³ã‚¯
ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹å¸‚å ´ã®â€œã‚²ãƒ¼ãƒ ãƒã‚§ãƒ³ã‚¸ãƒ£ãƒ¼â€ã¨ãªã‚‹ã‹ï¼Ÿ DatabricksãŒAIæ™‚ä»£ã®æ–°DBã€ŒLakebaseã€ã‚’ç™ºè¡¨
ãƒ¬ãƒãƒ¼ãƒˆ ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹å¸‚å ´ã®â€œã‚²ãƒ¼ãƒ ãƒã‚§ãƒ³ã‚¸ãƒ£ãƒ¼â€ã¨ãªã‚‹ã‹ï¼Ÿ DatabricksãŒAIæ™‚ä»£ã®æ–°DBã€ŒLakebaseã€ã‚’ç™ºè¡¨ 2025å¹´6æœˆ11æ—¥ã€ç±³å›½ã‚µãƒ³ãƒ•ãƒ©ãƒ³ã‚·ã‚¹ã‚³ã§Databricksã®å¹´æ¬¡ã‚¤ãƒ™ãƒ³ãƒˆã€ŒDATA + AI SUMMITã€ãŒé–‹å‚¬ã•ã‚ŒãŸã€‚Databricks CEOã®ã‚¢ãƒªãƒ»ã‚´ãƒ‡ã‚£ã‚·æ°ãŒã€Œåœ°çƒä¸Šã§æœ€å¤§è¦æ¨¡ã®ãƒ‡ãƒ¼ã‚¿ãŠã‚ˆã³AIã‚«ãƒ³ãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã€ã¨ç§°ã™ã‚‹æœ¬ã‚¤ãƒ™ãƒ³ãƒˆã®åˆæ—¥åŸºèª¿è¬›æ¼”ã«ã¯ã€ä¸–ç•Œ150ã‚«å›½ã‹ã‚‰2ä¸‡2000äººä»¥ä¸ŠãŒä¼šå ´ã«é›†ã„ã€ã‚ªãƒ³ãƒ©ã‚¤ãƒ³ã‚’å«ã‚ã‚‹ã¨6ä¸‡5000äººä»¥ä¸ŠãŒå‚åŠ ã—ãŸã€‚è¬›æ¼”ã§ã¯ã€DatabricksãŒæŽ²ã’ã‚‹ã€Œãƒ‡ãƒ¼ã‚¿ã¨AIã®æ°‘ä¸»åŒ–ã€ã¨ã„ã†ãƒŸãƒƒã‚·ãƒ§ãƒ³ã‚’æŽ¨é€²ã™ã‚‹ãŸã‚ã®ã€ã•ã¾ã–ã¾ãªæ–°ç™ºè¡¨ãŒè¡Œã‚ã‚ŒãŸã€‚ Databricks CEOã®ã‚¢ãƒªãƒ»ã‚´ãƒ‡ã‚£ã‚·æ° è¤‡é›‘ãªãƒ‡ãƒ¼ã‚¿åŸºç›¤ã®èª²é¡Œã«çµ‚æ¢ç¬¦ã‚’ã€æ–°DBã€ŒLakebaseã€ç™»å ´ å†’é ã€ã‚´ãƒ‡ã‚£ã‚·æ°ã¯ä¼æ¥ãŒç›´é¢ã™ã‚‹èª²é¡Œã¨ã—ã¦ã€ãƒ‡ãƒ¼ã‚¿ã¨AIã‚¤ãƒ³ãƒ•ãƒ©ã®
manboubird 2025/06/16
databricks

lakebase
ãƒªãƒ³ã‚¯
CIDR Proceedings
manboubird 2025/06/08
Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics

databricks

lakehouse

deltalake

dataWareHouse

dataManagement

cidr

paper
ãƒªãƒ³ã‚¯
Native Delta Lake Support in DuckDB
TL;DR: DuckDB now has native support for Delta Lake, an open-source lakehouse framework, with the delta extension. Over the past few months, DuckDB Labs has teamed up with Databricks to add first-party support for Delta Lake in DuckDB using the new delta-kernel-rs project. In this blog post we'll give you a short overview of Delta Lake, Delta Kernel and, of course, present the new DuckDB Delta ext
manboubird 2025/06/08
duckdb

databricks

deltalake

paper
ãƒªãƒ³ã‚¯
Announcing DuckDB Support for Delta Lake and a DuckDB Extension to Unity Catalog - Hannes MuÌˆhleisen
manboubird 2025/06/08
duckdb

databricks

video
ãƒªãƒ³ã‚¯
DSPyã§Databricksã®LLMãƒ‘ã‚¤ãƒ—ãƒ©ã‚¤ãƒ³ã‚’æœ€é©åŒ–ã—ã¾ã—ã‚‡ã†ï¼
éŽåŽ» 1 å¹´é–“ã€æ¥ç•Œãƒ¬ãƒ™ãƒ«ã®LLMãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã®ä¸–ç•Œã‚’ãƒ•ã‚©ãƒãƒ¼ã—ã¦ããŸæ–¹ãªã‚‰ã€æœ¬ç•ªé‹ç”¨ã«ãŠã‘ã‚‹ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã‚„ãƒ„ãƒ¼ãƒ«ã®å¤šã•ã«æ°—ä»˜ã„ã¦ã„ã‚‹ã§ã—ã‚‡ã†ã€‚ã‚¹ã‚¿ãƒ¼ãƒˆã‚¢ãƒƒãƒ—ä¼æ¥ã¯ã€æ¤œç´¢æ‹¡å¼µç”Ÿæˆ(RAG)ã®è‡ªå‹•åŒ–ã‹ã‚‰ã‚«ã‚¹ã‚¿ãƒ ãƒ•ã‚¡ã‚¤ãƒ³ãƒãƒ¥ãƒ¼ãƒ‹ãƒ³ã‚°ã¾ã§ã€å¤šæ§˜ãªLLMé–¢é€£ã‚µãƒ¼ãƒ“ã‚¹ã‚’æ¬¡ã€…ã«é–‹ç™ºã—ã¦ã„ã¾ã™ã€‚Langchainã¯ã€ãŠãã‚‰ãã“ã‚Œã‚‰ã™ã¹ã¦ã®æ–°ã—ã„ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã®ä¸ã§æœ€ã‚‚æœ‰åã§ã€2023 å¹´æ˜¥ä»¥é™ã€é€£éŽ–è¨€èªžãƒ¢ãƒ‡ãƒ« ã‚³ãƒ³ãƒãƒ¼ãƒãƒ³ãƒˆã®ç°¡å˜ãªãƒ—ãƒãƒˆã‚¿ã‚¤ãƒ—ä½œæˆã‚’å¯èƒ½ã«ã—ã¦ã„ã¾ã™ã€‚ãŸã ã—ã€æœ€è¿‘ã®é‡è¦ãªé–‹ç™ºã¯ã€ã‚¹ã‚¿ãƒ¼ãƒˆã‚¢ãƒƒãƒ—ä¼æ¥ã‹ã‚‰ã§ã¯ãªãã€å¦è¡“ç•Œã‹ã‚‰ç”Ÿã¾ã‚Œã¾ã—ãŸã€‚ 2023å¹´10æœˆã€Databricksã®å…±åŒå‰µè¨è€…ã§ã‚ã‚‹Matei Zahariaã®ã‚¹ã‚¿ãƒ³ãƒ•ã‚©ãƒ¼ãƒ‰ç ”ç©¶å®¤ã®ç ”ç©¶è€…ãŸã¡ãŒã€å®£è¨€åž‹ã®è¨€èªžãƒ¢ãƒ‡ãƒ«ã‚³ãƒ¼ãƒ«ã‚’è‡ªå·±æ”¹å–„åž‹ãƒ‘ã‚¤ãƒ—ãƒ©ã‚¤ãƒ³ã«ã‚³ãƒ³ãƒ‘ã‚¤ãƒ«ã™ã‚‹ãŸã‚ã®ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã§ã‚ã‚‹DSPyã‚’ç™ºè¡¨ã—ã¾ã—ãŸã€‚DSPyã®ä¸»è¦ãªè¦ç´ ã¯è‡ª
manboubird 2025/06/05
dspy

prompt

promptEngineering

llm

mlops

databricks
ãƒªãƒ³ã‚¯
Databricks Mosaic AIã«ã‚ˆã‚‹LLM ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã®è©•ä¾¡ - every Tech Blog
ã“ã®è¨˜äº‹ã¯ every Tech Blog Advent Calendar 2024 ã® 16æ—¥ç›®ã®è¨˜äº‹ã§ã™ã€‚ ã¯ã˜ã‚ã« ã“ã‚“ã«ã¡ã¯ã€‚ æ ªå¼ä¼šç¤¾ã‚¨ãƒ–ãƒªãƒ¼ã®é–‹ç™ºæœ¬éƒ¨ãƒ‡ãƒ¼ã‚¿&AIãƒãƒ¼ãƒ (DAI)ã§ãƒ‡ãƒ¼ã‚¿ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã‚’ã—ã¦ã„ã‚‹å‰ç”°ã§ã™ã€‚ ä»Šå›žã¯ã€Databricks Mosaic AIã«ã‚ˆã‚‹LLM ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã®è©•ä¾¡ã«ã¤ã„ã¦ã®ãŠè©±ã§ã™ã€‚ èƒŒæ™¯ è¿‘å¹´ã€LLMã‚’åˆ©ç”¨ã—ãŸã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ãŒå¢—ãˆã¦ãŠã‚Šã€DELISH KITCHENã§ã‚‚AIã«ã‚ˆã‚‹æ–™ç†ã‚¢ã‚·ã‚¹ã‚¿ãƒ³ãƒˆã¨ã—ã¦ã€Œãƒ‡ãƒªãƒƒã‚·ãƒ¥AIã€ã®æä¾›ã‚’é–‹å§‹ã—ã¾ã—ãŸã€‚ ãã®ã‚ˆã†ãªçŠ¶æ³ã®ä¸ã§ã€ã‚µãƒ¼ãƒ“ã‚¹ã«LLMã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã‚’çµ„ã¿è¾¼ã‚€éš›ã«ã¯ã€ã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã®è©•ä¾¡ãŒã¾ã™ã¾ã™é‡è¦ãªèª²é¡Œã¨ãªã£ã¦ã„ã¾ã™ã€‚ ã—ã‹ã—ã€LLMã‚¢ãƒ—ãƒªã‚±ãƒ¼ã‚·ãƒ§ãƒ³ã®å“è³ªã¯ã€ãƒ‡ãƒ¼ã‚¿ã®è³ªã€ãƒ¢ãƒ‡ãƒ«ã®æ€§èƒ½ã€ãƒ—ãƒãƒ³ãƒ—ãƒˆã€retrieverã®æ€§èƒ½ãªã©è¤‡æ•°ã®è¦ç´ ãŒå½±éŸ¿ã™ã‚‹ãŸã‚ã€è©•ä¾¡ã¯è¤‡é›‘ã§é›£ã—ã„èª²é¡Œã§ã™ã€‚ ã¾ãŸã€
manboubird 2025/05/19
mlflow

mlOps

mosaicAi

databricks

generativeAi

evaluation
ãƒªãƒ³ã‚¯
Next Generation Data Pipelines | Fennel
manboubird 2025/05/19
fennel

mlOps

databricks

featureEngineering
ãƒªãƒ³ã‚¯
Databricks + Neon
é–‹ç™ºè€…ãƒ•ã‚¡ãƒ¼ã‚¹ãƒˆã®ã‚µãƒ¼ãƒãƒ¼ãƒ¬ã‚¹ Postgres ä¼æ¥ Neon ã‚’è²·åŽã™ã‚‹ã“ã¨ã§åˆæ„ã—ãŸã“ã¨ã‚’ã€å–œã³ã¨ã¨ã‚‚ã«ç™ºè¡¨ã—ã¾ã™ã€‚ Neon ãƒãƒ¼ãƒ ã¯ã€é«˜é€Ÿæ€§ãƒ»å¼¾åŠ›çš„ã‚¹ã‚±ãƒ¼ãƒªãƒ³ã‚°ãƒ»ãƒ–ãƒ©ãƒ³ãƒï¼ãƒ•ã‚©ãƒ¼ã‚¯æ©Ÿèƒ½ã‚’å‚™ãˆãŸé©æ–°çš„ãªãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã‚¢ãƒ¼ã‚ãƒ†ã‚¯ãƒãƒ£ã‚’é–‹ç™ºã—ã¾ã—ãŸã€‚ ã“ã‚Œã‚‰ã®ç‰¹é•·ã¯é–‹ç™ºè€…ã«ã¨ã£ã¦é…åŠ›çš„ã§ã‚ã‚‹ã ã‘ã§ãªãã€AI ã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆã«ã‚‚æœ€é©ã§ã™ã€‚ Neon ã¨ Databricks ã¯é€£æºã—ã€é–‹ç™ºè€…ã¨ AI ã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆå‘ã‘ã« â€œã‚ªãƒ¼ãƒ—ãƒ³ã‹ã¤ã‚µãƒ¼ãƒãƒ¼ãƒ¬ã‚¹â€ ãªãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹åŸºç›¤ã‚’æä¾›ã—ã¦ã„ãã¾ã™ã€‚ æœ¬æ—¥ã€ç§ãŸã¡ã¯ é–‹ç™ºè€…ãƒ•ã‚¡ãƒ¼ã‚¹ãƒˆã®ã‚µãƒ¼ãƒãƒ¼ãƒ¬ã‚¹ Postgres ä¼æ¥ã€ŒNeonã€ ã‚’è²·åŽã™ã‚‹ã“ã¨ã§åˆæ„ã—ãŸã“ã¨ã‚’ç™ºè¡¨ã§ãã€å¤§å¤‰ã†ã‚Œã—ãæ€ã„ã¾ã™ã€‚Neon ã®å…±åŒå‰µæ¥è€…ãŸã¡ã¯ã€ã‚¹ãƒˆãƒ¬ãƒ¼ã‚¸ã¨ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒˆã‚’çœŸã«åˆ†é›¢ã—ãŸ Postgres ã‚’å†è¨è¨ˆã§ãã‚‹ã€ä¸–ç•Œã§ã‚‚ç¨€æœ‰ãªå˜åœ¨ã§ã™ã€‚å½¼ã‚‰ã¯ç¬¬ä¸€ç´šã® Postgres
manboubird 2025/05/19
postgres

databricks

M&A
ãƒªãƒ³ã‚¯
ï½¢LLMã¯ã‚³ãƒ¢ãƒ‡ã‚£ãƒ†ã‚£ãƒ¼ï½£ã€€ç±³ãƒ‡ãƒ¼ã‚¿ãƒ–ãƒªãƒƒã‚¯ã‚¹CEOãŒèªžã‚‹ - æ—¥æœ¬çµŒæ¸ˆæ–°èž
2013å¹´å‰µæ¥ã®ç±³ãƒ‡ãƒ¼ã‚¿ãƒ–ãƒªãƒƒã‚¯ã‚¹ã¯ä¼æ¥ã«ãƒ‡ãƒ¼ã‚¿åˆ†æžåŸºç›¤ã‚’æä¾›ã™ã‚‹ã€‚24å¹´1æœˆæœŸã®å£²ä¸Šé«˜ã¯16å„„ãƒ‰ãƒ«ï¼ˆç´„2400å„„å††ï¼‰ã‚’è¶…ãˆã€æŽ¨è¨ˆä¼æ¥ä¾¡å€¤ã¯6å…†å††ã‚’ä¸Šå›žã‚‹ã€‚äººå·¥çŸ¥èƒ½ï¼ˆAIï¼‰åˆ†é‡Žã§ã‚‚ç±³ãƒ¢ã‚¶ã‚¤ã‚¯MLã‚’13å„„ãƒ‰ãƒ«ã§è²·åŽã—ã¦æ³¨ç›®ã•ã‚ŒãŸã€‚ç”ŸæˆAIãŒèµ·ã“ã™æŠ€è¡“é©æ–°ã‚„ãƒ“ã‚¸ãƒã‚¹ã¸ã®å½±éŸ¿ã‚’ã©ã†ã¿ã‚‹ã‹ã€ã‚¢ãƒªãƒ»ã‚´ãƒ‡ã‚£ã‚·æœ€é«˜çµŒå–¶è²¬ä»»è€…ï¼ˆCEOï¼‰ã«èžã„ãŸã€‚ï¼ˆèžãæ‰‹ã¯ã‚·ãƒªã‚³ãƒ³ãƒãƒ¬ãƒ¼=å±±ç”°é¼å¤ªéƒŽï¼‰â€•â€•ç±³ã‚ªãƒ¼ãƒ—ãƒ³AIã®ã€ŒGPT
manboubird 2024/08/24
llm

generativeAi

databricks
ãƒªãƒ³ã‚¯
Best Practices for LLM Evaluation of RAG Applications
Unified governance for all data, analytics and AI assets
manboubird 2023/09/16
rag

llm

generativeAi

databricks
ãƒªãƒ³ã‚¯
Software Engineer - ML OSS
manboubird 2023/05/20
databricks

machineLearning
ãƒªãƒ³ã‚¯
ãƒ‡ãƒ¼ã‚¿è§£æžåŸºç›¤ã®ã€Œãƒ‡ãƒ¼ã‚¿ãƒ–ãƒªãƒƒã‚¯ã‚¹ã€ãŒä¼æ¥ä¾¡å€¤4å…†å††çªç ´ï¼ˆForbes JAPANï¼‰ - Yahoo!ãƒ‹ãƒ¥ãƒ¼ã‚¹
manboubird 2021/09/11
databricks
ãƒªãƒ³ã‚¯
å°Žå…¥äº‹ä¾‹ï¼šH&M | Databricks
manboubird 2021/03/03
databricks

h&m

fashion
ãƒªãƒ³ã‚¯
What is Delta Lake in Databricks? | Databricks on AWS
Delta Lake is the optimized storage layer that provides the foundation for tables in a lakehouse on Databricks. Delta Lake is open source software that extends Parquet data files with a file-based transaction log for ACID transactions and scala ble metadata handling. Delta Lake is fully compatible with Apache Spark APIs, and was developed for tight integration with Structured Streaming, allowing yo
manboubird 2020/12/05
databricks

dataManagement

optimization

deltalake
ãƒªãƒ³ã‚¯
Delta Lake ã§ã®ã‚¹ã‚ãƒ¼ãƒžï¼ˆschemaï¼‰DB ã®é©ç”¨ãƒ»å±•é–‹ã¨ã¯
ãƒ‡ãƒ¼ã‚¿ãƒ–ãƒªãƒƒã‚¯ã‚¹ã® Notebook ã‚·ãƒªãƒ¼ã‚ºã‚’è©¦ã™ ãƒ‡ãƒ¼ã‚¿ã¯å¸¸ã«é€²åŒ–ã—ã€è“„ç©ã•ã‚Œã¦ã„ãã¾ã™ã€‚ç§ãŸã¡äººé–“ã®æ—¥ã€…ã®çµŒé¨“ã¨ä¼¼ã¦ã„ã‚‹ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“ã€‚ç§ãŸã¡ã¯ã€è‡ªèº«ã®å‘¨ã‚Šã®ä¸–ç•Œã®å¤‰åŒ–ã«ã¤ã„ã¦ã„ããŸã‚ã«ã€å¸¸ã«æ–°ã—ã„ãƒ‡ãƒ¼ã‚¿ã‚’å–ã‚Šè¾¼ã¿ã€èªè˜ã—ã€ã¨ãã«ã¯ãã®ä¸ã‹ã‚‰æ–°ãŸãªæ¦‚å¿µã‚„è§£é‡ˆã‚’å¾—ã¾ã™ã€‚ã“ã®ã‚ˆã†ãªèªè˜ãƒ¢ãƒ‡ãƒ«ã¯ã€ã¾ã•ã«ãƒ†ãƒ¼ãƒ–ãƒ«ã®ã‚¹ã‚ãƒ¼ãƒžãã®ã‚‚ã®ã§ã™ã€‚ã©ã¡ã‚‰ã‚‚ã€æ–°ã—ãå¾—ã‚‹æƒ…å ±ã®åˆ†é¡žã¨å‡¦ç†ã®ã—ã‹ãŸã‚’æ±ºã‚ã‚‹å½¹å‰²ã‚’æŒã£ã¦ã„ã¾ã™ã€‚ ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã«ãŠã‘ã‚‹ã‚¹ã‚ãƒ¼ãƒžã¨ã¯ ï¼šãã‚‚ãã‚‚ã€Œã‚¹ã‚ãƒ¼ãƒžï¼ˆschemaï¼‰ã€ã¨ã¯ã€æ—¥æœ¬äººã«ã¨ã£ã¦ã‚‚é¦´æŸ“ã¿ã®ã‚ã‚‹ã€Œã‚¹ã‚ãƒ¼ãƒ ï¼ˆschemeï¼‰ã€ã¨ã„ã†è¨€è‘‰ã®æ´¾ç”Ÿèªžã§ã™ã€‚è¨ˆç”»ã‚„å›³ãªã©ã®æ„å‘³ã‚’æŒã¡ã€ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹é–¢é€£ã ã‘ã§ãªãã€å“²å¦ã‚„å¿ƒç†å¦ã§ä½¿ã‚ã‚Œã¦ã„ã‚‹è¨€è‘‰ã§ã‚‚ã‚ã‚Šã¾ã™ã€‚ã“ã®è¨˜äº‹ã§èª¬æ˜Žã™ã‚‹ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã‚¹ã‚ãƒ¼ãƒžï¼ˆDBã‚¹ã‚ãƒ¼ãƒžï¼‰ã¨ã¯ã€ç°¡å˜ã«è¨€ãˆã°ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã®æ§‹é€ ã‚„æ•´ç†ã®ä»•æ–¹ã®ã“ã¨ã§ã™ã€‚ç´°ã‹ãªå®šç¾©ã¯ã€ãƒ‡ãƒ¼ã‚¿
manboubird 2020/12/05
deltalake

dataLake

databricks

schemaManagement

schemaEvolution

schemaValidation

dataQuality
ãƒªãƒ³ã‚¯
Real-time Recommendations using Spark Comcast Labs
Databricks is the Data and AI company. More than 20,000 organizations worldwide â€” including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and ove...
manboubird 2020/12/01
pandas

koalas

databricks
ãƒªãƒ³ã‚¯
TechCrunch | Startup and Technology News
Limited space! Get on waitlist to be the first to know when tickets go live!
manboubird 2020/02/26
dataLake

dataWareHouse

databricks
ãƒªãƒ³ã‚¯
https://www.unsupervised-learnings.co.uk/post/setting-your-data-expectations-data-profiling-and-testing-with-the-great-expectations-library/
manboubird 2020/01/28
dataQuality

profiling

greatExpectations

deequ

monitoring

validation

databricks
ãƒªãƒ³ã‚¯
TechCrunch | Startup and Technology News
Limited space! Get on waitlist to be the first to know when tickets go live!
manboubird 2019/10/26
databricks
ãƒªãƒ³ã‚¯
1 2 æ¬¡ã®ãƒšãƒ¼ã‚¸

ãŠçŸ¥ã‚‰ã›

ã‚‚ã£ã¨èªã‚€

å…¬å¼Twitter

@HatenaBookmark
ãƒªãƒªãƒ¼ã‚¹ã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“ã‚¹ã®ãŠçŸ¥ã‚‰ã›
@hatebu
æœ€æ–°ã®äººæ°—ã‚¨ãƒ³ãƒˆãƒªãƒ¼ã®é…ä¿¡

ã‚ãƒ¼ãƒœãƒ¼ãƒ‰ã‚·ãƒ§ãƒ¼ãƒˆã‚«ãƒƒãƒˆä¸€è¦§

jæ¬¡ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

kå‰ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

lã‚ã¨ã§èªã‚€

eã‚³ãƒ¡ãƒ³ãƒˆä¸€è¦§ã‚’é–‹ã

oãƒšãƒ¼ã‚¸ã‚’é–‹ã

è¨å®šã‚’å¤‰æ›´ã—ã¾ã—ãŸx