DuckDB as the New jq
Recently, I’ve been interested in the DuckDB project (like a SQLite geared towards data applications). And one of the amazing features is that it has many data importers included without requiring extra dependencies. This means it can natively read and parse JSON as a database table, among many other formats. I work extensively with JSON day to day, and I often reach for jq when exploring document
DuckDBã§ãŠæ‰‹è»½ï¼ãƒ‡ãƒ¼ã‚¿ãƒ•ã‚§ãƒ‡ãƒ¬ãƒ¼ã‚·ãƒ§ãƒ³ - Techtouch Developers Blog
tl;dr ã¯ã˜ã‚ã« DuckDB ã¨ã¯ DuckDB ã§ã¯ä½•ãŒèªã‚ã‚‹ã®ã‹ 使ã£ã¦ã¿ã‚‹ S3 上ã®JSON ã‚’èªã‚“ã§ã¿ã‚‹ リレーショナルデータベース 他ツールã§ã¯ãªã DuckDB を使ã†ãƒ¡ãƒªãƒƒãƒˆ ã—ãã˜ã‚Šãƒã‚¤ãƒ³ãƒˆ （特ã«ãƒªãƒªãƒ¼ã‚¹ã•ã‚ŒãŸã°ã‹ã‚Šã®ï¼‰ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã«ã¯æ°—ã‚’ã¤ã‘ã‚‹ S3 ã®ã‚ªãƒ–ジェクト数ãŒå¤šã„å ´åˆä¸éƒ½åˆãŒã‚ã‚ŠãŒã¡ スレッドã®èª¿æ•´ãŒå¿…è¦ãªå ´åˆã‚‚ Redshift ã«ã¯æœªå¯¾å¿œ 終ã‚り㫠付録 MySQL ã®ãƒ‡ãƒ¼ã‚¿ã‚’èªã¿è¾¼ã‚€ä¾‹ã® MySQL å´ã®æº–å‚™ tl;dr DuckDB 便利ã よ。分æžä»¥å¤–ã§ã‚‚使ãˆã‚‹ã‚ˆ 色々ãªå ´æ‰€ã®ãƒ‡ãƒ¼ã‚¿ã‚’閲覧・çµåˆã§ãるよ。標準SQLも使ãˆã‚‹ã‚ˆ ãŸã ã—ã€ç´°ã‹ã„è½ã¨ã—ç©´ã¯è‰²ã€…ã‚ã‚‹ã®ã§æ°—ã‚’ã¤ã‘ã¦ã ã¯ã˜ã‚ã«2023å¹´4月ã«ãƒ‡ãƒ¼ã‚¿ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã¨ã—ã¦å…¥ç¤¾ã—ãŸmin（@not_rogue）ã§ã™ã€‚æš–ã‹ããªã‚‹ã«ã¤ã‚Œã€YouTube ã§è¦‹ãŸå—伊豆ãƒãƒ³ã‚°ãƒˆãƒ¬ã‚¤ãƒ« | æ¾å´Žç”ºã«è¡Œãæ©Ÿé‹ãŒ
リレーショナル・データベースã®ä¸–ç•Œ
åºæ–‡ ç§ã®ä»•äº‹ã¯ã€DBエンジニアã§ã™ã€‚ã¨ã„ã£ã¦ã‚‚別ã«æœ›ã‚“ã§ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã®ä¸–ç•Œã¸ããŸã‚ã‘ã§ã¯ãªãã€å½“åˆã€ç§ã¯ã“ã®åˆ†é‡ŽãŒé¢ç™½ãã‚ã‚Šã¾ã›ã‚“ã§ã—ãŸã€‚「Webç³»ã¯èŠ±å½¢ã€ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã¯æ—¥é™°ã€ã¨ã„ã†è¨€è‘‰ã‚‚å›ã‹ã‚Œã¦ã„ã¾ã—ãŸã€‚今ã§ã‚‚å›ã‹ã‚Œã¦ã„ã‚‹ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“。 ã§ã™ãŒã€ã—ã°ã‚‰ãデータベースを触ã£ã¦ã„ã‚‹ã†ã¡ã«ã€ç§ã¯ã“ã®ä¸–ç•Œã«ã¨ã¦ã‚‚興味深ã„テーマãŒå¤šãã‚ã‚‹ã“ã¨ã‚’知りã¾ã—ãŸã€‚ãªãœã‚‚ã£ã¨æ—©ãæ°—ã¥ã‹ãªã‹ã£ãŸã®ã‹ã€å¾Œæ‚”ã™ã‚‹ã“ã¨ã—ãã‚Šã§ã™ã€‚ ã‚‚ã¡ã‚ã‚“ã€è‡ªåˆ†ã®ä¸æ˜ŽãŒæœ€å¤§ã®åŽŸå› ã§ã™ãŒã€ã“ã®ä¸–ç•Œã«è¶³ã‚’è¸ã¿å…¥ã‚ŒãŸå½“時ã€å…ˆç”Ÿã‚‚ã€å°Žãã®æ›¸ã¨ãªã‚‹å…¥é–€æ›¸ã‚‚ãªã‹ã£ãŸã“ã¨ã‚‚事実ã§ã™ã€‚ 今ã§ã“ããƒã‚¤ãƒ–ルã¨ä»°ã『プãƒã‚°ãƒ©ãƒžã®ãŸã‚ã®SQL 第2版ã€ã‚‚新入社員ã«ã¯æ•·å±…ãŒé«˜ã™ãŽã¾ã—㟠(2015å¹´2月追記：ãã®å¾Œã€è‡ªåˆ†ã§ç¬¬4版を訳出ã§ããŸã®ã ã‹ã‚‰ã€ 人生ã¯ä½•ãŒã‚ã‚‹ã‹åˆ†ã‹ã‚‰ãªã„ã‚‚ã®ã§ã™)。 ãã“ã§ã€ã§ã™ã€‚ã“ã®ã‚µã‚¤ãƒˆã®ç›®çš„ã¯ã€ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã®ä¸–ç•Œã«è¶³ã‚’è¸ã¿
Streaming SQL in Data Mesh
Democratizing Stream Processing @ Netflix By Guil Pires, Mark Cho, Mingliang Liu, Sujay Jain Data powers much of what we do at Netflix. On the Data Platform team, we build the infrastructure used across the company to process data at scale. In our last blog post, we introduced “Data Mesh†— A Data Movement and Processing Platform. When a user wants to leverage Data Mesh to move and transform data,
Python: Just write SQL
I have been writing a lot more Go this past year. For those not familiar, Go favours a non-ORM, non-query-builder approach to interacting with databases. This comes naturally due to the sql package: A common interface to be used alongside database drivers. It’s very common to see actual SQL in Go, even in large projects. On the other hand, Python does not have anything in the standard library that
GitHub - Dataherald/dataherald: Interact with your SQL database, Natural Language to SQL using LLMs
Query your relational data in natural language. | | Docs | Homepage Dataherald is a natural language-to-SQL engine built for enterprise-level question answering over relational data. It allows you to set up an API from your database that can answer questions in plain English. You can use Dataherald to: Allow business users to get insights from the data warehouse without going through a data analys
SQL language
The page describes the SQL dialect recognized by Calcite’s default SQL parser. Grammar SQL grammar in BNF-like form. statement: setStatement | resetStatement | explain | describe | insert | update | merge | delete | query statementList: statement [ ';' statement ]* [ ';' ] setStatement: [ ALTER { SYSTEM | SESSION } ] SET identifier '=' expression resetStatement: [ ALTER { SYSTEM | SESSION } ] RESE
vscode-dbt-power-userã§dbtã®é–‹ç™ºã‚„レビューを効率化ã™ã‚‹ - yasuhisa's blog
背景 vscode-dbt-power-userãŒã‚ˆã‹ã£ãŸã¨ã“゠定義ã«ã•ã£ã¨è¡Œã‘ã‚‹ / 戻れる(Go to definitionãŒä½¿ãˆã‚‹) VSCode内ã§ãƒ¢ãƒ‡ãƒ«é–“ã®ãƒªãƒãƒ¼ã‚¸ãŒè¦‹ã‚Œã‚‹ VSCode内ã‹ã‚‰dbtã®ãƒ¢ãƒ‡ãƒ«ã‚’ã•ã£ã¨å®Ÿè¡Œã§ãã‚‹ モデルファイルã®å˜ç‹¬ã®å®Ÿè¡Œã‚‚ç°¡å˜ ã‚³ãƒ³ãƒ‘ã‚¤ãƒ«æ¸ˆã¿ã®SQLファイルをã•ã£ã¨ãƒ—レビューã§ãã‚‹ ã¾ã¨ã‚ 補足: vscode-dbt-power-userã®å°Žå…¥æ–¹æ³• 背景 dbtã¯å‰è·æ™‚代ã‹ã‚‰å«ã‚ã‚‹ã¨äºŒå¹´ä»¥ä¸Šä½¿ã£ã¦ã„ã¦ã€SQLã§DWHやデータマートã®é–‹ç™ºã‚’ã—よã†ã¨æ€ã†ã¨ã‚‚ã¯ã‚„ã“ã‚Œãªã—ã§ã¯ç”Ÿãられãªã„ãらã„ã«ã¯ä¾¿åˆ©ã«ãªã£ã¦ã„る。dbtãŒã‚ã£ã¦ã‚‚大変ãªã‚¯ã‚¨ãƒªã¯å¤§å¤‰ã§ã¯ã‚ã‚‹ãŒã€å¤§å¤‰ã•ã¯å¤§åˆ†ç·©å’Œã—ã¦ãれる。dbtãŒãªãã¦ã€1ã¤ã®SQLãŒ1000行以上ã‚ã‚Šã€ä¸é–“クエリãŒãƒ†ã‚¹ãƒˆã‚‚ã•ã‚Œã¦ã„ãªã„ã€ã¨ã„ã†çŠ¶æ³ã¯ã‚‚ã†æˆ»ã‚ŠãŸããªã„...。 dbtã«é™ã‚‰ãšã§ã‚ã‚‹ãŒã€ã‚³ãƒ¼ãƒ‰ã¯æ›¸ã„ã¦ã„ã‚‹ã¨
Prisma: Node.js & TypeScriptå‘ã‘ã®å®Œç’§ãªORM
本記事ã¯ã€@nikolasburkæ°ã«ã‚ˆã‚‹ã€ŒThe Complete ORM for Node.js & TypeScriptã€ï¼ˆ2021å¹´4月21日公開）ã®æ—¥æœ¬èªžç¿»è¨³ã‚’ã€è‘—者ã®è¨±å¯ã‚’å¾—ã¦æŽ²è¼‰ã—ã¦ã„ã‚‹ã‚‚ã®ã§ã™ã€‚今後Prismaã®å…¬å¼ãƒ–ãƒã‚°ã®æ—¥æœ¬èªžç¿»è¨³ã®è¨±å¯ã‚’ã„ãŸã ã„ãŸã®ã§ã€ã“ã®ã‚¢ã‚«ã‚¦ãƒ³ãƒˆã§ç¶™ç¶šã—ã¦ç¿»è¨³ã—ã¦ã„ãã¾ã™ã€‚※画åƒã‚„リンクã¯å…¬å¼ã®Blogã‹ã‚‰ãŠå€Ÿã‚Šã—ã¦ã„ã¾ã™ã€‚ Prismaã¯ã€Node.jsã¨TypeScriptã®ãŸã‚ã®æ¬¡ä¸–代ORMã§ã™ã€‚2年以上ã®é–‹ç™ºæœŸé–“を経ã¦ã€ã™ã¹ã¦ã®PrismaツールãŒè£½å“化ã®æº–å‚™ãŒæ•´ã£ãŸã“ã¨ã‚’ãŠä¼ãˆã§ãã‚‹ã“ã¨ã«èˆˆå¥®ã—ã¦ã„ã¾ã™! コンテンツ オブジェクト-リレーショナルマッピングã®æ–°ã—ã„パラダイムã‚らゆるスタックã«å¯¾å¿œã™ã‚‹Prisma ミッションクリティカルãªã‚¢ãƒ—リケーションã§ã®é‹ç”¨ã«å¯¾å¿œ オープンソースを超ãˆã¦ ãŠæ‰‹ä¼ã„ã§ãã‚‹ã“ã¨ã¯ï¼Ÿ Prismaを始ã‚よã†
ZOZOTOWNã®åŸºå¹¹ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ã‚’リプレースã—ãŸè©± - ZOZO TECH BLOG
ã¯ã˜ã‚ã« ã“ã‚“ã«ã¡ã¯ã€æŠ€è¡“本部SRE部ZOZOSREãƒãƒ¼ãƒ ã®å €å£ã§ã™ã€‚普段ã¯ZOZOTOWNã®ã‚ªãƒ³ãƒ—レミスã¨ã‚¯ãƒ©ã‚¦ãƒ‰ã®æ§‹ç¯‰ãƒ»é‹ç”¨ã«æºã‚ã£ã¦ã„ã¾ã™ã€‚ã¾ãŸDBREã¨ã—ã¦ZOZOTOWNã®ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹å…¨èˆ¬ã®é‹ç”¨ãƒ»ä¿å®ˆã‚‚兼務ã—ã¦ãŠã‚Šã¾ã™ã€‚ ZOZOTOWNã§ã¯SQL Serverã‚’ä¸å¿ƒã¨ã—ãŸå„種DBMSãŒç¨¼åƒã—ã¦ã„ã¾ã™ã€‚ãã®ä¸ã§ZOZOTOWNサービスã®æ ¹å¹¹ã¨ãªã‚‹ã„ã‚ゆる基幹データベース（以下ã€åŸºå¹¹DBã¨å‘¼ã¶ï¼‰ã‚’5å¹´ã¶ã‚Šã«ãƒªãƒ—レースã—ã¾ã—ãŸã€‚ 基幹DB群ã¯ã€å•†å“æƒ…å ±ã€åœ¨åº«æƒ…å ±ã€æ³¨æ–‡æƒ…å ±ã€ä¼šå“¡æ§˜æƒ…å ±ã€ãƒ–ãƒ©ãƒ³ãƒ‰æ§˜æƒ…å ±ã€é…é€ç®¡ç†ã€ã‚ãƒ£ãƒ³ãƒšãƒ¼ãƒ³æƒ…å ±ã€åˆ†æžç³»æƒ…å ±ãªã©ZOZOTOWNサービスã«ãŠã‘ã‚‹ã»ã¼å…¨ã¦ã®æƒ…å ±ã‚’ç®¡ç†ã—ã¦ã„ã‚‹ã‚‚ã®ã¨ãªã‚Šã¾ã™ã€‚ リプレースã®ãƒ¢ãƒãƒ™ãƒ¼ã‚·ãƒ§ãƒ³ã¯5å¹´ã®ãƒãƒ¼ãƒ‰ã‚¦ã‚§ã‚¢ï¼ˆä»¥ä¸‹ã€HWã¨å‘¼ã¶ï¼‰ä¿å®ˆæœŸé™çµ‚了ãŠã‚ˆã³SQL Serverã®End Of Life（以下ã€EOLã¨å‘¼ã¶ï¼‰ã‚’è¿Žãˆã‚‹ãŸã‚ã€HW
SQL style guide by Simon Holywell
Overview General Do Avoid Naming conventions General Tables Columns Aliasing or correlations Stored procedures Uniform suffixes Query syntax Reserved words White space Indentation Preferred formalisms Create syntax Choosing data types Specifying default values Constraints and keys Designs to avoid Appendix Reserved keyword reference Column data types SQL style guide Overview You can use this set o
ä¿å®ˆæ€§ã¨ç”Ÿç”£æ€§ã‚’両立ã™ã‚‹åˆ†æžç”¨SQLæ§‹é€ åŒ–ã®4原則 〜 æ§‹é€ åŒ–ãƒ—ãƒã‚°ãƒ©ãƒŸãƒ³ã‚°ã®è€ƒãˆæ–¹ã‚’SQLã«é©ç”¨ã™ã‚‹
ãƒ¤ãƒ•ãƒ¼æ ªå¼ä¼šç¤¾ã¯ã€2023å¹´10月1æ—¥ã«LINEãƒ¤ãƒ•ãƒ¼æ ªå¼ä¼šç¤¾ã«ãªã‚Šã¾ã—ãŸã€‚LINEãƒ¤ãƒ•ãƒ¼æ ªå¼ä¼šç¤¾ã®æ–°ã—ã„ブãƒã‚°ã¯ã“ã¡ã‚‰ã§ã™ã€‚LINEヤフー Tech Blog ã“ã‚“ã«ã¡ã¯ã€‚Yahoo!広告ã®ãƒ‡ãƒ¼ã‚¿ãƒžãƒ¼ã‚±ãƒ†ã‚£ãƒ³ã‚°ã‚½ãƒªãƒ¥ãƒ¼ã‚·ãƒ§ãƒ³ï¼ˆä»¥ä¸‹ã€DMS）を開発ã—ã¦ã„るデータアナリストã®è–„ç”°ã§ã™ã€‚ ã¿ãªã•ã‚“ã¯ã€ä¸é–“テーブルåŒå£«ãŒè¤‡é›‘ã«çµ¡ã¿åˆã„変更ã—よã†ã«ã‚‚影響範囲を推定ã§ããšã€æ‰‹ãŒã¤ã‘られãªã„分æžãƒ‘イプラインã®ä¿å®ˆã§è‹¦åŠ´ã•ã‚ŒãŸçµŒé¨“ã¯ãªã„ã§ã—ょã†ã‹ï¼Ÿ ç§ã®ãƒãƒ¼ãƒ ã§ã¯æ•°åƒè¡Œã«ãŠã‚ˆã¶åˆ†æžç”¨SQLをリファクタリングã—ã¦ã€ä¿å®ˆæ€§ã¨ç”Ÿç”£æ€§ã‚’両立ã™ã‚‹åˆ†æžãƒ‘イプラインã«ç”Ÿã¾ã‚Œå¤‰ã‚らã›ã‚‹ã“ã¨ãŒã§ãã¾ã—ãŸã€‚ ã“ã®è¨˜äº‹ã§ã¯ãƒªãƒ•ã‚¡ã‚¯ã‚¿ãƒªãƒ³ã‚°ã‚’通ã—ã¦ç¢ºç«‹ã—ãŸã€åˆ†æžç”¨SQLã‚’æ§‹é€ åŒ–ã™ã‚‹ãŸã‚ã®4原則を紹介ã—ã¾ã™ã€‚4原則をæ„è˜ã—ãªãŒã‚‰SQLを書ãã“ã¨ã§ã€é«˜å‡é›†ãƒ»ç–Žçµåˆãªåˆ†æžãƒ‘イプラインを作るã“ã¨ãŒã§ãã¾ã™ã€‚ ã“ã®è¨˜äº‹ã§ã¯å‡é›†åº¦ã¨çµåˆåº¦
Enabling static analysis of SQL queries at Meta
UPM is our internal standalone library to perform static analysis of SQL code and enhance SQL authoring. UPM takes SQL code as input and represents it as a data structure called a semantic tree. Infrastructure teams at Meta leverage UPM to build SQL linters, catch user mistakes in SQL code, and perform data lineage analysis at scale. Executing SQL queries against our data warehouse is important to
SPAã‚»ã‚ュリティ超入門 | ドクセル
ã‚¹ãƒ©ã‚¤ãƒ‰æ¦‚è¦ SPA(Single Page Application)ã®æ™®åŠãŒä¸€å±¤é€²ã‚“ã§ãŠã‚Šã€å¾“æ¥åž‹ã®MPAを知らãªã„ウェブ開発者も生ã¾ã‚Œã¤ã¤ã‚るよã†ã§ã™ã€‚SPA対応ã®ãƒ•ãƒ¬ãƒ¼ãƒ ワークã§ã¯åŸºæœ¬çš„ãªè„†å¼±æ€§ã«ã¤ã„ã¦ã¯å¯¾ç–機能ãŒç”¨æ„ã•ã‚Œã¦ã„ã¾ã™ãŒã€ãã‚Œã«ã‚‚é–¢ã‚らãšã€è„†å¼±æ€§è¨ºæ–ç‰ã§åŸºæœ¬çš„ãªè„†å¼±æ€§ãŒæŒ‡æ‘˜ã•ã‚Œã‚‹ã‚±ãƒ¼ã‚¹ã¯ã‚€ã—ã‚増ãˆã¤ã¤ã‚ã‚Šã¾ã™ã€‚ 本セッションã§ã¯ã€Laravelã¨Reactã§é–‹ç™ºã—ãŸã‚¢ãƒ—リケーションをモデルã¨ã—ã¦ã€SQLインジェクションã€ã‚¯ãƒã‚¹ã‚µã‚¤ãƒˆã‚¹ã‚¯ãƒªãƒ—ティングã€èªå¯åˆ¶å¾¡ä¸å‚™ç‰ã®è„†å¼±æ€§ã®å®Ÿä¾‹ã‚’紹介ã—ãªãŒã‚‰ã€ç¾å®Ÿçš„ãªå¯¾ç–ã«ã¤ã„ã¦ç´¹ä»‹ã—ã¾ã™ã€‚Laravelã‚„React以外ã®ãƒ•ãƒ¬ãƒ¼ãƒ ワーク利用者ã«ã‚‚役立ã¤èª¬æ˜Žã‚’心ãŒã‘ã¾ã™ã€‚ PHPカンファレンス2022ã§ã®è¬›æ¼”資料ã§ã™ã€‚ PHPカンファレンスã§ã®å‹•ç”»URL https://www.youtube.com/watch?v=jZ6sWyGxcCs
ランã‚ング
ランã‚ング
ランã‚ング
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
GitHub - xo/usql: Universal command-line interface for SQL databases
Querydsl - Unified Queries for Java
Everything Is a Funnel, But SQL Doesn’t Get It
https://twitter.com/ai_database/status/1726025818621804992
T5QL: Taming language models for SQL generation
ArcaDB: A Container-based Disaggregated Query Engine for Heterogenous Computational Environments
{{#tags}}- {{label}}
{{/tags}}