èæ¯ Dockerã³ã³ãããç«ã¦ãããã«ã¦ã§ã¢ã«ææããã®ã§ãµã¤ãã¼ã»ãã¥ãªãã£ã®åèãå ¼ãã¦ã¡ã¢æ¸ããã¦ã¿ã¾ããã 注æäºé ãã«ã¦ã§ã¢ã«ææãã被害ã®å¯¾å¦æ¹æ³ãè¨è¿°ãã¦ãã¾ãããã«ã¦ã§ã¢èªä½ã®æ©è½ã解æã®è§£èª¬ã§ã¯ãªãä¸è¬å©ç¨è ã¦ã¼ã¶ã¼ã®è¦ç¹ããææçµè·¯ã¨å¯¾å¿æ¹æ³ã«ã¤ãã¦ã®è¨è¿°ã«ãªãã¾ãã ãã«ã¦ã§ã¢ææç¶æ³ çç¶ Dockerã³ã³ããã稼åããããã¹ãã®ãã¼ãã¢ãã¬ã¼ã¸(CPUè² è·)ã常æ4ãè¶ ããç¶æ³ã«ãªã£ã¦ãã¾ãããã¤ã¾ã400%ã§ãã¹ãããã«å転ãã¦ã訳ã§ããªã ä¾ãããªãã¨ã´ã¡ã³ã²ãªãªã³åå·æ©ãæ´èµ°ãã¦ãã¤ã¡ãããã³ã³ã½ã¼ã«ç»é¢ã«åãã£ã¦å«ãã§ããã¨ããã§ãï¼éï¼ ãããªãã¨ã¯ã©ã¦ããµã¼ãã¹ã®AWSã¨ãã ã¨è±æã§è¦åã¢ã©ã¼ããé£ãã§æ¥ã¾ããæ¯æ¥èª²éããã§è¨å¤§ãªå©ç¨æéè«æ±ãæ¥ããã¨ã«ãªãã¾ããæãããï¼ï¼ ç¶æ³ã®èª¿æ» CPUã®å©ç¨ç¶æ³ãã¡ã¢ãªã®ä½¿ç¨éãªã©ã調æ»ãããã¼ã«é¡ããã
PostgreSQLã«ã¯BRINã¤ã³ããã¯ã¹ï¼Block Range Indexï¼ã¨ããæ©è½ãããããã°ãã¼ã¿ã«ä»å±ããã¿ã¤ã ã¹ã¿ã³ãå¤ãªã©ãè¿ããå¤ãæã£ããã¼ã¿ãç©ççã«è¿æ¥ããã¨ããç¹å¾´ãæã£ã¦ããã¨ããæ¤ç´¢ç¯å²ãå¹ççã«çµãè¾¼ãããã«ä½¿ç¨ããäºãã§ããã ãã®æ©è½ã¯PG-Stromã§ã対å¿ãã¦ããããã®è©³ç´°ã¯ä»¥åã®ã¨ã³ããªã§ã解説ãã¦ããã kaigai.hatenablog.com ããã¤ã¾ãã§èª¬æããã¨ãæç³»åã®ãã°ãã¼ã¿ã®ããã«å¤§åã追è¨ï¼Insert-Onlyï¼ã§ããããã¤ã¿ã¤ã ã¹ã¿ã³ãå¤ã®ããã«è¿ããå¤å士ãè¿æ¥ãã¦ããå ´åã1MBã®ãããã¯((pages_per_rangeãããã©ã«ãã®128ã®å ´åã8kB * 128 = 1MB))ãã¨ã«ãã®æå°å¤ï¼æ大å¤ãè¨é²ãã¦ãããã¨ã§ãæããã«æ¤ç´¢æ¡ä»¶ã«ãããããªãç¯å²ããèªã¿é£ã°ãäºãã§ããã ä¾ãã°ä»¥ä¸ã®ä¾ã§ããã°ãWHE
ç®æ¬¡ ã¯ã¼ã«ã¼ãã¼ãã®ä½æ Digdagã¨Embulkã®Dockerãã«ã Kubernetesã«Digdag/Embulkãããã㤠Redashã®å°å ¥ ã¾ã¨ã Kubernetesä¸ã«åæç°å¢ãæ§ç¯ããæ©ä¼ããã£ãã®ã§ã©ã®ããã«æ§ç¯ããããç´¹ä»ãã¾ããåããããªæ§æã§Kubernetesä¸ã§æ§ç¯ããã®ã¯3åç®ã«ãªã£ãã®ã§æ§ç¯æ¹æ³ãæ´ç·´ããã¦ãã¾ãããæ§æã¯ä»¥ä¸ã®ããã«ãªã£ã¦ãã¾ãã MySQL(RDS): ãµã¼ãã¹ã®ãã¼ã¿ãã¼ã¹ãããã®ãã¼ãã«ããBigQueryã«Embulkã§ãã¼ã¿ãã¨ã¯ã¹ãã¼ããã¾ãã PostgreSQL(RDS): Digdagã®ãã¼ã¿ãã¼ã¹ãä»åæ°ãã«ã¤ããã¾ããã Digdag: ãã¼ã¿ãã¼ã¹ã®ã¨ã¯ã¹ãã¼ããªã©ãå®è¡ããã¿ã¹ã¯ã¹ã±ã¸ã¥ã¼ã©ã失æããã¨ãã«ãªãã©ã¤ãã§ãã¾ãã Embulk: ãã©ã°ã¤ã³ã使ã£ã¦ãã¼ã¿ãã¼ã¹ãMySQLããBigQueryã«ã¨
æ°åã³ããâ¦æææ¡å¤§ãæ¢ã¾ããªãã§ãã⦠é§ãåºãGISå±ã¨ãã¦ã¯ä½ãããã®ãã¼ã¿ãå°å³ä¸ã«ä¹ãã¦å¯è¦åããããªã¼ã¨ã¯æã£ã¦ãããã§ãããå ¬éããã¦ããã¼ã¿ã£ã¦å¤§æµãPDFãªãã§åéããã®ãã£ã¡ãããã©ãããâ¦ã¨å°»è¾¼ã¿ãã¦ãã¾ããã ãï¼ï¼ï¼ï¼ï¼ï¼ éå¶å©ç®çã§ããã°ãã¼ã¿ã®è¤è£½ã»å¼ç¨ã»è»¢è¼èªç±ã§ç´ æ´ããããã¼ã¿ãæä¾ãã¦ããã¦ãããµã¤ãæ§ï¼https://gis.jag-japan.com/covid19jp/ï¼ãè¦ã¤ããã®ã§ãã®ãã¼ã¿ã使ã£ã¦å°å³ä¸ã«è¡¨ç¾ãã¦ããããã¨æãã¾ãï¼ï¼xf ä»åã¯åå¼·ãã¦ãã«å¤å°åé·ãªæ§æï¼docker-composeã§ã¢ããªã±ã¼ã·ã§ã³ãµã¼ãã¼ã¨DBãµã¼ãã¼ãç«ã¡ä¸ãã¦ãããããGeoJSONãAPIã§é ä¿¡ãããªã©ï¼ã«ãªã£ã¦ãã¾ãã ãã¼ã¿ã®åå¾ æ©éãã¼ã¿ãè¦ã¦ã¿ã¾ãããï¼ ä¸è¿°ã®ãµã¤ãã«ã¢ã¯ã»ã¹ããå·¦ä¸ã®csvã®ãªã³ã¯ãã¯ãªãã¯ãã¾ãã ããã¨ä»¥ä¸ã®ãã
ã¯ããã« ãã£ã¶ãã©ãããã ã¼ãã¾ãã§ãã ãã®è¨äºã¯ãæ¬çªç°å¢ã§ããããã¡ãã£ã人 Advent Calendar 2019 - Qiitaã®11æ¥ç®ã®è¨äºã§ãã ããã¯ãä¸éå端ãªç¥èã§ãµã¼ãã¹ãéç¨ãã¦ããçµæãã¿ã¤ãã«éãã®å¤§å¤±æããã¦ãã¾ã£ãã話ã§ããå人éçºã§ã®åºæ¥äºãªã®ã§ãæ¥åã§èµ·ãããã¨ãã¨èè¬ãæ¡ããã¦ããæ¹ã¯ãå®å¿ãã ããã èªãã®ããããæ¥ããããã¬ãã«ã§ãããæãã®ããã«æãã¦ããã¾ãã ãã®è¨äºãèªãã§ã»ãã人 åãã¦ã¤ã³ã¿ã¼ãããä¸ã«ãµã¼ãã¹ãå ¬éãããã¨ãã¦ãã人 å太éã®å©ç¨è æ§(ãã®å ´ããåããã¦ãæ¹ãã¦ãè©«ã³ç³ãä¸ãã¾ããæ¬å½ã«ç³ã訳ãããã¾ããã§ããã) èæ¯ã¨ã Discordèªã¿ä¸ãBot ãå太éãã«ã¦ããããã¾ãã www.dayaman.work å©ç¨è ãç´10ä¸äºº ãããã®VPSã«ã¦Appãµã¼ã2å°ãDBãµã¼ã1å°ã§éç¨ åãµã¼ãã®æ»æ´»ç£è¦
Homebrewã§PostgreSQLãã¤ã³ã¹ãã¼ã«ããããã¨ããã®ã§ããã権éå¨ããªã©ã§è©°ã¾ã£ãã®ã§ãåå¿é²ã¨ãã¦æ®ãã¦ãã¾ãã èªåãPostgreSQLãã¤ã³ã¹ãã¼ã«ããéã«åèã«ããURLãå¼µã£ã¦ããã®ã§ãä½µãã¦ç¢ºèªãã¦é ããã°ã§ãã Homebrewã§PostgreSQLãã¤ã³ã¹ãã¼ã« $ãbrew install postgresql ==> Downloading https://homebrew.bintray.com/bottles/postgresql-11.1.mojave.bottle.tar.gz Already downloaded: /Users/suzuki_naoto/Library/Caches/Homebrew/downloads/0c14020ff4817a892c73344b81dbd6f31e0d6e83ca0778de6fe3b4d4ecba7
$ initdb /usr/local/var/postgres -E utf8 The files belonging to this database system will be owned by user "dai". This user must also own the server process. The database cluster will be initialized with locale "ja_JP.UTF-8". initdb: could not find suitable text search configuration for locale "ja_JP.UTF-8" The default text search configuration will be set to "simple". Data page checksums are disa
â»æ³¨:å¤ãæ å ±ãªã®ã§ä»®æ³ç°å¢ä¸ã®linuxãªã©ã«æ§ç¯ãããã¨ããå§ããã¾ãã DBãSQLç³»ã®æ¬ãè²·ã£ã¦åå¼·ããã 大æµWindowsç¨ã®ç°å¢æ§ç¯ããä¹ã£ã¦ããªãã£ãã®ã§ ã¾ãå®å ¨ãªç´ 人ã§ããªãããã¨æ¾æ¼«ãªãã¨ã«æ¤è¨¼ç°å¢ãç¨æããªãã£ãã Indexã¨ãå®è¡è¨ç»çãæ¤è¨¼ããããã« èªåã®macOs Sierra + homebrewãªç°å¢ã«postgresqlãå°å ¥ããã ç°å¢æ å ± Postgresqlã®ãã¼ã¸ã§ã³:9.6.1 macOs Sierraã®ãã¼ã¸ã§ã³:10.12.2 postgresqlå°å ¥
ååã«å¼ãç¶ããScrapyã使ã£ã¦ãã®æ¥è¨ã®ã¯ãã¼ãªã³ã°ãè¡ãã¾ãã github.com ä»åã¯ã¯ãã¼ãªã³ã°ã§å¾ãããå¤ããããªãã¼ã·ã§ã³ãã¦PostgreSQLã«ä¿åããPipelineãå®è£ ãã¾ããSpiderã®å®è£ ã¯ååã®æ稿ãåèã«ãã¦ã¿ã¦ãã ããã ohke.hateblo.jp ãã¡ãã®æ¸ç±ãåèã«ãã¦ã¾ãã Pipeline Scrapyã«ãããPipelineã¯ãSpiderãã¯ãã¼ãªã³ã°ã»ã¹ã¯ã¬ã¤ãã³ã°ããå¤ã«å¯¾ãã¦ãããªãã¼ã·ã§ã³ãã§ãã¯ãæ°¸ç¶åãªã©ã®å¾å¦çãè¡ãããã®ä»çµã¿ã§ãã Spiderãåå¾ããå¤ãItemã«è©°ãã¦è¿ãã¨ãåªå é ä½ã«å¾ã£ã¦è¤æ°ã®ã¿ã¹ã¯ãå®è¡ããã¾ãã ããã§ã¯ä¾ã¨ãã¦ãååä½æããarchive_spiderã使ããåå¾ããè¨äºã¿ã¤ãã«ã»æ稿æ¥ã®ãã©ã¼ãããããã§ãã¯ããPipelineã¨ãPostgreSQLã«ä¿åããPipelineãä½
PostgreSQL9.6ã§contrib/pg_trgmã«å ¥ãword similarityã«ã¤ãã¦ã®ã¡ã¢ã§ãããªãªã¼ã¹ãã¼ãã¯ãã¡ãã Add support for "word similarity" to contrib/pg_trgm (Alexander Korotkov, Artur Zakirov) These functions and operators measure the similarity between one string and the most similar single word of another string. #ããããSimilarityæ¤ç´¢ã¨ã¯ï¼ ææ¸éã®é¡ä¼¼åº¦(ã©ãã ãä¼¼ã¦ããã)ãç®åºãã¦ãæå®ããé¾å¤ä»¥ä¸ã®é¡ä¼¼åº¦ãæã£ãæç« ãé¸æããã¨ããæ¤ç´¢ã§ãã PostgreSQLã®contrib/pg_trgmã使ãã¨å©ç¨ã§ããsimil
ãªãªã¼ã¹ãé害æ å ±ãªã©ã®ãµã¼ãã¹ã®ãç¥ãã
ææ°ã®äººæ°ã¨ã³ããªã¼ã®é ä¿¡
å¦çãå®è¡ä¸ã§ã
j次ã®ããã¯ãã¼ã¯
kåã®ããã¯ãã¼ã¯
lãã¨ã§èªã
eã³ã¡ã³ãä¸è¦§ãéã
oãã¼ã¸ãéã
{{#tags}}- {{label}}
{{/tags}}