GitHub - control-theory/gonzo: Gonzo! The Go based TUI log analysis toolYou signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
OpenTelemetry ã® Log を使ã„ã“ãªãã†
Jagu'e'r オブザーãƒãƒ“リティ分科会 Meetup #3 https://jaguer-o11y-sre.connpass.com/event/359593/
ä¸åº¦ãˆãˆ! ãƒã‚®ãƒ³ã‚° - kawasima
2025/4/24ã«é–‹å‚¬ã—㟠#アーã‚部 『ä¸åº¦ãˆãˆ! ãƒã‚®ãƒ³ã‚°ã€ã®å†…容を編集ã—ãŸã‚‚ã®ã§ã™ã€‚ ãƒã‚®ãƒ³ã‚°ã«ã¾ã¤ã‚ã‚‹å•é¡Œã®æ§‹é€ ãƒã‚®ãƒ³ã‚°ã«ã¾ã¤ã‚ã‚‹ç¾å ´ã§ã‚ˆã見られるå•é¡Œã«ã¯ä»¥ä¸‹ã®ã‚ˆã†ãªã‚‚ã®ãŒã‚ã‚Šã¾ã™ã€‚ ãƒã‚®ãƒ³ã‚°ã‚¬ã‚¤ãƒ‰ãƒ©ã‚¤ãƒ³ã‚’定義ã—ã¦ã€é–‹ç™ºè€…ã«å‘¨çŸ¥ã—実装ã—ã¦ã‚‚らã£ãŸãŒã€å®Ÿéš›é‹ç”¨ã—ã¦ã¿ã‚‹ã¨ã€å½¹ã«ç«‹ãŸãªã„ãƒã‚°ãŒå¤šã™ãŽã‚‹ã€‚ 担当者ãŒç•°ãªã‚‹ã¨ã€ãƒã‚°å‡ºåŠ›ã™ã‚‹ç²’度ãŒå¾®å¦™ã«ç•°ãªã‚Šã€è§£æžã«æ™‚é–“ãŒã‹ã‹ã‚‹ã€‚ ãƒã‚°ã®å½¢å¼ã‚„内容ãŒçµ±ä¸€ã•ã‚Œã¦ãŠã‚‰ãšã€åˆ†æžãƒ„ールã§ã®æ´»ç”¨ãŒé›£ã—ã„。 ãƒã‚®ãƒ³ã‚°ãƒ™ã‚¹ãƒˆãƒ—ラクティスã¯æŽ¢ã™ã¨ã„ãã¤ã‹è¦‹ã¤ã‹ã‚Šã¾ã™ã€‚ https://www.dataset.com/blog/the-10-commandments-of-logging/ https://betterstack.com/community/guides/logging/logging-best-practices/ https://new
ãƒã‚®ãƒ³ã‚°è¨è¨ˆå¤§å…¨ - kawasima
#WIP アプリケーションè¨è¨ˆè€…視点ã§ã€ã©ã†ãƒã‚°è¨è¨ˆã™ã‚‹ã‹? ã«ã¤ã„ã¦ã¾ã¨ã‚ã¾ã™ã€‚ 使途別 ç•°ãªã‚‹ç”¨é€”ã§ä½¿ã‚れるã®ã§ã€ãれ毎ã«ãƒ•ã‚¡ã‚¤ãƒ«ã‚„トップカテゴリã§åˆ†é¡žã™ã‚‹ã€‚ トレーシング 世ã®ä¸ã®ãƒã‚®ãƒ³ã‚°ãƒ©ã‚¤ãƒ–ラリã®ä¸»ãªæƒ³å®šç”¨é€”ã¯ã“ã‚Œã§ã‚る。必è¦ãªç®‡æ‰€ã‚„レベルã«å¿œã˜ã¦å‡ºåŠ›åˆ¶å¾¡ã‚’ã™ã‚‹ãŸã‚ã«ã€ãƒã‚°ã‚«ãƒ†ã‚´ãƒªã‚„ãƒã‚°ãƒ¬ãƒ™ãƒ«ãŒå˜åœ¨ã™ã‚‹ã€‚ 目的 アプリケーションã®æ€§èƒ½ã‚’モニタリングã™ã‚‹ トラブル時ã«ä½•ã®ãƒ¡ã‚½ãƒƒãƒ‰ãŒå‘¼ã°ã‚ŒãŸã‹ã‚’追跡ã™ã‚‹ å¿…è¦é …ç›® タイムスタンプ リクエストID Microservicesã®ã‚ˆã†ã«ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ã‚’ã¾ãŸã„ã§ãƒˆãƒ¬ãƒ¼ã‚·ãƒ³ã‚°ã—ãŸã„å ´åˆã¯ã€ä¸Šæµã§ç™ºè¡Œã•ã‚ŒãŸãƒªã‚¯ã‚¨ã‚¹ãƒˆIDを引ã渡ã—ã¦ã€ãれをãƒã‚°ã«å‡ºåŠ›ã™ã‚‹ã€‚ メソッドå 手段 AOPã§ãƒ¡ã‚½ãƒƒãƒ‰ã®Enter/Exitã§è‡ªå‹•çš„ã«ãƒã‚®ãƒ³ã‚°ã•ã‚Œã‚‹ã‚ˆã†ã«è¨å®šã™ã‚‹ã€‚ モニタリング メトリクスAPIやヘルスãƒã‚§ãƒƒã‚¯ç”¨ã®ã‚¨ãƒ³ãƒ‰ãƒã‚¤ãƒ³ãƒˆã‚’用æ„ã™ã‚‹ã®ãŒå½“ãŸã‚Šå‰ã«ãªã£ãŸç¾
GitHub - Brandon-c-tech/RAG-logger: RAG Logger is an open-source logging tool designed specifically for Retrieval-Augmented Generation (RAG) applications. It serves as a lightweight, open-source alternative to LangSmith, focusing on RAG-specific logging n
{ "timestamp": "2024-03-20 10:00:00", "query": "What is machine learning?", "total_time": 8.5, "steps": { "query_understanding": { "name": "query_understanding", "start_time": 1234567890.0, "end_time": 1234567891.0, "duration": 1.0, "metadata": { "detected_intent": "definition_query", "topic": "machine_learning", "confidence": 0.95 } }, "text_embedding": { "name": "text_embedding", "start_time": 1
Grafanaã§S3ã«ç½®ã„ã¦ã‚ã‚‹ãƒã‚°ãƒ•ã‚¡ã‚¤ãƒ«ã‚’å‚ç…§ã™ã‚‹ - ãã—ゃã¿ã®ã¡ã‚Šç´™
ãã£ã‹ã‘ã¯ä»¥ä¸‹ã®ãƒ„イートを見ã‹ã‘ãŸã“ã¨ã§ã—ãŸã€‚ Grafana ã‚’ç«‹ã¦ã‚‹ã ã‘ã§ã€S3 ã«ãƒã‚°ä¿å˜ã—ã¦ãŠã‘ã°ã€å¥½ããªã ã‘グラフãŒä½œã‚Œã‚‹æ™‚代ãŒæ¥ã¦ã—ã¾ã£ãŸã€‚ãªã‚“ã¨ã„ã†ã“ã¨ã 。 https://t.co/uuYgm8dLIr— V (@voluntas) 2025å¹´2月13æ—¥ ãªã«ã“れ良ã•ãã†ã€‚Grafanaã‚‚DuckDBも全然知らんã‘ã©ã€‚ ã¨ã„ã†ã“ã¨ã§è©¦ã—ã¦ã¿ã¾ã—ãŸã€‚ ã‚ーアイテムgrafana-duckdb-datasourceã¨ã„ã†Grafanaプラグインを使用ã—ã¾ã™ã€‚ github.com å‰æ Grafana Version: v11.6.0-82874 (e5b49a406f) Container Image: grafana/grafana-enterprise:main-ubuntu Container OS: Ubuntu 22.04.5 LTS Container CPU
Every System is a Log: Avoiding coordination in distributed applications
Every System is a Log: Avoiding coordination in distributed applications How Restate works, Part 1Posted January 22, 2025 by Stephan Ewen and Jack Kleeman and Giselle van Dongen â€Â 13 min read Building resilient distributed applications remains a tough challenge. It should be possible to focus almost entirely on the business logic and the complexity inherent to the domain. Instead, you need to revi
Best Logging Practices for Safeguarding Sensitive Data | Better Stack Community
Data breaches have become a pressing concern in our interconnected world, emphasizing the critical need to protect sensitive information. Logs act as significant repositories of sensitive data. When fall into the wrong hands, they can pose substantial security risks. These logs often contain personally identifiable information (PII), financial details, authentication credentials, and other sensiti
Pythonã®loggingã«ã¤ã„ã¦åŸºæœ¬çš„ãªéƒ¨åˆ†ã‚’確èªã™ã‚‹ | DevelopersIO
DA事æ¥æœ¬éƒ¨ã®æ¨ªå±±ã§ã™ã€‚ 今回ã¯ã€‚普段ã‹ã‚‰åˆ©ç”¨ã—ã¦ã„ã‚‹Pythonã®loggingã«ã¤ã„ã¦ãªã‚“ã¨ãªãã§ç†è§£ã—ã¦ã„る部分ãŒå¤šã‹ã£ãŸãŸã‚ã€åŸºæœ¬çš„ãªã“ã¨ã‹ã‚‰å¾©ç¿’ã—ãŸã®ã§ç¢ºèªã—ãŸå†…容を共有ã—ã¾ã™ã€‚ å‰ææ¡ä»¶ 本記事ã§åˆ©ç”¨ã—ã¦ã„ã‚‹Pythonã®ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã¯ä»¥ä¸‹ã«ãªã‚Šã¾ã™ã€‚ Python: 3.8.13 確èªã—ãŸå†…容 今回ã¯ã€ä»¥ä¸‹ã®é …ç›®ã«ã¤ã„ã¦å…¬å¼ãƒ‰ã‚ュメントをä¸å¿ƒã«ç¢ºèªã‚’è¡Œã„ã¾ã—ãŸã€‚ ãƒã‚°ãƒ¬ãƒ™ãƒ« loggerã®å®šç¾©æ–¹æ³•ã®ç¨®é¡ž è¨å®šãƒ•ã‚¡ã‚¤ãƒ«å½¢å¼ Formatter 一番簡å˜ãªåˆ©ç”¨ä¾‹ import logging logging.warning('Watch out!') # will print a message to the console logging.info('I told you so') # will not print anything WARNING:root:Watch out!
[å°ãƒã‚¿]uvicornã¨logging
[å°ãƒã‚¿]uvicornã¨logging FastAPIã¨çµ„ã¿åˆã‚ã›ã¦ã‚ˆã使ã‚れるuvicornã§ã™ãŒã€loggingã§è»½ããƒãƒžã£ãŸã®ã§ã¾ã¨ã‚ã¦ãŠãã¾ã™ã€‚ uvicornコマンドã§èµ·å‹•ã™ã‚‹å ´åˆ uvicornã¯å†…部ã§loggingを使ã£ã¦ãƒã‚°ã‚’åãã¾ã™ã€‚下ã®ã‚·ãƒ³ãƒ—ルãªã‚³ãƒ¼ãƒ‰ã§è©¦ã—ã¦ã¿ã¾ã™ã€‚ from fastapi import FastAPI app = FastAPI() @app.get("/hello/{name}") def hello(name: str) -> dict: s = f"Hello, {name}!" return {"message": s}
![[å°ãƒã‚¿]uvicornã¨logging](https://cdn-ak-scissors.b.st-hatena.com/image/square/58e78d58b1bf5302f9f9a65a84a789ffd9c4c4a8/height=288;version=1;width=512/https%3A%2F%2Fres.cloudinary.com%2Fzenn%2Fimage%2Fupload%2Fs--sFJnGlVN--%2Fc_fit%252Cg_north_west%252Cl_text%3Anotosansjp-medium.otf_55%3A%25255B%2525E5%2525B0%25258F%2525E3%252583%25258D%2525E3%252582%2525BF%25255Duvicorn%2525E3%252581%2525A8logging%252Cw_1010%252Cx_90%252Cy_100%2Fg_south_west%252Cl_text%3Anotosansjp-medium.otf_34%3Adagger1%252Cx_220%252Cy_108%2Fbo_3px_solid_rgb%3Ad6e3ed%252Cg_south_west%252Ch_90%252Cl_fetch%3AaHR0cHM6Ly9zdG9yYWdlLmdvb2dsZWFwaXMuY29tL3plbm4tdXNlci11cGxvYWQvYXZhdGFyL2M4NDU5Y2E2OTguanBlZw%3D%3D%252Cr_20%252Cw_90%252Cx_92%252Cy_102%2Fco_rgb%3A6e7b85%252Cg_south_west%252Cl_text%3Anotosansjp-medium.otf_30%3A%2525E6%2525A0%2525AA%2525E5%2525BC%25258F%2525E4%2525BC%25259A%2525E7%2525A4%2525BE%2525E3%252583%252586%2525E3%252583%252583%2525E3%252582%2525AF%2525E3%252583%252595%2525E3%252583%2525A9%2525E3%252583%252583%2525E3%252582%2525B0%252Cx_220%252Cy_160%2Fbo_4px_solid_white%252Cg_south_west%252Ch_50%252Cl_fetch%3AaHR0cHM6Ly9zdG9yYWdlLmdvb2dsZWFwaXMuY29tL3plbm4tdXNlci11cGxvYWQvYXZhdGFyLzgxN2U5ZWU4MzAuanBlZw%3D%3D%252Cr_max%252Cw_50%252Cx_139%252Cy_84%2Fv1627283836%2Fdefault%2Fog-base-w1200-v2.png)
Building a distributed log using S3 (under 150 lines of Go) - blag
I will show how we can implement a durable, distributed, and highly available log using S3. This post is the third part in the series: Disaggregated Storage - a brief introductionZero Disk ArchitectureBuilding a distributed log using S3tl;dr The code is open source, comes with tests and open issues to contribute: s3-log LogI love logs. The log is the heart of data and event streaming systems. A da
S3ã«ã‚ã‚‹ALBãƒã‚°ã®èª¿æŸ»ã¯AthenaよりDuckDBã®ã»ã†ãŒç°¡å˜ - road288ã®æ—¥è¨˜
AWSã®ALB(Application Load Balancer)ã®ãƒã‚°ã¯S3ã«ç½®ã‹ã‚Œã‚‹ãŒã€ã“ã®ä¸èº«ã‚’サクッã¨èª¿ã¹ãŸã„ã¨ãã€Athenaを使ã†æ–¹æ³•ãŒæ¨™æº–çš„ã§ã€ä¸‹è¨˜ã§æ¡ˆå†…ã•ã‚Œã¦ã„るよã†ã«ãƒ‘ーティション射影(Partition Projection)ã§ãƒ†ãƒ¼ãƒ–ルを作ã£ã¦Athenaã‹ã‚‰ã‚¯ã‚¨ãƒªã™ã‚‹ã€‚ パーティション射影を使用ã—㦠Athena 㧠ALB アクセスãƒã‚°ç”¨ãƒ†ãƒ¼ãƒ–ルを作æˆã™ã‚‹ - Amazon Athena ç§ã‚‚従æ¥ã¯ãã®æ–¹æ³•ã‚’使ã£ã¦ã„ãŸãŒã€Athenaã¯ãƒ–ラウザã‹ã‚‰ä½¿ã†ã¨å‹•ä½œãŒã‚‚ã£ã•ã‚Šã—ã¦ã„ã‚‹ã—ã€æ±ºã¾ã£ãŸã‚¯ã‚¨ãƒªã‚’1回ãり実行ã—ã¦çµæžœã‚’å–å¾—ã—ãŸã„ã ã‘ã®ã¨ããªã‚‰ã¾ã ã—ã‚‚ã€æŽ¢ç´¢çš„ã«ã‚¯ã‚¨ãƒªã‚’何発も実行ã—ãŸã„ã¨ãã«ã¯ä½¿ã„å‹æ‰‹ãŒæ‚ªã„。 最近他ã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã§DuckDBを使ã†ã‚ˆã†ã«ãªã£ã¦ã€ä½¿ã„å‹æ‰‹ã®è‰¯ã•ã«æ„Ÿå‹•ã—ã¦ã„ãŸãŒã€DuckDBã¯ALBã®ãƒã‚°ã‚’探索的ã«èª¿ã¹ãŸã„ã¨ãã«ã‚‚ã‚ã£ã¡ã‚ƒä½¿ãˆã‚‹ã¨æ€ã£ãŸ
役立ã¤ãƒã‚¯ã‚™ã«å–り組もã†
JJUG CCC 2024 Fall 2024-10-27 https://jjug.doorkeeper.jp/events/177443
ãƒã‚°åŸºç›¤ã®Fluentdã‚’Fluent Bitã«ç§»è¡Œã—ã¦ç›£è¦–ツールを実装ã—ãŸè©± - Mirrativ Tech Blog
ã¯ã˜ã‚ã¾ã—ã¦ã€Azuma（@azuma_alvin）ã§ã™ã€‚ç¾åœ¨å¤§å¦é™¢ã®1年生ã§ã€2024å¹´2月ã‹ã‚‰4ヶ月間ミラティブã®ã‚¤ãƒ³ãƒ•ãƒ©ãƒãƒ¼ãƒ ã«ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ³ã¨ã—ã¦å‚åŠ ã—ã¾ã—ãŸã€‚普段ã¯ã‚¤ãƒ³ãƒ•ãƒ©ã‚„MLOpsã¨ã„ã£ãŸé ˜åŸŸã«èˆˆå‘³ãŒã‚ã‚Šã€æœ€è¿‘ã¯Vim環境ã®æ•´å‚™ãŒãƒžã‚¤ãƒ–ームã§ã™ã€‚ 本記事ã§ã¯ã€ãƒã‚°åŸºç›¤ã‚’Fluentdã‹ã‚‰Fluent Bitã¸éƒ¨åˆ†ç§»è¡Œã—ãŸçµŒç·¯ã¨ãã®2種類ã®ç›£è¦–ツールã®å®Ÿè£…ã«ã¤ã„ã¦ãŠè©±ã—ã—ã¾ã™ã€‚ 記事ã®æœ€å¾Œã«ã€ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ³ã‹ã‚‰è¦‹ãŸã‚¤ãƒ³ãƒ•ãƒ©ãƒãƒ¼ãƒ ã®ç‰¹å¾´ã¨ç§ãŒ4ヶ月間ã§å¦ã‚“ã ã“ã¨ã‚’紹介ã—ã¦ã„ã¾ã™ã€‚興味ãŒã‚ã‚‹æ–¹ã¯æœ«å°¾ã¾ã§ã‚¹ã‚¯ãƒãƒ¼ãƒ«ã—ã¦ãœã²ã”覧ãã ã•ã„。 1. 背景ã¨ç›®çš„ 2. ミラティブã®ãƒã‚°åŸºç›¤ã«ã¤ã„㦠3. ãƒã‚°æ¬ æã®åŽŸå› 調査 Fluentdã®ãƒãƒƒãƒ•ã‚¡ãƒªãƒ³ã‚°ã®ä»•çµ„ã¿ fsnotifyを用ã„ãŸãƒãƒƒãƒ•ã‚¡ãƒªãƒ³ã‚°ã®è¦³å¯Ÿ è² è·è©¦é¨“ 日付時刻フォーマットã¨ãƒ¯ã‚¤ãƒ«ãƒ‰ã‚«ãƒ¼ãƒ‰ã«ã‚ˆã‚‹ãƒã‚°æ¬ æ ãƒã‚°ä¿å˜ã¨ã‚µãƒ¼ãƒãƒ¼ã‚¿ã‚¤ãƒ スタン
Find your logs data with Explore Logs: No LogQL required! | Grafana Labs
Solutions All end-to-end solutions Opinionated solutions that help you get there easier and faster
æ§‹é€ åŒ–ãƒã‚°ã®ãƒ•ã‚©ãƒ¼ãƒžãƒƒãƒˆ logfmt vs JSON lines - methaneã®ãƒ–ãƒã‚°
æ§‹é€ åŒ–ãƒã‚°ã®ãƒ—ラクティスをã‚ã¡ã“ã¡ã§èª¿ã¹ã¦ã„ãŸã‚‰ã€logfmtを推奨ã™ã‚‹è¨˜äº‹ã‚’見ã¤ã‘ãŸã®ã§èª¿ã¹ã¦ã¿ã¾ã—ãŸã€‚ å…ˆã«çµè«–を言ã†ã¨ã€JSON linesを使ã£ã¦ãŠãã®ãŒè‰¯ã•ãã†ã§ã™ã€‚ logfmt ã«ã¤ã„㦠logfmtã¨ã¯ã‚¹ãƒšãƒ¼ã‚¹åŒºåˆ‡ã‚Šã§ key=value を並ã¹ãŸãƒ•ã‚©ãƒ¼ãƒžãƒƒãƒˆã§ã™ã€‚æ–‡å—列ã«ã¯ã‚¯ã‚©ãƒ¼ãƒˆã¨ã‚¨ã‚¹ã‚±ãƒ¼ãƒ—ã«ã‚ˆã£ã¦ã‚¹ãƒšãƒ¼ã‚¹ã‚„改行をå«ã‚られã¾ã™ã€‚ at=info method=GET path=/ host=mutelight.org fwd="124.133.52.161" dyno=web.2 connect=4ms service=8ms status=200 bytes=1653 (logfmt ã‹ã‚‰å¼•ç”¨) ã‚ã¡ã“ã¡ã§ logfmt ã®ãƒªãƒ•ã‚¡ãƒ¬ãƒ³ã‚¹ã¨ã—ã¦ç´¹ä»‹ã•ã‚Œã¦ã„ã‚‹ã®ã¯ã“ã®è¨˜äº‹ã§ã™ã€‚ https://brandur.org/logfmt 発明ã•ã‚ŒãŸã®ã¯ã©ã“ã‹åˆ†ã‹ã‚Šã¾ã›ã‚“ãŒã€æµè¡Œ
alpã®ä½¿ã„æ–¹(発展編)
基本編ã¯ã“ã¡ã‚‰ã€‚ 主ã«ISUCONã§ä½¿ã‚ã‚Œã¦ã„るアクセスãƒã‚°é›†è¨ˆãƒ„ール alp ã®ä½¿ã„方をã€ISUCON10オンライン予é¸å•é¡Œã®ã‚¢ã‚¯ã‚»ã‚¹ãƒã‚°ã‚’例ã«ã€åŸºæœ¬ã‚ˆã‚Šã‚‚å°‘ã—発展ã—ãŸä½¿ã„方を紹介ã—ã¾ã™ã€‚ alpã¯ã€ãƒ‡ãƒ•ã‚©ãƒ«ãƒˆã§90,95,99パーセンタイルを出力ã—ã¾ã™ãŒã€ãれ以外ã®ãƒ‘ーセンタイルを出力ã—ãŸã„ã“ã¨ãŒã‚ã‚‹ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“。ãã®ã‚ˆã†ãªã¨ãã¯--percentilesを使用ã—ã¾ã™ã€‚ $ cat /path/to/access.log | alp json -m '/api/estate/[0-9]+,/api/chair/[0-9]+,/api/recommended_estate/[0-9]+' --sort avg -r --show-footers -o count,1xx,2xx,3xx,4xx,5xx,min,max,avg,sum,p50,p75 --percentiles 50,7
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
GitHub - Textualize/toolong: A terminal application to view, tail, merge, and search log files (plus JSONL).
Custom logging with slog Making Logging Fun Again!
スタックトレース始ã‚ã¦ã¿ãŸ
{{#tags}}- {{label}}
{{/tags}}