How thousands of ‘overworked, underpaid’ humans train Google’s AI to seem smart
Sawyer is one among the thousands of AI workers contracted for Google through Japanese conglomerate Hitachi’s GlobalLogic to rate and moderate the output of Google’s AI products, including its flagship chatbot Gemini, launched early last year, and its summaries of search results, AI Overviews. The Guardian spoke to 10 current and former employees from the firm. Google contracts with other firms fo
DatabricksãŒã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆè©•ä¾¡ã®çµ„ã¿è¾¼ã¿LLM判æ–ã«å¤§å¹…ãªæ”¹å–„を発表
評価方法論ç§ãŸã¡ã¯ã€æ–°ã—ã„回ç”ã®æ£ç¢ºã•ã‚’判æ–ã™ã‚‹ãŸã‚ã«ã€HotPotQAã‚„Natural Questionsã‚’å«ã‚€å¦è¡“データセットã®ã‚³ãƒ¬ã‚¯ã‚·ãƒ§ãƒ³ã§è©•ä¾¡ã‚’è¡Œã„ã¾ã™ã€‚ã¾ãŸã€æ–°ã—ã„LLM判æ–ã®å®Ÿç”¨çš„ãªåˆ©ç‚¹ã‚’強調ã™ã‚‹ãŸã‚ã«ã€é‡‘èžã€æ–‡æ›¸åŒ–ã€HRãªã©ã®æ¥ç•Œã§ç‰¹å®šã®ãƒ¦ãƒ¼ã‚¹ã‚±ãƒ¼ã‚¹ã‚’モデル化ã™ã‚‹ãŠå®¢æ§˜ã‹ã‚‰å¯„è´ˆã•ã‚ŒãŸãƒ‡ãƒ¼ã‚¿ã‚»ãƒƒãƒˆã®å†…部ベンãƒãƒžãƒ¼ã‚¯ã‚‚使用ã—ã¾ã™ã€‚（ãŠæ°—軽ã«ãŠå•ã„åˆã‚ã›ãã ã•ã„ [email protected]ã‚ãªãŸãŒDatabricksã®é¡§å®¢ã§ã€ç§ãŸã¡ãŒã‚ãªãŸã®ãƒ¦ãƒ¼ã‚¹ã‚±ãƒ¼ã‚¹ã®ãŸã‚ã«ã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆè©•ä¾¡ã‚’改善ã™ã‚‹ã®ã‚’助ã‘ãŸã„å ´åˆã€‚) å¦è¡“データセットã¨ç”£æ¥ãƒ‡ãƒ¼ã‚¿ã‚»ãƒƒãƒˆã®ãã‚Œãžã‚Œã«ãŠã‘る（質å•ã€ç”Ÿæˆã•ã‚ŒãŸå›žç”ã€å‚照回ç”）ã®ä¸‰ã¤çµ„ã¿ã«ã¤ã„ã¦ã€æˆ‘々ã¯è¤‡æ•°ã®äººé–“ã®ãƒ©ãƒ™ãƒ©ãƒ¼ã«ç”Ÿæˆã•ã‚ŒãŸå›žç”ã®æ£ç¢ºã•ã‚’評価ã—ã¦ã‚‚らã„ã€å¤šæ•°æ±ºã«ã‚ˆã‚Šé›†ç´„ã•ã‚ŒãŸãƒ©ãƒ™ãƒ«ã‚’å¾—ã¦ã„ã¾ã™ã€‚ãŸã ã—ã€å¤šæ•°æ±ºãŒå¾—られãªã„例ã¯é™¤å¤–ã—ã¾ã™ã€‚集約ã•ã‚ŒãŸãƒ©
強化ã•ã‚ŒãŸ エージェント評価機能 ãŒç™»å ´ï¼
パイãƒãƒƒãƒˆã‹ã‚‰æœ¬ç•ªã¾ã§ - 自動評価ã€å°‚門家ã®ãƒ•ã‚£ãƒ¼ãƒ‰ãƒãƒƒã‚¯ã€ä¸¡ãƒ•ã‚§ãƒ¼ã‚ºã‚’通ã˜ãŸæ˜Žç¢ºãªå復パスã§GenAIã®æŽ¡ç”¨ã‚’効率化ã—ã¾ã™ã€‚ カスタマイズå¯èƒ½ãªGenAI評価 - カスタムメトリクスを定義ã—ã€æ–°ã—ã„Guidelines AI Judgeを使用ã—ã€æŸ”軟ãªå…¥å‡ºåŠ›ã‚¹ã‚ーマをæŒã¤ä»»æ„ã®ãƒ¦ãƒ¼ã‚¹ã‚±ãƒ¼ã‚¹ã‚’評価ã—ã¾ã™ã€‚ シームレスãªå°‚門家ã¨ã®ã‚³ãƒ©ãƒœãƒ¬ãƒ¼ã‚·ãƒ§ãƒ³ - æ›´æ–°ã•ã‚ŒãŸãƒ¬ãƒ“ューアプリã¯ã€ãƒ•ã‚£ãƒ¼ãƒ‰ãƒãƒƒã‚¯ã®åŽé›†ã¨è©•ä¾¡ãƒ‡ãƒ¼ã‚¿ã‚»ãƒƒãƒˆã®ç®¡ç†ã‚’ç°¡ç´ åŒ–ã—ã¾ã™ã€‚ 今週åˆã‚ã€Databricks ã«ãŠã‘ã‚‹æ–°ãŸãªã‚¨ãƒ¼ã‚¸ã‚§ãƒ³ãƒˆé–‹ç™ºæ©Ÿèƒ½ã‚’発表ã—ã¾ã—ãŸã€‚ç§ãŸã¡ã¯ã“ã‚Œã¾ã§æ•°ç™¾ç¤¾ã®é¡§å®¢ã¨è©±ã‚’ã™ã‚‹ä¸ã§ã€ãƒ‘イãƒãƒƒãƒˆæ®µéšŽã‚’超ãˆã¦æ´»ç”¨ã‚’進ã‚る上ã§ã€å…±é€šã™ã‚‹2ã¤ã®èª²é¡ŒãŒã‚ã‚‹ã“ã¨ã«æ°—ã¥ãã¾ã—ãŸã€‚ 1ã¤ç›®ã¯ã€ãƒ¢ãƒ‡ãƒ«ã®æœ¬ç•ªç’°å¢ƒã§ã®æ€§èƒ½ã«å¯¾ã™ã‚‹ä¿¡é ¼ãŒæŒã¦ãªã„ã“ã¨ã€‚ 2ã¤ç›®ã¯ã€æ”¹å–„ã®ãŸã‚ã®æ˜Žç¢ºãªãƒ—ãƒã‚»ã‚¹ãŒç¢ºç«‹ã•ã‚Œã¦ã„ãªã„ã“ã¨ã€‚ ã“ã®2ã¤ã®èª²é¡ŒãŒåŽŸ
AI時代ã«æ±‚ã‚られるエンジニアã¯ã©ã†å¤‰ã‚る？　AIæ¥ç•Œã‚’リードã™ã‚‹PKSHA社ãŒè€ƒãˆã‚‹3ã¤ã®èƒ½åŠ›ã¨ã¯ - Findy Engineer Lab
è¿‘ã„å°†æ¥ã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã«æ±‚ã‚られる役割やスã‚ルãŒå¤§ãã変ã‚ã‚‹ã®ã§ã¯ãªã„ã‹â€•â€•ã€€è¿‘å¹´ã®æ€¥é€ŸãªAI技術ã®ç™ºå±•ã«ä¼´ã„ã€ãã†ã„ã£ãŸè€ƒãˆæ–¹ãŒå¼·ã¾ã£ã¦ã„ã一方ã€ä¾ç„¶ã¨ã—ã¦å…ˆè¡Œãä¸é€æ˜Žãªå´é¢ã‚‚ã‚ã‚Šã¾ã™ã€‚今後ã€ãƒ—ãƒãƒ€ã‚¯ãƒˆã‚„ãã®é–‹ç™ºã«AIãŒæµ¸é€ã—ã¦ã„ããªã‹ã§ã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®ã‚ã‚Šæ–¹ã¨ã¯ã©ã®ã‚ˆã†ã«å¤‰ã‚ã£ã¦ã„ãã®ã§ã—ょã†ã‹ã€‚ 今回ã¯ã€2012å¹´ã«å‰µæ¥ã—ã€ChatGPTã®ç™»å ´ä»¥å‰ã‹ã‚‰AIã®ç¤¾ä¼šå®Ÿè£…ã«å–り組ã¿ç¶šã‘ã¦ã„ã‚‹æ ªå¼ä¼šç¤¾PKSHA Technology（以下ã€PKSHA）ã«ã‚¤ãƒ³ã‚¿ãƒ“ュー。åŒç¤¾ã®åŸ·è¡Œå½¹å“¡å…¼VPoEã§ã‚ã‚‹æ¾æ±Ÿå®æ¨¹ã•ã‚“ã«ã€Œã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®å½¹å‰²ã‚„求ã‚られる能力ã®å¤‰åŒ–ã€ã€Œãƒ—ãƒãƒ€ã‚¯ãƒˆåŠã³ç¤¾å†…活用ã®ä»•æ–¹ã€ãªã©ã‚’伺ã„ã¾ã—ãŸã€‚ 生æˆAIã¯ä¸‡èƒ½ã§ã¯ãªãã€ä¾¿åˆ©ãªãƒ„ールã®ã²ã¨ã¤ ――生æˆAIãŒç™»å ´ã—ãŸã“ã¨ã§ã€ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã®å½¹å‰²ã¯ã©ã®ã‚ˆã†ã«å¤‰åŒ–ã—ã¦ããŸã®ã§ã—ょã†ã‹ã€‚ 我々PKSHAã¯ã€ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ を活用ã—ã¦ç¤¾ä¼šã®èª²é¡Œã‚’解決ã—ã¦ã„ã‚‹
第8回 Data-Centric AI勉強会 ~Human-in-the-Loop機械å¦ç¿’ 特別回~ (2024/05/10 19:30〜)
新機能 イベントå‚åŠ è€…é™å®šã®æ‡‡è¦ªä¼šã‚„ミニイベント開催ã«å¯¾å¿œã—ãŸã€Œã‚µãƒ–イベント機能ã€ã‚’リリースã—ã¾ã—ãŸã€‚ イベントé‹å–¶ã‚’ç°¡ç´ åŒ–ã—ã€å‚åŠ è€…ã®ç™»éŒ²æ¼ã‚Œã‚„確èªæ¼ã‚Œã‚’防ãã®ã«ã”活用ãã ã•ã„。 主催ã®æ–¹ã¯ã‚µãƒ–イベントを作æˆã™ã‚‹ã‹ã‚‰ã€å‚åŠ è€…ã®æ–¹ã¯ã‚µãƒ–イベントãŒè¨å®šã•ã‚Œã¦ã„るイベントã«å‚åŠ ã™ã‚‹ã‹ã‚‰è©³ç´°ã‚’ã”確èªã„ãŸã ã‘ã¾ã™ã€‚ 5月 10 第8回 Data-Centric AI勉強会 ~Human-in-the-Loop機械å¦ç¿’ 特別回~ 「Human-in-the-Loop機械å¦ç¿’ã€ã®ç¿»è¨³è€…3åã‚’ãŠæ‹›ãã™ã‚‹ç‰¹åˆ¥å›žï¼
20分ã§åˆ†ã‹ã‚‹ Human-in-the-Loop 機械å¦ç¿’ã«ãŠã‘るアノテーションã¨ãƒ’ューマンコンピューターインタラクションã®çœŸé«„
第8回 Data-Centric AI勉強会 ~Human-in-the-Loop機械å¦ç¿’ 特別回~ã®ç™ºè¡¨å†…容ã§ã™ã€‚ https://dcai-jp.connpass.com/event/315963/ 書ç±ã€ŒHuman-in-the-Loop 機械å¦ç¿’ã€ã«ãŠã„ã¦ã€ç¿»è¨³ã‚’担当ã—ãŸç« （1,7,8,…
GitHub - opendilab/awesome-RLHF: A curated list of reinforcement learning with human feedback resources (continually updated)
(The following section was automatically generated by ChatGPT) RLHF typically refers to "Reinforcement Learning with Human Feedback". Reinforcement Learning (RL) is a type of machine learning that involves training an agent to make decisions based on feedback from its environment. In RLHF, the agent also receives feedback from humans in the form of ratings or evaluations of its actions, which can
Ultimate Guides to Destiny 2 Raids
Unleash Your Guardian in Destiny 2 Discover the world of Destiny 2 through our expert guides and stay updated with the latest events. Master the Art of Combat Learn strategies for Raids and Crucible battles from seasoned players and become a legendary Guardian. Stay Legendary, Stay Informed Keep up with game updates and event news. Your epic Destiny 2 journey begins here!
ランã‚ング
ãŠçŸ¥ã‚‰ã›
ランã‚ング
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
Hundreds of Google AI Workers Were Fired Amid Fight Over Working Conditions
When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration
Knowledge Transfer in Human-AI Collaboration
Generate synthetic data with BigQuery DataFrames and LLMs | Google Cloud Blog
agent-evaluation-metrics-guidelines-review-app - Databricks
Expanding Knowledge Graphs with Humans in the Loop
Bloomberg - Are you a robot?
DEW 2020 - 1st Data Excellence Workshop
Data Excellence: Better Data for Better AI
What I Learned From Attending Scale Transform 2021 — James Le
https://scale.com/blog/how-we-scale-machine-learning
Document AI | Google Cloud
{{#tags}}- {{label}}
{{/tags}}