Genie 2: A large-scale foundation world model
Research Genie 2: A large-scale foundation world model Published 4 December 2024 Authors Jack Parker-Holder, Philip Ball, Jake Bruce, Vibhavari Dasagi, Kristian Holsheimer, Christos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung, Michael Dennis, Sultan Kenjeyev, Shangbang Long, Vlad Mnih, Harris Chan, Maxime Gazeau, Bonnie Li, Fabio Pardo, Luyu Wang
大è¦æ¨¡è¨€èªžãƒ¢ãƒ‡ãƒ«ã®Fine-tuningã«ã‚ˆã‚‹ãƒ‰ãƒ¡ã‚¤ãƒ³çŸ¥è˜ç²å¾—ã®æ¤œè¨Ž - Preferred Networks Research & Development
本記事ã¯ã€2023å¹´å¤å£ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ³ã‚·ãƒƒãƒ—プãƒã‚°ãƒ©ãƒ ã§å‹¤å‹™ã•ã‚ŒãŸç«¹ç”°æ‚ 哉ã•ã‚“ã«ã‚ˆã‚‹å¯„稿ã§ã™ã€‚ ã¯ã˜ã‚ã« 2023年度ã®PFNå¤å£ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ³ã«å‚åŠ ã—ãŸã€æ±äº¬å¤§å¦å¤§å¦é™¢å·¥å¦ç³»ç ”究科ã®ç«¹ç”°æ‚ 哉ã¨ç”³ã—ã¾ã™ã€‚å¦éƒ¨ã§ã¯ç”»åƒç”Ÿæˆã®ç ”究をã—ã¦ã„ã¾ã—ãŸãŒã€æŠ€è¡“ã®ç¤¾ä¼šå®Ÿè£…をより俯瞰的ã«å¦ã³ãŸã„ã¨æ€ã„ã€ç¾åœ¨ã¯æŠ€è¡“経営戦略å¦å°‚æ”»ã§æ•™è‚²å·¥å¦ã®ç ”究をã—ã¦ã„ã¾ã™ã€‚ インターンã§ã¯ã€Œæ©Ÿæ¢°å¦ç¿’技術ã®ç¤¾ä¼šå®Ÿè£…ã€ã‚’テーマã«ã€LLM（Large Language Model）ã«ãƒ‰ãƒ¡ã‚¤ãƒ³çŸ¥è˜ã‚’ç¿’å¾—ã•ã›ã‚‹ã“ã¨ã«å–り組ã¿ã¾ã—ãŸã€‚様々ãªè¨å®šã«ãŠã„ã¦ã€ä¸»ã«è‹±èªžã§å¦ç¿’ã•ã‚ŒãŸãƒ¢ãƒ‡ãƒ«ã§ã‚ã‚‹LLaMA2ã«å¯¾ã—ã¦æ—¥æœ¬èªžã®ãƒ‡ãƒ¼ã‚¿ã§ã®Fine-tuningã‚’è¡Œã„ã€LoRAã‚„Instruction Tuningã€ãƒ‰ãƒ¡ã‚¤ãƒ³çŸ¥è˜ã®ç¿’å¾—ã«é–¢ã™ã‚‹çŸ¥è¦‹ã‚’å¾—ã‚‹ã“ã¨ãŒã§ããŸã¨æ€ã„ã¾ã™ã€‚本記事ã§ã¯ã€ãã“ã§åˆ©ç”¨ã—ãŸæŠ€è¡“ã®ç´¹ä»‹ã¨ã€æ—¥æœ¬èªžã«ãŠã‘るドメイン知è˜ã®ç¿’å¾—ã«é–¢ã™ã‚‹å®Ÿé¨“ã€
大è¦æ¨¡è¨€èªžãƒ¢ãƒ‡ãƒ«ã®é©šç•°ã¨è„…å¨
2022å¹´11月ã«Open AIãŒå…¬é–‹ã—ãŸChatGPTãŒä¸–ç•Œã§æ³¨ç›®ã‚’集ã‚ã¦ã„る。一般ドメインã‹ã¤å¤šè¨€èªžã§ã€å¾“æ¥ã®ãƒãƒ£ãƒƒãƒˆãƒœãƒƒãƒˆã¨ã¯ãƒ¬ãƒ™ãƒ«ã®ç•°ãªã‚‹é«˜å“質ã®å¯¾è©±ã‚’リアルタイムã«å®Ÿç¾ã™ã‚‹ã‚µãƒ¼ãƒ“スを（Research Preview版ã§ã¯ã‚ã‚‹ãŒï¼‰ç„¡æ–™ã§æä¾›ã—ã€æ¤œç´¢ã€é‡‘èžã€åºƒå‘Šã€æ•™è‚²ã€æ³•å‹™ãªã©ã®åºƒç¯„囲ãªåˆ†é‡Žã®è»¢â€¦
LLaMA: Open and Efficient Foundation Language Models - Meta Research | Meta Research
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is co
What are foundation models? | IBM Research Blog
The future of AI is flexible, reusable AI models that can be applied to just about any domain or industry task. Over the last decade, we’ve seen an explosion of applications for artificial intelligence. In that time, we’ve seen AI go from a purely academic endeavor to a force powering actions across myriad industries and affecting the lives of millions each day. In recent years, we’ve managed to b
æ˜ åƒå†…ã®çŠ¬ã‚’猫ã«å¤‰ãˆã‚‰ã‚Œã‚‹AI　テã‚ストã®ã¿ã§å‹•ç”»ç·¨é›†ãŒå¯èƒ½ã€€Googleãªã©ã€ŒDreamixã€é–‹ç™º
Innovative Tech： ã“ã®ã‚³ãƒ¼ãƒŠãƒ¼ã§ã¯ã€ãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ã®æœ€æ–°ç ”究を紹介ã™ã‚‹Webメディア「Seamlessã€ã‚’主宰ã™ã‚‹å±±ä¸‹è£•æ¯…æ°ãŒåŸ·ç†ã€‚æ–°è¦æ€§ã®é«˜ã„科å¦è«–文を山下æ°ãŒãƒ”ックアップã—ã€è§£èª¬ã™ã‚‹ã€‚Twitter: ï¼ shiropen2 Google Researchã¨ã‚¤ã‚¹ãƒ©ã‚¨ãƒ«ã®The Hebrew University of Jerusalemã«æ‰€å±žã™ã‚‹ç ”究者らãŒç™ºè¡¨ã—ãŸè«–文「Dreamix: Video Diffusion Models are General Video Editorsã€ã¯ã€ãƒ†ã‚ストã«å¿œã˜ãŸå‹•ç”»ç·¨é›†ãŒã§ãる拡散モデルをæ案ã—ãŸç ”ç©¶å ±å‘Šã§ã‚る。 テã‚ストã‹ã‚‰ç”»åƒã‚’生æˆã™ã‚‹æ‹¡æ•£ãƒ¢ãƒ‡ãƒ«ã¨åŒæ§˜ã«ã€ãƒ†ã‚ストã‹ã‚‰å‹•ç”»ã¸ã®å¤‰æ›ãƒ¢ãƒ‡ãƒ«ã‚‚è¿‘å¹´æ案ã•ã‚Œã¦ã„る。ã ãŒãƒ†ã‚ストã«å¿œã˜ã¦å‹•ç”»ç·¨é›†ã§ãる拡散モデルã¯ç¾çŠ¶ã§ã¯ã»ã¨ã‚“ã©ãªã„。 ã“ã®ç ”究ã§ã¯ã€å…¥åŠ›å‹•ç”»ã‚’テã‚ストプãƒãƒ³ãƒ—トã§é«˜åº¦ã«ç·¨é›†
MusicLM
MusicLM: Generating Music From Text |paper|dataset| Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, Christian Frank Google Research Abstract We introduce MusicLM, a model generating high-fidelity music from text descriptions such as "a calming violin melody bac
google-research/maxim – Run with an API on Replicate
Run time and cost This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 137 seconds. The predict time for this model varies significantly based on the inputs. MAXIM: Multi-Axis MLP for Image Processing (CVPR 2022 Oral) This repo is the official implementation of [CVPR 2022 Oral] paper: “MAXIM: Multi-Axis MLP for Image Processing†by Zhengzhong Tu, Hossein Talebi, Han Zha
Kubernetesã«ãŠã‘るコンテナ起動時間高速化ã«å‘ã‘ãŸæ¤œè¨Ž - Preferred Networks Research & Development
本投稿ã¯PFN2022 å¤å£å›½å†…インターンシップã«å‚åŠ ã•ã‚ŒãŸæ±Ÿå¹³æ™ºä¹‹ã•ã‚“ã«ã‚ˆã‚‹å¯„稿ã§ã™ã€‚ ã¯ã˜ã‚ã« PFN2022 å¤å£å›½å†…インターンシップã«å‚åŠ ã—ã¦ã„ãŸæ±Ÿå¹³æ™ºä¹‹ã§ã™ã€‚ç¾åœ¨ä¿®å£«ï¼‘å¹´ã§ã€å¤§å¦ã§ã¯åˆ†æ•£ã‚·ã‚¹ãƒ†ãƒ やクラウド技術ã«ã¤ã„ã¦ç ”究ã—ã¦ã„ã¾ã™ã€‚ 今回ã®ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ³ã‚·ãƒƒãƒ—ã§ã¯ã€ã€ŒJP04. Kubernetesã«ãŠã‘るコンテナ実行環境ã®æ”¹å–„ã€ã¨ã„ã†ãƒ†ãƒ¼ãƒžã§ã‚³ãƒ³ãƒ†ãƒŠèµ·å‹•æ™‚é–“ã®é«˜é€ŸåŒ–ã«å–り組ã¿ã¾ã—ãŸã€‚ 背景 PFNã§ã¯æ©Ÿæ¢°å¦ç¿’基盤ã¨ã—ã¦Kubernetesクラスタを使用ã—ã¦ãŠã‚Šã€ãƒªã‚µãƒ¼ãƒãƒ£ã‚„エンジニアã¯Kubernetesクラスタ上ã®Pod内ã§æ©Ÿæ¢°å¦ç¿’やシミュレーションãªã©ã®è¨ˆç®—ã‚’è¡Œã£ã¦ã„ã¾ã™ã€‚スケジューラã«ã‚ˆã£ã¦ãƒŽãƒ¼ãƒ‰ã«ã‚¢ã‚µã‚¤ãƒ³ã•ã‚ŒãŸå¾Œã«Pod内ã«ã‚³ãƒ³ãƒ†ãƒŠãŒèµ·å‹•ã•ã‚Œã¾ã™ãŒã€ãƒŽãƒ¼ãƒ‰ä¸Šã«ã‚³ãƒ³ãƒ†ãƒŠã‚¤ãƒ¡ãƒ¼ã‚¸ã®ã‚ャッシュãŒãªã„å ´åˆã«ã‚³ãƒ³ãƒ†ãƒŠèµ·å‹•ãŒé…ã„ã¨ã„ã†å•é¡ŒãŒã‚ã‚Šã¾ã—ãŸã€‚計算ã¯PFNã®ç ”究開発ã«ãŠã‘る主è¦
Preferred Networks ã«ãŠã‘ã‚‹ç ”ç©¶æ´»å‹• - Preferred Networks Research & Development
ã“ã‚“ã«ã¡ã¯ã€æ–°ã—ã執行役員兼 Chief Research Strategist ã«å°±ä»»ã—ãŸç§‹è‘‰ã§ã™ã€‚就任ã®æŒ¨æ‹¶ã‚’å…¼ãã¦ã€PFN ã«ãŠã‘ã‚‹ç ”ç©¶æ´»å‹•ã«é–¢ã™ã‚‹è€ƒãˆã‚’共有ã—ãŸã„ã¨æ€ã„ã¾ã™ã€‚ PFN ã«ãŠã‘ã‚‹ç ”ç©¶ã¨ã¯ä½•ã‹ï¼Ÿ 何ãŒç ”究ã§ã‚り何ãŒç ”究ã§ãªã„ã‹ã¨ã„ã†å¢ƒç•Œã‚’引ãã®ã¯éžå¸¸ã«é›£ã—ãã€ã¾ãŸãれをç©æ¥µçš„ã«è¡Œã†æ„味もã‚ã‚Šã¾ã›ã‚“ã€‚ç ”ç©¶ã¨ã¯ã€Œç ”ãŽæ¾„ã¾ã—究ã‚ã‚‹ã“ã¨ã€ã‚’語義ã¨ã—ã€ä¸€èˆ¬ã«ã€ç‰©äº‹ã«ã¤ã„ã¦æ·±ã調査・考察を行ã„事実を解明ã—ãŸã‚Šç™ºæ˜Žã‚’è¡Œã£ãŸã‚Šã™ã‚‹ã“ã¨ã‚’指ã—ã¾ã™ã€‚ PFN ã§ã¯æŒ‘戦的ã§ã‚ã‚Šä¸ç¢ºå®Ÿæ€§ã®é«˜ã„プãƒã‚¸ã‚§ã‚¯ãƒˆãŒå¤§éƒ¨åˆ†ã‚’å ã‚ã¦ãŠã‚Šã€ã»ã¼å…¨ã¦ã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆãŒå°‘ãªã‹ã‚‰ãšç ”究的å´é¢ã‚’ä¼´ã„ã¾ã™ã€‚深層å¦ç¿’関連ã®ã‚³ã‚¢æŠ€è¡“ã®ç ”究開発ã¯å‹¿è«–ã€ãã®å¿œç”¨ã«é–¢ã—ã¦ã‚‚データやタスクã«å¿œã˜ãŸé©åˆ‡ãªæ‰‹æ³•ã®é¸æŠžã‚„éžè‡ªæ˜Žãªå·¥å¤«ãŒãªã‘ã‚Œã°ä¸Šæ‰‹ãã„ã‹ãªã„ã“ã¨ãŒæ®†ã©ã§ã™ã€‚ã¾ãŸã€ãƒãƒœãƒ†ã‚£ã‚¯ã‚¹ã€ã‚³ãƒ³ãƒ”ュータビジョンã€è‡ªç„¶è¨€èªžå‡¦ç†ç‰ã®ã‚ˆã†ãªå¤š
OpenAIã€å¤±æ•—ã‚’æˆåŠŸã«ç½®ãæ›ãˆã¦å¦ç¿’ã™ã‚‹å¼·åŒ–å¦ç¿’アルゴリズム「HERã€ç™ºè¡¨
OpenAIã¯ã€å¤±æ•—ã‹ã‚‰å¦ã¶å¼·åŒ–å¦ç¿’アルゴリズム「HER（Hindsight Experience Replay）ã€ã¨ã€ãã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ を使用ã—ã¦ç‰©ç†ãƒãƒœãƒƒãƒˆã§å‹•ä½œã™ã‚‹ãƒ¢ãƒ‡ãƒ«ã‚’訓練ã™ã‚‹ãŸã‚ã®8ã¤ã®ã‚·ãƒŸãƒ¥ãƒ¬ãƒ¼ãƒˆã•ã‚ŒãŸãƒãƒœãƒƒãƒˆç’°å¢ƒã‚’発表ã—ã¾ã—ãŸã€‚ 訓練ã™ã‚‹ãƒãƒœãƒƒãƒˆç’°å¢ƒã«ã¯ã€Fetch researchプラットフォームã¨ShadowHandãƒãƒœãƒƒãƒˆãŒå«ã¾ã‚Œã¾ã™ã€‚ãƒãƒœãƒƒãƒˆã‚¢ãƒ¼ãƒ ãŒç‰©ä½“ã‚’æ“作ã™ã‚‹ã„ãã¤ã‹ã®ã‚¿ã‚¹ã‚¯ãŒç”¨æ„ã•ã‚Œã¦ãŠã‚Šã€ã©ã‚Œã‚‚目標ã§ã‚るゴールãŒè¨å®šã•ã‚Œã‚´ãƒ¼ãƒ«ã«å‘ã‹ã£ã¦å‹•ä½œã—ã¾ã™ã€‚ 例ãˆã°ã€ä»¥ä¸‹ã®ç”»åƒã¯ã€é»’ã®ãƒ‘ックを押ã™ã‚ˆã†ã«å¼¾ã„ã¦èµ¤ä¸¸ï¼ˆã‚´ãƒ¼ãƒ«ï¼‰ã«åœæ¢ã•ã›ã‚‹ã¨ã„ã†ã‚¿ã‚¹ã‚¯ãŒè¡Œã‚ã‚Œã¾ã™ã€‚ HERã®ç‰¹å¾´ã¯ã€ä»®ã«ã‚´ãƒ¼ãƒ«ã¨é•ã†å ´æ‰€ã«é»’パックãŒåœæ¢ã—ãŸå ´åˆã§ã‚‚ã€ãã‚Œã¯ãã‚Œã§æˆåŠŸã¨ç½®ãæ›ãˆã¦å¦ç¿’ã—ã¾ã™ã€‚é•ã†å ´æ‰€ã¸é»’パックãŒã‚¹ãƒ©ã‚¤ãƒ‰ã—ãŸã¨ã—ã¦ã‚‚ã€ãã“ã‚’ãƒãƒ¼ãƒãƒ£ãƒ«ã‚´ãƒ¼ãƒ«ã¨ã—ã¦å†è¨å®šã™ã‚‹ã“ã¨ã§ä»Šå¾Œã¸ç”Ÿã‹ã™ã¨ã€‚ ã‚‚
Microsoftã€äººå·¥çŸ¥èƒ½ãƒ©ãƒœã€ŒMicrosoft Research AIã€è¨ç«‹
ç±³Microsoftã¯7月12日（ç¾åœ°æ™‚間）ã€äººå·¥çŸ¥èƒ½ï¼ˆAIï¼‰å°‚é–€ç ”ç©¶ãƒ©ãƒœã€ŒMicrosoft Research AIã€ï¼ˆMSR AI）を立ã¡ä¸Šã’ãŸã¨ç™ºè¡¨ã—ãŸã€‚米ワシントン州レッドモンドã®æœ¬ç¤¾ã‚ャンパスã«è¨ç½®ã—ã€åºƒç¯„ãªAIプãƒã‚¸ã‚§ã‚¯ãƒˆã®ãƒãƒ–ã¨ã™ã‚‹ã€‚トップã«ç«‹ã¤ã®ã¯ãƒ‡ã‚£ãƒ¼ãƒ—ラーニング分野ã§åを知られるMicrosoft Researchã®ãƒ†ã‚¯ãƒ‹ã‚«ãƒ«ãƒ•ã‚§ãƒãƒ¼ã€ã‚¨ãƒªãƒƒã‚¯ãƒ»ãƒ›ãƒ¼ãƒ“ッツåšå£«ã 。 åŒç¤¾ã¯æ˜¨å¹´9月ã€AI採用ã®è£½å“やサービスã®é–‹ç™ºã‚’主眼ã¨ã™ã‚‹5000人è¦æ¨¡ã®æ–°éƒ¨é–€ã€ŒMicrosoft AI and Research Groupã€ã‚’è¨ç«‹ã—ãŸã€‚ MSR AIã®ç›®çš„ã¯ã€ã‚ˆã‚Šæ±Žç”¨çš„ãªå¦ç¿’システムã®æ§‹ç¯‰ã 。100人以上ã®ç ”究者ãŒå¤šæ§˜ãªAI関連プãƒã‚¸ã‚§ã‚¯ãƒˆã«å–り組む。 åŒç¤¾ãŒ2月ã«ç™ºè¡¨ã—ãŸãƒ‰ãƒãƒ¼ãƒ³ã®AIå¦ç¿’å‘ã‘シミュレーション環境ã®ãƒ—ãƒã‚¸ã‚§ã‚¯ãƒˆã‚‚ã“ã®ãƒ©ãƒœã«çµ±åˆã•ã‚Œã‚‹ã€‚ Microsoft AI and
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
AI in software engineering at Google: Progress and the path ahead
VideoPoet – Google Research
stabilityai (Stability AI)
BigCode for Large Language Models - ServiceNow Blog
データマイニング・機械å¦ç¿’分野ã®æ¦‚è¦
{{#tags}}- {{label}}
{{/tags}}