Foundations of Vector Retrieval
Vectors are universal mathematical objects that can represent text, images, speech, or a mix of these data modalities. That happens regardless of whether data is represented by hand-crafted features or learnt embeddings. Collect a large enough quantity of such vectors and the question of retrieval becomes urgently relevant: Finding vectors that are more similar to a query vector. This monograph is
The Llama 3 Herd of Models | Research - AI at Meta
概è¦Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirica
Mixture-of-Agents Enhances Large Language Model Capabilities
Recent advances in large language models (LLMs) demonstrate substantial capabilities in natural language understanding and generation tasks. With the growing number of LLMs, how to harness the collective expertise of multiple LLMs is an exciting open direction. Toward this goal, we propose a new approach that leverages the collective strengths of multiple LLMs through a Mixture-of-Agents (MoA) met
公開論文ã‹ã‚‰å¦ã¶ Google ã®ãƒ†ã‚¯ãƒŽãƒã‚¸ãƒ¼ : パート 3：データベース技術編 | Google Cloud å…¬å¼ãƒ–ãƒã‚°
Google Cloud ã®ã‚µãƒ¼ãƒ“スã¯ã€Google ãŒé•·å¹´ã«ã‚ãŸã£ã¦æ§‹ç¯‰ã—ã¦ããŸã‚°ãƒãƒ¼ãƒãƒ«ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã€ãã—ã¦ã€ä¸–ç•Œå„地ã®ãƒ‡ãƒ¼ã‚¿ã‚»ãƒ³ã‚¿ãƒ¼ã«ã‚ˆã£ã¦æä¾›ã•ã‚Œã¦ã„ã¾ã™ã€‚ã“ã‚Œã¯ã€Google 検索をã¯ã˜ã‚ã¨ã™ã‚‹ã•ã¾ã–ã¾ãª Google ã®ã‚µãƒ¼ãƒ“スを支ãˆã‚‹ã‚¤ãƒ³ãƒ•ãƒ©ã§ã‚‚ã‚ã‚Šã€ãã®ä¸Šã§ã¯ã€Google 独自ã®æŠ€è¡“を活用ã—ãŸã•ã¾ã–ã¾ãªãƒŸãƒ‰ãƒ«ã‚¦ã‚§ã‚¢ãŒç¨¼åƒã—ã¦ã„ã¾ã™ã€‚ Google ã®ã‚¨ãƒ³ã‚¸ãƒ‹ã‚¢ã¯ã€è‡ªåˆ†ãŸã¡ãŒé–‹ç™ºã—ãŸæŠ€è¡“ã®è©³ç´°ã‚’è«–æ–‡ã¨ã—ã¦å…¬é–‹ã—ã¦ãŠã‚Šã€ã“ã‚Œã¾ã§ã«å…¬é–‹ã•ã‚ŒãŸè«–æ–‡ã¯ã€Google ã®ç ”究ãƒãƒ¼ãƒ ã€Google Research ã® Web サイトã«ã‚ã‚‹ Publication Database ã§æ¤œç´¢ã§ãã¾ã™ã€‚ã“ã®ãƒ–ãƒã‚°ã‚·ãƒªãƒ¼ã‚ºã§ã¯ã€æ¬¡ã®4ã¤ã®åˆ†é‡Žã«åˆ†ã‘ã¦ã€Google Cloud ã®æŠ€è¡“ã«é–¢é€£ã®æ·±ã„論文を紹介ã—ã¦ã„ãã¾ã™ã€‚ 分散処ç†åŸºç›¤ï¼ˆã‚³ãƒ³ãƒ†ãƒŠæŠ€è¡“）ã¨ãƒ‡ãƒ¼ã‚¿ã‚»ãƒ³ã‚¿ãƒ¼ ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯æŠ€è¡“ データベ
Dense Passage Retrieval for Open-Domain Question Answering
@inproceedings{karpukhin-etal-2020-dense, title = "Dense Passage Retrieval for Open-Domain Question Answering", author = "Karpukhin, Vladimir and Oguz, Barlas and Min, Sewon and Lewis, Patrick and Wu, Ledell and Edunov, Sergey and Chen, Danqi and Yih, Wen-tau", editor = "Webber, Bonnie and Cohn, Trevor and He, Yulan and Liu, Yang", booktitle = "Proceedings of the 2020 Conference on Empirical Metho
APE
Large Language Models Are Human-Level Prompt EngineersICLR 2023Yongchao Zhou*, Andrei Ioan Muresanu*, Ziwen Han*, Keiran Paster, Silviu Pitis, Harris Chan, Jimmy Ba[ ArXiv | GitHub | Colab  | Demo ] By conditioning on natural language instructions, large language models (LLMs) have displayed impressive capabilities as general-purpose computers. However, task performance depends significantly on th
Language Models are Few-Shot Learners
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few
ã€è‡ªç„¶è¨€èªžå‡¦ç†ç·¨ã€‘機械å¦ç¿’/ディープラーニングã®ãŠã™ã™ã‚è«–æ–‡30é¸ | スã‚ルアップAI Journal
ã¯ã˜ã‚㫠今回ã¯è‡ªç„¶è¨€èªžå‡¦ç†ç·¨ã¨ã—ã¦ã€æ©Ÿæ¢°ç¿»è¨³ã«ãŠã‘る最é©ãªèªžå½™ã®ã‚µã‚¤ã‚ºã‚’求ã‚るアルゴリズムやテã‚スト生æˆã‚¿ã‚¹ã‚¯ã«ãŠã‘るデータ拡張ã®æ–°æ‰‹æ³•ãªã©ã®æœ€æ–°è«–文を7本ã”紹介ã—ã¾ã™ï¼è‘—者実装ãŒå…¬é–‹ã•ã‚Œã¦ã„ã‚‹ã‚‚ã®ã¯ã€ãã®æƒ…å ±ã‚‚ä½µã›ã¦ã¾ã¨ã‚ã¾ã—ãŸã€‚è«–æ–‡ã¯è‡ªå‹•æ©Ÿæ¢°å¦ç¿’（AutoML）を専門ã¨ã—ICMLãªã©ã®ãƒˆãƒƒãƒ—カンファレンスã¸ã®è«–文採択経験もã‚る斉藤ã¨ã€éœ€è¦äºˆæ¸¬ãƒ»ç•°å¸¸æ¤œçŸ¥ãªã©æ§˜ã€…ãªåˆ†é‡Žã§æ©Ÿæ¢°å¦ç¿’/ディープラーニングã®ç”£æ¥å¿œç”¨ã«å–り組んã§ããŸå°ç¸£ãŒä¸å¿ƒã¨ãªã£ã¦ã‚¹ã‚ルアップAI講師陣ã«ã¦åŽ³é¸ã—ã¾ã—ãŸã€‚ãœã²ã€ä»Šå¾Œã®å¦ã³ã«ã”活用ãã ã•ã„ï¼ ã¾ãŸã€ãŠã™ã™ã‚ã®è«–æ–‡30é¸ã‚’ã¾ã¨ã‚ã¦ã„る下記ã®è¨˜äº‹ã‚‚åˆã‚ã›ã¦ã”覧ãã ã•ã„。 Vocabulary Learning via Optimal Transport for Neural Machine Translation 実装ã®URL：https://github.com/J
Recurrent Neural Network Based Language Model
Recurrent neural network based language model TomaÌsÌŒ Mikolov1,2 , Martin KarafiaÌt1 , LukaÌsÌŒ Burget1 , Jan “Honza†CÌŒernockyÌ1 , Sanjeev Khudanpur2 1 Speech@FIT, Brno University of Technology, Czech Republic 2 Department of Electrical and Computer Engineering, Johns Hopkins University, USA {imikolov,karafiat,burget,cernocky}@fit.vutbr.cz, [email protected] Abstract A new recurrent neural network
ECCV2022ã®æžåˆˆã‚Šè«–æ–‡ (1) - Qiita
æ¦‚è¦ ã“ã®è¨˜äº‹ã§ã¯ã€ECCV2022ã®æžåˆˆã‚Šè«–文を紹介ã—ã¾ã™ã€‚1 Prune Your Model Before Distill It 概è¦ï¼šæžåˆˆã‚Šã—ã¦ã‹ã‚‰è’¸ç•™ã™ã‚‹æ–¹æ³•ã®æ案 æ–°è¦æ€§ï¼šUnstructuredæžåˆˆã‚Šã‹ã‚‰ã®è’¸ç•™ã§åŠ¹æžœãŒä¸ŠãŒã‚‹ã“ã¨ã‚’示ã—ãŸã€‚ç†è«–解æžã—ã€æžåˆˆã‚Šã¯label smoothã¨åŒã˜åŠ¹æžœãŒã‚ã‚‹ã“ã¨ã‚’示ã—ãŸã€‚ ã‚モ：æžåˆˆã‚Š(Sparsityã«ã‚ˆã‚‹æ£å‰‡åŒ–)ã¯label smoothよりé‡ã¿ãŒå¹³å‡çš„ã«å°ã•ããªã‚‹ãŸã‚蒸留ã—ã‚„ã™ããªã‚‹ã¨ä¸»å¼µã—ãŸã€‚ 評価：ResNet〜MobileNetV2ã§å®Ÿé¨“ã—ãŸã€‚ CPrune: Compiler-Informed Model Pruning for Efficient Target-Aware DNN Execution 概è¦ï¼šCPrune. TVMãªã©ã§ã€ã‚³ãƒ³ãƒ‘イラã«æžåˆˆã‚Šæƒ…å ±ã‚’é€šçŸ¥ã—ã¦æœ€é©åŒ–ã™ã‚‹æžåˆˆã‚Šã€‚ æ–°è¦æ€§ï¼šãƒ¢ãƒ‡ãƒ«åœ§ç¸®ã¨ã‚³ãƒ³ãƒ‘イラ最é©
Rust Bibliography
Rust Bibliography This is a reading list of material relevant to Rust. It includes prior research that has - at one time or another - influenced the design of Rust, as well as publications about Rust. Type system Alias burying - We tried something similar and abandoned it. External uniqueness is unique enough Macros that work together Making ad-hoc polymorphism less ad hoc Region based memory mana
ã€ãƒ¡ã‚¿ã‚µãƒ¼ãƒ™ã‚¤ã€‘基盤モデル / Foundation Models
cvpaper.challenge 㮠メタサーベイ発表スライドã§ã™ã€‚ cvpaper.challengeã¯ã‚³ãƒ³ãƒ”ュータビジョン分野ã®ä»Šã‚’æ˜ ã—ã€ãƒˆãƒ¬ãƒ³ãƒ‰ã‚’創り出ã™æŒ‘戦ã§ã™ã€‚論文サマリ作æˆãƒ»ã‚¢ã‚¤ãƒ‡ã‚£ã‚¢è€ƒæ¡ˆãƒ»è°è«–・実装・論文投稿ã«å–り組ã¿ã€å‡¡ã‚†ã‚‹çŸ¥è˜ã‚’共有ã—ã¾ã™ã€‚ http://xpaperchallenge.org/cv/Read less
è«–æ–‡ã¾ã¨ã‚：（DALL・E 2論文）Hierarchical Text-Conditional Image Generation with CLIP Latents
タイトル：Hierarchical Text-Conditional Image Generation with CLIP Latents リンク：https://cdn.openai.com/papers/dall-e-2.pdf 所属：OpenAI コード：DALL・E 2ã®å‰ã®æ®µéšŽã®GLIDEã¯å…¬é–‹ã•ã‚Œã¦ã„ã‚‹ãŒã€ã“ã‚Œã¯éžå…¬é–‹ï¼ˆ2022å¹´4月時点） ã–ã£ãã‚Šã„ã†ã¨ 訓練済ã¿CLIPã¨æ‹¡æ•£ãƒ¢ãƒ‡ãƒ«ã‚’使ã£ãŸã€ãƒ†ã‚スト→画åƒã«ã‚ˆã‚‹ç”Ÿæˆãƒ¢ãƒ‡ãƒ« CLIPã®ç”»åƒåŸ‹ã‚è¾¼ã¿ç‰¹å¾´ã‚’生æˆã™ã‚‹ã‚ˆã†ãªã€æ‹¡æ•£ãƒ¢ãƒ‡ãƒ«ãƒ™ãƒ¼ã‚¹ã®äº‹å‰åˆ†å¸ƒãƒ¢ãƒ‡ãƒ«ã‚’開発 å…ˆè¡Œç ”ç©¶ã®GLIDEã®æ”¹è‰¯ã¨ã„ã†ä½ç½®ã¥ã‘ã ãŒã€å¤šæ§˜æ€§ã«å¯Œã‚€ç”»åƒç”Ÿæˆã«æˆåŠŸ è¦æ—¨ 2段階ã®ãƒ¢ãƒ‡ãƒ«ã‚’æ案 テã‚ストã‚ャプションを与ãˆã¦CLIPç”»åƒåŸ‹ã‚è¾¼ã¿ã‚’生æˆã™ã‚‹äº‹å‰å‡¦ç† ç”»åƒåŸ‹ã‚è¾¼ã¿ã‚’æ¡ä»¶ã¨ã—ã¦ç”»åƒã‚’生æˆã™ã‚‹ãƒ‡ã‚³ãƒ¼ãƒ€ ç”»åƒè¡¨ç¾ã‚’明示的ã«ç”Ÿæˆã™ã‚‹ã“ã¨ã§ã€å†™å®Ÿæ€§ã¨ã‚ャプションã®
Hierarchical Text-Conditional Image Generation with CLIP Latents
Contrastive models like CLIP have been shown to learn robust representations of images that capture both semantics and style. To leverage these representations for image generation, we propose a two-stage model: a prior that generates a CLIP image embedding given a text caption, and a decoder that generates an image conditioned on the image embedding. We show that explicitly generating image repre
Learning Transferable Architectures for Scalable Image Recognition
Developing neural network image classification models often requires significant architecture engineering. In this paper, we study a method to learn the model architectures directly on the dataset of interest. As this approach is expensive when the dataset is large, we propose to search for an architectural building block on a small dataset and then transfer the block to a larger dataset. The key
ãŠçŸ¥ã‚‰ã›
ランã‚ング
ランã‚ング
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
A zero-inflated endemic-epidemic model with an application to measles time series in Germany
Papers with Code - Language Models are Unsupervised Multitask Learners
Dropout Reduces Underfitting
https://research.google.com/pubs/archive/34417.pdf
Papers with Code - The latest in Machine Learning
{{#tags}}- {{label}}
{{/tags}}