STaR: Bootstrapping Reasoning With Reasoning
Generating step-by-step "chain-of-thought" rationales improves language model performance on complex reasoning tasks like mathematics or commonsense question-answering. However, inducing language model rationale generation currently requires either constructing massive rationale datasets or sacrificing accuracy by using only few-shot inference. We propose a technique to iteratively leverage a smal
テã‚ストã‹ã‚‰ã®å®Ÿä¸–ç•Œç†è§£ã«å‘ã‘ã¦
IBIS2023 ä¼ç”»ã‚»ãƒƒã‚·ãƒ§ãƒ³1 Vision and Languageã®æœ€å‰ç·š テã‚ストã‹ã‚‰ã®å®Ÿä¸–ç•Œç†è§£ã«å‘ã‘ã¦
Contrastive Representation Learning
Date: May 31, 2021 | Estimated Reading Time: 39 min | Author: Lilian Weng The goal of contrastive representation learning is to learn such an embedding space in which similar sample pairs stay close to each other while dissimilar ones are far apart. Contrastive learning can be applied to both supervised and unsupervised settings. When working with unsupervised data, contrastive learning is one of
NLPã¨Vision-and-Languageã®åŸºç¤Žãƒ»æœ€æ–°å‹•å‘ (2) / DEIM Tutorial Part 2 Vision-and-Language
DEIM2023 第15回データ工å¦ã¨æƒ…å ±ãƒžãƒã‚¸ãƒ¡ãƒ³ãƒˆã«é–¢ã™ã‚‹ãƒ•ã‚©ãƒ¼ãƒ©ãƒ ãƒãƒ¥ãƒ¼ãƒˆãƒªã‚¢ãƒ«è¬›æ¼”資料 Part2: Vision-and-Language
The Best GPUs for Deep Learning in 2023 — An In-depth Analysis
Deep learning is a field with intense computational requirements, and your choice of GPU will fundamentally determine your deep learning experience. But what features are important if you want to buy a new GPU? GPU RAM, cores, tensor cores, caches? How to make a cost-efficient choice? This blog post will delve into these questions, tackle common misconceptions, give you an intuitive understanding
30分ã§å®Œå…¨ç†è§£ã™ã‚‹Transformerã®ä¸–ç•Œ
ã¯ã˜ã‚ã« åˆã‚ã¾ã—ã¦ã€‚ZENKIGENデータサイエンスãƒãƒ¼ãƒ ã®ã¯ã¾ãªã™ã§ã™ã€‚æ£å¼ãªæ‰€å±žã¯DeNAデータ本部AI技術開発部ãªã®ã§ã™ãŒ[1]ã€æ¥å‹™å§”託ã¨ã„ã†å½¢ã§ä»Šå¹´åº¦ã‹ã‚‰æ·±å±¤å¦ç¿’ç³»ã®é–‹ç™ºç‰ã«æºã‚ã£ã¦ã„ã¾ã™ã€‚ 深層å¦ç¿’界隈ã§ã¯ã€2017å¹´ã«è¡æ’ƒçš„ãªã‚¿ã‚¤ãƒˆãƒ«ï¼ˆAttention Is All You Need）ã®è«–æ–‡ãŒç™ºè¡¨ã•ã‚Œã¦ã‹ã‚‰æ—©5å¹´åŠã€å…ƒå‡ºè‡ªã®æ©Ÿæ¢°ç¿»è¨³ã‚¿ã‚¹ã‚¯ã‚’大ãã越ãˆã¦ã€Transformer関連ã®æŠ€è¡“ãŒæ§˜ã€…ãªé ˜åŸŸã§ç”¨ã„られる汎用アーã‚テクãƒãƒ£ã¨ã—ã¦ç›®è¦šã¾ã—ã発展ã—続ã‘ã¦ã„ã¾ã™ã€‚ 今回ã¯ãã‚“ãªTransformerãŒç¾æ™‚点ã¾ã§ã«ã©ã®ã‚ˆã†ã«æ´»ç”¨ã•ã‚Œã¦ããŸã‹ã€ã¾ãŸã€ã©ã®ã‚ˆã†ã«å·¥å¤«ã•ã‚Œã¦ããŸã‹ã‚’ã–ã£ãã‚Šã¨ä¿¯çž°ã—ã€æµã‚Œã‚’ãŠã•ã‚‰ã„ã™ã‚‹ç›®çš„ã®è¨˜äº‹ã«ãªã‚Šã¾ã™ã€‚本記事ã®å¤§æž ã¯ã€2021年時点ã§ã®ã‚µãƒ¼ãƒ™ã‚¤è«–æ–‡ã§ã‚ã‚‹ A Survey of Transformers ã«å€£ã„ã¤ã¤ã€é©å®œã€2023å¹´2月上旬ç¾åœ¨ã¾
Making Deep Learning go Brrrr From First Principles
Making Deep Learning Go Brrrr From First Principles So, you want to improve the performance of your deep learning model. How might you approach such a task? Often, folk fall back to a grab-bag of tricks that might've worked before or saw on a tweet. "Use in-place operations! Set gradients to None! Install PyTorch 1.10.0 but not 1.10.1!" It's understandable why users often take such an ad-hoc appro
ECCV2022ã®æžåˆˆã‚Šè«–æ–‡ (3) - Qiita
æ¦‚è¦ ã“ã®è¨˜äº‹ã§ã¯ã€ECCV2022ã®æžåˆˆã‚Šè«–文を紹介ã—ã¾ã™ã€‚1 ãã®1 ãã®2 Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruning 概è¦ï¼šNASベースã®æžåˆˆã‚Šã€‚(NASã§åˆˆã£ã¦Fine-tuneã™ã‚‹) æ–°è¦æ€§ï¼š(1) 探索時ã¨æŽ¢ç´¢å¾Œã®ç²¾åº¦ãŒå¤‰ã‚ã‚‹ (2)Fine-tuneãŒå¿…è¦ ãªèª²é¡Œã®å¯¾ç–ã‚’ã™ã‚‹ã€‚ ã‚モ：(1) ãƒã‚¹ã®é¢ã®å¤‰åŒ–ãŒå°‘ãªã„å ´åˆã¯è¡¨ç¾èƒ½åŠ›ãŒé«˜ã„ã“ã¨ã‚’実験的ã«ç¤ºã—ãŸã€‚(2)EKG(アンサンブル)ã§é«˜é€Ÿã«æŽ¢ã™ã€‚ 評価：ResNetã§å®Ÿé¨“ã—ãŸã€‚ SPViT: Enabling Faster Vision Transformers via Latency-aware Soft Token Pruning 概è¦ï¼šSPViT. Latency-awareã§ç”»åƒæ¯Žã«æœ€é©åŒ–ã™ã‚‹ã‚¿ã‚¤ãƒ—ã®
å¿™ã—ã„人ã®ãŸã‚ã®Tuning Playbook | Hippocampus's Garden
å¿™ã—ã„人ã®ãŸã‚ã®Tuning PlaybookJanuary 26, 2023 |  11 min read |  2,712 views jadeep-learning先日ã€Google Researchã¨Harvard大å¦ã®ãƒ¡ãƒ³ãƒãƒ¼ãŒã€ŽDeep Learning Tuning Playbookã€ã‚’公開ã—ã¾ã—ãŸã€‚ã“ã®è³‡æ–™ã¯ã€æ€§èƒ½ã®è‰¯ã„深層å¦ç¿’モデルを作る方法（特ã«ã€ãƒã‚¤ãƒ‘ーパラメータã®æ±ºã‚方）を解説ã—ãŸã‚‚ã®ã§ã™ã€‚ãƒã‚¤ãƒ‘ーパラメータã¯æ·±å±¤å¦ç¿’ã®æˆå¦ã‚’支é…ã™ã‚‹é‡è¦ãªè¦ç´ ã®ä¸€ã¤ã§ã™ã€‚ã—ã‹ã—ã€ãƒã‚¤ãƒ‘ーパラメータã®æ±ºã‚æ–¹ã«é–¢ã™ã‚‹çŸ¥è˜ã¯è«–文や教科書ã§ã¯å–り上ã’られãšã€ã€Œé”人ãŸã¡ã€ã«å±žäººåŒ–ã—ã¦ã„ã¾ã—ãŸã€‚Tuning Playbookã¯ã€ã“れを言語化ã—ã¦è³‡æ–™ã«ã¾ã¨ã‚よã†ã¨ã„ã†é‡è¦ãªè©¦ã¿ã§ã™ 1。 Tuning Playbookã¯ç´ 晴らã—ã„資料ã§ã™ãŒã€ä»–人ã«æ°—軽ã«è–¦ã‚ã‚‹ã«ã¯é•·ã™ãŽã‚‹ã¨æ„Ÿã˜ã¾ã—ãŸã€‚ãã“ã§
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
DeepSpeed Meetup in Japan on May 23, 2024
HammingMesh: A Network Topology for Large-Scale Deep Learning
Revealing the Dark Secrets of Masked Image Modeling
FastMIM: Expediting Masked Image Modeling Pre-training for Vision
MimCo: Masked Image Modeling Pre-training with Contrastive Teacher
ICLR 2023 Conference
Stanford MedAI
GitHub - NVIDIA/FasterTransformer: Transformer related optimization, including BERT, GPT
MoEfication: Transformer Feed-forward Layers are Mixtures of Experts
potts-xcs224u-webinar2023 - public version
https://qiita.com/tags/%E8%AB%96%E6%96%87%E8%AA%AD%E3%81%BF/feed
{{#tags}}- {{label}}
{{/tags}}