Going deeper with Image Transformers
Transformers have been recently adapted for large scale image classification, achieving high scores shaking up the long supremacy of convolutional neural networks. However the optimization of image transformers has been little studied so far. In this work, we build and optimize deeper transformer networks for image classification. In particular, we investigate the interplay of architecture and opt
ABEJA GPTモデルã«ãŠã‘るアーã‚テクãƒãƒ£ã®å·¥å¤« - ABEJA Tech Blog
1. ã¯ã˜ã‚ã« 2. å…ˆè¡Œç ”ç©¶ã‹ã‚‰ã®å¦ã³ 3. å‰æ 4. アーã‚テクãƒãƒ£å¤‰æ›´å€™è£œ 活性化関数ã®å¤‰æ›´ (SwishGLU) Transformer layerã®ä¸¦åˆ—化 biasパラメータ除去 Input-Output Embeddingã®å…±æœ‰ (Weight tying) 5. å°è¦æ¨¡ãƒ¢ãƒ‡ãƒ«ã§ã®å®Ÿé¨“ 実験è¨å®š Transformer layerã®ä¸¦åˆ—化 SwishGLUã®é©ç”¨ Bias parameterã®é™¤åŽ» bias削除ã®å®Ÿé¨“ 最åˆã‚‚ã—ãã¯æœ€å¾Œã®biasã ã‘を残㙠Input-Output Embeddingã®å…±æœ‰ (Weight tying) 6. ä¸è¦æ¨¡ãƒ¢ãƒ‡ãƒ«ã§ã®å®Ÿé¨“ 実験è¨å®š モデルサイズã§ã®æ¯”較 Shared Input output embeddings (weight tying) Transformer layerã®ä¸¦åˆ—化 SwishGLUã®é©ç”¨ 7. 13Bパラメーター
OpenCV 使ã£ã¦ã¿ãŸ ～6. 物体èªè˜ç·¨ï½ž
今度ã¯ã•ã‚‰ã«è¤‡é›‘ãªå½¢ã§ã‚るト音記å·ã‚’èªè˜ã—ã¦ã¿ã‚‹ã€‚複雑ãªå½¢ã®èªè˜ã¯ï¼ˆä¾‹ãˆã°é¡”èªè˜ãªã©ï¼‰ã¯ CascadeClassifier ã§è¡Œã†ã€‚å˜ç´”ãªèªè˜æ©Ÿã‚’多段（カスケード）ã«çµ„ã¿åˆã‚ã›ã‚‹ã“ã¨ã§èªè˜ç²¾åº¦ã‚’上ã’る手法らã—ã„。機械å¦ç¿’ã—ãŸãƒ‡ãƒ¼ã‚¿ã‚’使ã£ã¦èªè˜ã‚’è¡Œã†ã‚ã‘ãªã®ã§ã€å¦ç¿’データを作るã®ãŒã‚ã‚“ã©ãã•ã„。 ã¨æ€ã£ã¦ãŸã‚‰ç´ 晴らã—ã„ページを発見ã—ãŸã€‚ã¾ã•ã«ç¥žãƒ„ール。ã‚ã‚ŠãŒãŸã使ã‚ã›ã¦ã„ãŸã ãã¾ã™ã€‚ã“ã®ã‚µã‚¤ãƒˆã®ã¨ãŠã‚Šã«ã‚„る。ã£ã¦ã„ã†ã¨èª¬æ˜ŽãŒçµ‚ã‚ã£ã¦ã—ã¾ã†ã€‚ ã¨ã‚Šã‚ãˆãšã€opencv_createsamples をインストールã™ã‚‹ã€‚パッケージãŒã‚ã‹ã‚‰ãªã„ã®ã§ apt-file ã§æŽ¢ã™ã€‚ apt-file search opencv_createsamples libopencv-dev パッケージã«å«ã¾ã‚Œã¦ã„る（Ubuntuã®å ´åˆï¼‰ã“ã¨ãŒã‚ã‹ã£ãŸã®ã§ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ«ã€‚ sudo apt-get install
OpenCV 2.4.2ã§åˆ†é¡žå™¨ã‚’作る - shkh's blog
最近ã€OpenCVã§éŠã‚“ã§ã„ã¦ã€ä»˜å±žã®é¡”検出用ã®åˆ†é¡žå™¨ã®ç²¾åº¦ãŒã‚ã¾ã‚Šè‰¯ããªã„ã®ã§ã€è‡ªåˆ†ã§ä½œã£ã¦ã¿ã‚‹ã“ã¨ã«ã—ãŸã€‚ドã‚ュメントãŒã¨ã£æ•£ã‚‰ã‹ã£ã¦ã„ã‚‹ã®ã§ã€ãƒ¡ãƒ¢ã¨ã—ã¦æ®‹ã™ã€‚ 次ã®ç”»åƒã¯ä»˜å±žã®åˆ†é¡žå™¨ã‚’使ã£ã¦æ¤œå‡ºã—ãŸã‚‚ã®ã€‚False-Positiveも多ãã¦ã€ã†ã¾ã顔をæ•æ‰ã§ãã¦ã„ãªã„ã®ãŒã‚ã‹ã‚‹ã€‚ OpenCVã¯ãƒãƒ¼ã‚¸ãƒ§ãƒ³2.4.2を使ã£ãŸã€‚ æ‰‹é †ã¨ã—ã¦ã¯ サンプルã®åŽé›† ラベル付㑠分類器ã®ç”Ÿæˆ ã¨ãªã‚‹ã€‚ 1. サンプルã®åŽé›† 今回ã¯ã€ãƒã‚¸ãƒ†ã‚£ãƒ–サンプルã¨ãƒã‚¬ãƒ†ã‚£ãƒ–サンプル用ã«ã‚ã‚ã›ã¦10392枚集ã‚ãŸã€‚ ç”»åƒæ¤œç´¢ã®APIã§ã¯ã€Bing Search API on AzureãŒä¸€ç•ªã„ã„ã¨æ€ã†ã€‚Googleã¯åˆ¶é™ãŒãã¤ãã¦ã€Yahooã¯ã‚¯ãƒ¬ã‚«ã®ç™»éŒ²ãŒè¦ã‚‹ã€‚ Bingã¯ã“ã¡ã‚‰ã§ç„¡æ–™ãƒ—ランを登録ã™ã‚Œã°è‰¯ã„。使ã†ã¨ãã¯Authヘッダã«ã‚¢ã‚«ã‚¦ãƒ³ãƒˆã‚ーを入れã¦ã‚„ã‚Œã°è‰¯ã„。スクリプトã¯ä»¥ä¸‹ã«ã‚ã’ã¦ãŠã„ãŸã€‚ htt
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations
Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
Stable and low-precision training for large-scale vision-language models
{{#tags}}- {{label}}
{{/tags}}