Gradients without Backpropagation
Using backpropagation to compute gradients of objective functions for optimization has remained a mainstay of machine learning. Backpropagation, or reverse-mode differentiation, is a special case within the general family of automatic differentiation algorithms that also includes the forward mode. We present a method to compute gradients based solely on the directional derivative that one can comp
Introducing TensorFlow Graph Neural Networks
Posted by Sibon Li, Jan Pfeifer and Bryan Perozzi and Douglas Yarrington Today, we are excited to release TensorFlow Graph Neural Networks (GNNs), a library designed to make it easy to work with graph structured data using TensorFlow. We have used an earlier version of this library in production at Google in a variety of contexts (for example, spam and anomaly detection, traffic estimation, YouTub
RNNs Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients?
Recurrent neural networks (RNNs) are particularly well-suited for modeling long-term dependencies in sequential data, but are notoriously hard to train because the error backpropagated in time either vanishes or explodes at an exponential rate. While a number of works attempt to mitigate this effect through gated recurrent units, well-chosen parametric constraints, and skip-connections, we develop
è¦æ³¨ç›®ï¼ŸDeepGBM: ニューラルãƒãƒƒãƒˆ+GBDTï¼ˆé€Ÿå ±ï¼‰ - Paoã®ãƒ†ãƒƒã‚¯ãƒ–ãƒã‚°
KDD2019ã®Paper一覧ã§æ°—ã«ãªã‚‹ã‚‚ã®ãŒã‚ã£ãŸã®ã§ç´¹ä»‹ã—ã¾ã™ã€‚ ※記載時点ã§ã¾ã 論文公開ã€ç™ºè¡¨ã•ã‚Œã¦ãŠã‚‰ãšã€ã“ã¡ã‚‰é®®åº¦é‡è¦–ã®è¨˜äº‹ã§ã™ã€‚ 内容ã«èª¤ã‚ŠãŒã‚ã‚‹å¯èƒ½æ€§ã¯å分ã‚ã‚‹ã®ã§ã”了承ãã ã•ã„。 DeepGBMã¨ã¯ データマイニングã®ãƒˆãƒƒãƒ—カンファレンスKDD2019ã§ç™ºè¡¨ã•ã‚Œã‚‹äºˆå®šã®æ‰‹æ³•ã§ã™ã€‚ Guolin Ke, Zhenhui Xu, Jia Zhang, Jiang Bian, and Tie-yan Liu. "DeepGBM: A Deep Learning Framework Distilled by GBDT for Online Prediction Tasks." In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM
AutoML | Literature on Neural Architecture Search
Maintained by Difan Deng and Marius Lindauer. The following list considers papers related to neural architecture search. It is by no means complete. If you miss a paper on the list, please let us know. Please note that although NAS methods steadily improve, the quality of empirical evaluations in this field are still lagging behind compared to other areas in machine learning, AI and optimization.
Graph Neural Networks概観
Graph Neural Networks (GNNs)ã«ã¤ã„ã¦ã¾ã¨ã‚ã¾ã—㟠実際ã«ã©ã‚“ãªã“ã¨ã«ä½¿ãˆã‚‹ã‹ã‚¤ãƒ¡ãƒ¼ã‚¸ã‚’æŒã£ã¦ã‚‚らã†ã“ã¨ã‚’æ„è˜ã—ã¾ã—ãŸã€‚ 内容ã¯ä»¥ä¸‹ã®é€šã‚Šã§ã™ã€‚ ãƒ»ã‚°ãƒ©ãƒ•æ§‹é€ ã¨ã¯ ・GNNsã®ç™ºå±• ・ã„ãã¤ã‹ã®GNNsã®ç‰¹å¾´ ・NLPã«ãŠã‘る利用例 ・ãã®ä»–ã®é ˜åŸŸã«ãŠã‘る利用…
Visualizing memorization in RNNs
Inspecting gradient magnitudes in context can be a powerful tool to see when recurrent units use short-term or long-term contextual understanding. This connectivity visualization shows how strongly previous input characters influence the current target character in an autocomplete problem. For example, in the prediction of “grammar†the GRU RNN initially uses long-term memorization but as more cha
Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
The rising popularity of intelligent mobile devices and the daunting computational cost of deep learning-based models call for efficient and accurate on-device inference schemes. We propose a quantization scheme that allows inference to be carried out using integer-only arithmetic, which can be implemented more efficiently than floating point inference on commonly available integer-only hardware.
統計的声質変æ›ã®ç´¹ä»‹ - nico-opendata
nico-opendataniconicoã§ã¯ã€å¦è¡“分野ã«ãŠã‘る技術発展ã¸ã®å¯„与を目的ã¨ã—ã¦ã€ ç ”ç©¶è€…ã®æ–¹ã‚’対象ã«å„種サービスã®ãƒ‡ãƒ¼ã‚¿ã‚’公開ã—ã¦ã„ã¾ã™ã€‚ ニコニコ動画コメントç‰ãƒ‡ãƒ¼ã‚¿ã‚»ãƒƒãƒˆ(æ ª)ドワンゴåŠã³(有)未æ¥æ¤œç´¢ãƒ–ラジルã¨å›½ç«‹æƒ…å ±å¦ç ”究所ãŒå”力ã—ã¦ç ”究者ã«æä¾›ã—ã¦ã„るデータセットã§ã™ã€‚ ニコニコ動画コメントç‰ã®ãƒ‡ãƒ¼ã‚¿ãŒåˆ©ç”¨å¯èƒ½ã§ã™ã€‚ åˆ©ç”¨ç”³è«‹ãƒ•ã‚©ãƒ¼ãƒ â€»å›½ç«‹æƒ…å ±å¦ç ”究所ã¸ãƒªãƒ³ã‚¯ã—ã¾ã™ ニコニコ大百科データ(æ ª)ドワンゴåŠã³(有)未æ¥æ¤œç´¢ãƒ–ラジルã¨å›½ç«‹æƒ…å ±å¦ç ”究所ãŒå”力ã—ã¦ç ”究者ã«æä¾›ã—ã¦ã„るデータセットã§ã™ã€‚ ニコニコ大百科ã®ãƒ‡ãƒ¼ã‚¿ãŒåˆ©ç”¨å¯èƒ½ã§ã™ã€‚ åˆ©ç”¨ç”³è«‹ãƒ•ã‚©ãƒ¼ãƒ â€»å›½ç«‹æƒ…å ±å¦ç ”究所ã¸ãƒªãƒ³ã‚¯ã—ã¾ã™ Nico-IllustデータセットComicolorization: Semi-Automatic Manga ColorizationChie Furusawa*ã€Kazuyuki Hi
DNN推論用ライブラリ「Menohã€ãƒªãƒªãƒ¼ã‚¹ã«ã¤ã„㦠- Preferred Networks Research & Development
Python以外も使ã„ãŸããªã„ã§ã™ã‹ï¼Ÿ  特ã«DeepLearning界隈ã§ï¼Ž Menoh開発者ã®å²¡ç”°ã§ã™ï¼Žã“ã®è¨˜äº‹ã§ã¯Menohã®ç´¹ä»‹ã¨é–‹ç™ºã«è‡³ã£ãŸå‹•æ©Ÿã«ã¤ã„ã¦èª¬æ˜Žã—ã¾ã™ï¼Ž Menohã®ãƒ¬ãƒã‚¸ãƒˆãƒªï¼šÂ https://github.com/pfnet-research/menoh Menoh（メノウ）ã¯å¦ç¿’済ã¿ã®DNNモデルをONNXå½¢å¼ã‹ã‚‰èªã¿è¾¼ã‚“ã§å‹•ä½œã•ã›ã‚‹æŽ¨è«–専用ã®ãƒ©ã‚¤ãƒ–ラリã§ã™ï¼Žå®Ÿè£…ã¯C++ã§æ›¸ãã¾ã—ãŸãŒï¼ŒC言語ã®ã‚¤ãƒ³ã‚¿ãƒ¼ãƒ•ã‚§ãƒ¼ã‚¹ã‚’æŒãŸã›ã¦ï¼Œä»–ã®è¨€èªžç”¨ã‹ã‚‰ã‚‚ãã®æ©Ÿèƒ½ã‚’呼ã³å‡ºã—ã‚„ã™ãã—ã¦ã‚ã‚Šã¾ã™ï¼Žãƒªãƒªãƒ¼ã‚¹æ™‚点ã§C++版ラッパーã¨C#版ラッパー，Haskell版ラッパーãŒã‚り，Ruby版ラッパーã¨NodeJS版ラッパー，Java（JVM）版ラッパーãŒé–‹ç™ºä¸ã§ã™ï¼Žãƒãƒƒã‚¯ã‚¨ãƒ³ãƒ‰ã«ã¯Intelã®é–‹ç™ºã—ã¦ã„ã‚‹MKL-DNNを採用ã—，GPUãŒç„¡ãã¦ã‚‚Intel CPUãŒä½¿ãˆã‚‹ç’°å¢ƒã§é«˜é€Ÿã«ãƒ¢ãƒ‡ãƒ«ã®
Hyperbolic Attention Networks
We introduce hyperbolic attention networks to endow neural networks with enough capacity to match the complexity of data with hierarchical and power-law structure. A few recent approaches have successfully demonstrated the benefits of imposing hyperbolic geometry on the parameters of shallow networks. We extend this line of work by imposing hyperbolic geometry on the activations of neural networks
WaveGAN Demo
Please try again on a different browser or refresh if you change your mind :) Chris Donahue, Julian McAuley, Miller Puckette This is a demo of our WaveGAN method trained on drum sound effects (paper, code). All drum sounds are synthesized in browser by a neural network. Shortcuts: Keys 1-8 play sounds. Shift+[1-8] changes sounds. Space starts/stops sequencer.
DensePose: Dense Human Pose Estimation In The Wild
In this work, we establish dense correspondences between RGB image and a surface-based representation of the human body, a task we refer to as dense human pose estimation. We first gather dense correspondences for 50K persons appearing in the COCO dataset by introducing an efficient annotation pipeline. We then use our dataset to train CNN-based systems that deliver dense correspondence 'in the wi
DensePose
Dense human pose estimation aims at mapping all human pixels of an RGB image to the 3D surface of the human body. We introduce DensePose-COCO, a large-scale ground-truth dataset with image-to-surface correspondences manually annotated on 50K COCO images. We propose DensePose-RCNN, a variant of Mask-RCNN, to densely regress part-specific UV coordinates within every human region at multiple frames p
Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes
Recurrent neural networks are nowadays successfully used in an abundance of applications, going from text, speech and image processing to recommender systems. Backpropagation through time is the algorithm that is commonly used to train these networks on specific tasks. Many deep learning frameworks have their own implementation of training and sampling procedures for recurrent neural networks, whi
ランã‚ング
ランã‚ング
メンテナンス
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
最近ã®Deep Learning (NLP) 界隈ã«ãŠã‘ã‚‹Attention事情
Anchor Boxes — The key to quality object detection
[DL Hacks]BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding![[DL Hacks]BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://cdn-ak-scissors.b.st-hatena.com/image/square/b6daf77a3db668161a6707d95bec4f65d9f8bdcf/height=288;version=1;width=512/https%3A%2F%2Fcdn.slidesharecdn.com%2Fss_thumbnails%2F20181129suzukibert1-181204064830-thumbnail.jpg%3Fwidth%3D640%26height%3D640%26fit%3Dbounds)
http://archive.xsig.ipsj.or.jp/history/ACSI/2016/download/ACSI2016-tutorial2.pdf
Chatbot and Related Research Paper Notes with Images
{{#tags}}- {{label}}
{{/tags}}