Implementation of Layer-sequential unit-variance (LSUV) initialization which is proposed by All you need is a good init in PyTorch.
- PyTorch 1.0+
from lsuv import lsuv_init
model = lsuv_init(ResNet34(), train_loader, needed_std=1.0, std_tol=0.1,
max_attempts=10, do_orthonorm=True, device=device)