Unsupervised batch normalization
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2020, Virtual, Online, Amerika Birleşik Devletleri, 14 - 19 Haziran 2020, cilt.2020-June, ss.3994-3999, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Cilt numarası: 2020-June
- Doi Numarası: 10.1109/cvprw50498.2020.00467
- Basıldığı Şehir: Virtual, Online
- Basıldığı Ülke: Amerika Birleşik Devletleri
- Sayfa Sayıları: ss.3994-3999
- Boğaziçi Üniversitesi Adresli: Evet
Özet
Batch Normalization is a widely used tool in neural networks to improve the generalization and convergence of training. However, on small datasets due to the difficulty of obtaining unbiased batch statistics it cannot be applied effectively. In some cases, even if there is only a small labeled dataset available, there are larger unlabeled datasets from the same distribution. We propose using such unlabeled examples to calculate batch normalization statistics, which we call Unsupervised Batch Normalization (UBN). We show that using unlabeled examples for batch statistic calculations results in a reduction of the bias of the statistics, as well as regularization leveraging the data manifold. UBN is easy to implement, computationally inexpensive and can be applied to a variety problems. We report results on monocular depth estimation, where obtaining dense labeled examples is difficult and expensive. Using unlabeled samples, and UBN, we obtain an increase in accuracy of more than 6% on the KITTI dataset, compared to using traditional batch normalization only on the labeled samples.