id,collection,dc.contributor.author,dc.date.accessioned,dc.date.available,dc.date.issued,dc.description.abstract[en],dc.format.extent,dc.identifier.uri,dc.language,dc.rights.uri,dc.subject.ddc,dc.subject[en],dc.title,dc.type,dcterms.accessRights.openaire,dcterms.bibliographicCitation.articlenumber,dcterms.bibliographicCitation.journaltitle,dcterms.bibliographicCitation.url,dcterms.bibliographicCitation.volume,refubium.affiliation,refubium.affiliation.other,refubium.note.author,refubium.resourceType.isindependentpub "04e14d65-507b-4f1a-a71a-b9e00685a548","fub188/16","Richter, Lorenz||Boustati, Ayman||Nüsken, Nikolas||Ruiz, Francisco J. R.||Akyildiz, Ömer Deniz","2020-10-22T05:51:39Z","2020-10-22T05:51:39Z","2020","We analyse the properties of an unbiased gradient estimator of the evidence lower bound (ELBO) for variational inference, based on the score function method with leave-one-out control variates. We show that this gradient estimator can be obtained using a new loss, defined as the variance of the log-ratio between the exact posterior and the variational approximation, which we call the log-variance loss. Under certain conditions, the gradient of the log-variance loss equals the gradient of the (negative) ELBO. We show theoretically that this gradient estimator, which we call VarGrad due to its connection to the log-variance loss, exhibits lower variance than the score function method in certain settings, and that the leave-one-out control variate coefficients are close to the optimal ones. We empirically demonstrate that VarGrad offers a favourable variance versus computation trade-off compared to other state-of-the-art estimators on a discrete variational autoencoder (VAE)","25 S.","https://refubium.fu-berlin.de/handle/fub188/28607||http://dx.doi.org/10.17169/refubium-28356","eng","http://www.fu-berlin.de/sites/refubium/rechtliches/Nutzungsbedingungen","500 Naturwissenschaften und Mathematik::510 Mathematik::510 Mathematik","VarGrad||Neural Information Processing Systems||Low-Variance Gradient Estimator","VarGrad: A Low-Variance Gradient Estimator for Variational Inference","Wissenschaftlicher Artikel","open access","10436v1","arXiv.org","https://arxiv.org/abs/2010.10436v1||https://nips.cc/Conferences/2020/AcceptedPapersInitial","2010","Mathematik und Informatik","SFB 1114||Projekt A02||Projekt A05","34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada","no"