TY - GEN
T1 - Approximating probability distributions by RELu networks
AU - Mukherjee, Manuj
AU - Tchamkerten, Aslan
AU - Yousefi, Mansoor
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021/4/11
Y1 - 2021/4/11
N2 - How many neurons are needed to approximate a target probability distribution using a neural network with a given input distribution and approximation error? This paper examines this question for the case when the input distribution is uniform, and the target distribution belongs to the class of histogram distributions. We obtain a new upper bound on the number of required neurons, which is strictly better than previously existing upper bounds. The key ingredient in this improvement is an efficient construction of the neural nets representing piecewise linear functions. We also obtain a lower bound on the minimum number of neurons needed to approximate the histogram distributions.
AB - How many neurons are needed to approximate a target probability distribution using a neural network with a given input distribution and approximation error? This paper examines this question for the case when the input distribution is uniform, and the target distribution belongs to the class of histogram distributions. We obtain a new upper bound on the number of required neurons, which is strictly better than previously existing upper bounds. The key ingredient in this improvement is an efficient construction of the neural nets representing piecewise linear functions. We also obtain a lower bound on the minimum number of neurons needed to approximate the histogram distributions.
UR - https://www.scopus.com/pages/publications/85113292729
U2 - 10.1109/ITW46852.2021.9457598
DO - 10.1109/ITW46852.2021.9457598
M3 - Conference contribution
AN - SCOPUS:85113292729
T3 - 2020 IEEE Information Theory Workshop, ITW 2020
BT - 2020 IEEE Information Theory Workshop, ITW 2020
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2020 IEEE Information Theory Workshop, ITW 2020
Y2 - 11 April 2021 through 15 April 2021
ER -