Bounds on the Approximation Power of Feedforward Neural Networks

Mohammad Mehrabi; Aslan Tchamkerten; Mansoor I. Yousefi

Bounds on the Approximation Power of Feedforward Neural Networks

Sharif University of Technology

Research output: Contribution to journal › Conference article › peer-review

Abstract

The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes of functions, such as strongly convex functions. Second, an upper bound is established on the difference of two neural networks with identical weights but different activation functions.

Original language	English
Pages (from-to)	3453-3461
Number of pages	9
Journal	Proceedings of Machine Learning Research
Volume	80
Publication status	Published - 1 Jan 2018
Event	35th International Conference on Machine Learning, ICML 2018 - Stockholm, Sweden Duration: 10 Jul 2018 → 15 Jul 2018

Cite this

@article{fa96cf5a4a664be6876be12bb9367590,

title = "Bounds on the Approximation Power of Feedforward Neural Networks",

abstract = "The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes of functions, such as strongly convex functions. Second, an upper bound is established on the difference of two neural networks with identical weights but different activation functions.",

author = "Mohammad Mehrabi and Aslan Tchamkerten and Yousefi, \{Mansoor I.\}",

note = "Publisher Copyright: {\textcopyright} 2018 by the author(s).; 35th International Conference on Machine Learning, ICML 2018 ; Conference date: 10-07-2018 Through 15-07-2018",

year = "2018",

month = jan,

day = "1",

language = "English",

volume = "80",

pages = "3453--3461",

journal = "Proceedings of Machine Learning Research",

issn = "2640-3498",

}

TY - JOUR

T1 - Bounds on the Approximation Power of Feedforward Neural Networks

AU - Mehrabi, Mohammad

AU - Tchamkerten, Aslan

AU - Yousefi, Mansoor I.

PY - 2018/1/1

Y1 - 2018/1/1

N2 - The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes of functions, such as strongly convex functions. Second, an upper bound is established on the difference of two neural networks with identical weights but different activation functions.

AB - The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes of functions, such as strongly convex functions. Second, an upper bound is established on the difference of two neural networks with identical weights but different activation functions.

UR - https://www.scopus.com/pages/publications/105020572284

M3 - Conference article

AN - SCOPUS:105020572284

SN - 2640-3498

VL - 80

SP - 3453

EP - 3461

JO - Proceedings of Machine Learning Research

JF - Proceedings of Machine Learning Research

T2 - 35th International Conference on Machine Learning, ICML 2018

Y2 - 10 July 2018 through 15 July 2018

ER -

Bounds on the Approximation Power of Feedforward Neural Networks

Abstract

Other files and links

Fingerprint

Cite this