When Neurons Fail

Neural networks have been traditionally considered robust in the sense that their precision degrades gracefully with the failure of neurons and can be compensated by additional learning phases. Nevertheless, critical applications for which neural networks are now appealing solutions, cannot afford any additional learning at run-time. In this paper, we view a multilayer neural network as a distributed system of which neurons can fail independently, and we evaluate its robustness in the absence of any (recovery) learning phase. We give tight bounds on the number of neurons that can fail without harming the result of a computation. To determine our bounds, we leverage the fact that neuralactivation functions are Lipschitz-continuous. Our bound isgiven in the form of quantity, we call the Forward ErrorPropagation, computing this quantity only requires looking atthe topology of the network, while experimentally assessingthe robustness of a network requires the costly experiment oflooking at all the possible inputs and testing all the possibleconfigurations of the network corresponding to different failuresituations, facing a discouraging combinatorial explosion. We distinguish the case of neurons that can fail and stop their activity (crashed neurons) from the case of neurons that can fail by transmitting arbitrary values (Byzantine neurons). In the crash case, our bound involves the number of neuronsper layer, the Lipschitz constant of the neural activationfunction, the number of failing neurons, the synaptic weightsand the depth of the layer where the failure occurred. In thecase of Byzantine failures, our bound involves, in addition, thesynaptic transmission capacity. Interestingly, as we show inthe paper, our bound can easily be extended to the case wheresynapses can fail. We present three applications of our results. The first is aquantification of the effect of memory cost reduction on theaccuracy of a neural network. The second is a quantification ofthe amount of information any neuron needs from its precedinglayer, enabling thereby a boosting scheme that prevents neuronsfrom waiting for unnecessary signals. Our third applicationis a quantification of the trade-off between neural networksrobustness and learning cost.