Impacts of Feedback Current Value and Learning Rate on Equilibrium Propagation Performance

The use of the Equilibrium Propagation algorithm for analog neural networks was introduced in 2020 as an alternative to Backpropagation [1]. In the existing works of analog implementation of Equilibrium Propagation, the impacts of the learning rate, alpha (α), and the scaling factor of the feedback current, beta (β), have not been discussed. This work analyzes the impacts of the scaling factor of feedback current and the learning rate together with the ratio of those two parameters on the algorithm convergence. An Equilibrium Propagation circuit has been implemented on Cadence Virtuoso for a simple task to test the impacts of alpha (α) and beta (β) parameters. Numerical simulations are carried out in a Python-Spectre interface that we implemented. Detecting the optimum ranges for alpha (α) and beta (β) values is particularly influential on the algorithm performance. Our simulation results show that the algorithm only converges for distinctive alpha (α) and beta (β) values. For the tasks we experimented with, the scaling factor should be smaller than 0.1 (β < 0.1), and the learning rate should be larger than 5 × 10-6 and smaller than 0.1 (5 × 10-6 ≤ α < 0.1) to make the algorithm converge.