Beyond Low-rank Decomposition: A Shortcut Approach for Efficient On-Device Learning

On-device learning has emerged as a promising direction for AI development, particularly because of its potential to reduce latency issues and mitigate privacy risks associated with deviceserver communication, while improving energy efficiency. Despite these advantages, significant memory and computational constraints still represent major challenges for its deployment. Drawing on previous studies on low-rank decomposition methods that address activation memory bottlenecks in backpropagation, we propose a novel shortcut approach as an alternative. Our analysis and experiments demonstrate that our method can reduce activation memory usage, even up to 120.09× compared to vanilla training, while also reducing overall training FLOPs up to 1.86× when evaluated on traditional benchmarks. The code is available at https://github.com/LeTrungNguyen/ICML2025-ASI.git.