Fisher-Preconditioned Influence for Efficient Machine Unlearning

Machine unlearning aims to efficiently remove the influence of designated training data from a deployed model while preserving utility on retained data. However, practical unlearning under tight compute budgets remains challenging: few-step updates can be unstable, overly sensitive to step size, and can cause undesired drift on retained samples. This paper proposes Influence Fisher (FL), a lightweight unlearning operator that combines (i) a forget-retain specialization mask derived from dataset-specific importance statistics, and (ii) a curvature-aware, few-step parameter update that targets the forget set while bounding drift on the retain set. We further introduce stabilizers - update clipping and optional step splitting - to improve robustness under limited update budgets. Extensive evaluations on image classification unlearning settings show that FL achieves a competitive trade-off between forgetting fidelity, retained accuracy, and compute cost, and remains stable across a broad range of dampening factors. We provide systematic ablations on the mask, curvature preconditioning, and hyperparameters, and compare against retraining, NegGrad, and representative approximate unlearning baselines under a unified evaluation protocol with multiple random seeds.