2025 · survey paper · machine unlearning

A Survey of LoRA-based Machine Unlearning for LLMs: Methods, Taxonomy, and Evaluation

An adapter-centric survey of how LoRA modules forget, preserve utility, and support auditable deletion workflows in large language models.

Delong Li, Guangsheng Yu, Xu Wang, Yanna Jiang, Wencheng Yang, Bin Liang, Wei Ni

University of Technology Sydney · University of Southern Queensland · Edith Cowan University

Paper PDF Preprint BibTeX

Method families6 adapter-space categories

Operator lenss, G, P, C, M

EvaluationTOFU + continual deletion + PrivLeak

Roadmap from LoRA unlearning motivation to taxonomy, evaluation, and governance — The survey connects the motivation for LoRA-based unlearning with a unified adapter-space view, method taxonomy, evaluation protocol, and deployment considerations.

Summary

What this paper contributes

An adapter-centric problem formulation

The survey formalizes LoRA unlearning under a frozen backbone and explicit retain and forget splits, making the adapter checkpoint the locus of deletion.

A unified operator view

Methods are decomposed into score construction, preconditioning, projection, bounding, and masking for coherent cross-family comparison.

A six-family taxonomy

Existing approaches are organized into gradient, influence, Fisher, structural, bounded-dynamics, and pruning or sparse variants.

Evaluation and deployment guidance

The paper consolidates TOFU-style reporting, utility and privacy trade-offs, continual deletion behavior, and governance concerns.

Abstract

Why LoRA-based unlearning deserves its own survey

Machine unlearning is becoming a practical requirement for deployed LLMs, but many existing techniques assume full-model access and retraining budgets that are difficult to sustain in real systems.

Modern adaptation pipelines increasingly rely on parameter-efficient fine-tuning, especially LoRA modules attached to a frozen backbone and distributed as lightweight checkpoints.

This survey focuses on adapter-centric forgetting: how deletion requests can be implemented primarily in LoRA space, how method families trade forgetting against retention, and what those choices imply for privacy leakage, continual updates, and operational governance.