Abstract

Machine unlearning (MU) aims to expunge a designated forgotten set from a trained model without costly retraining, yet the existing techniques overlook two critical blind spots: "over-unlearning" that deteriorates retained data near the forgotten set, and post-hoc "relearning" attacks that aim to resurrect the forgotten knowledge. Focusing on class-level unlearning, we first derive an over-unlearning metric, OU@epsilon, which quantifies collateral damage in regions proximal to the forget set, where over-unlearning mainly occurs. Next, we expose an unforeseen relearning threat on MU, ie, the Prototypical Relearning Attack, which exploits the per-class prototype of the forget class with just a few samples, and easily restores the pre-unlearning performance. To counter both blind spots in class-level unlearning, we introduce Spotter, a plug-and-play objective that combines (i) a masked knowledge-distillation penalty on the nearby region of forget classes to suppress OU@epsilon, and (ii) an intra-class dispersion loss that scatters forget-class embeddings, neutralizing Prototypical Relearning Attacks. Spotter achieves state-of-the-art results across CIFAR, TinyImageNet, and CASIA-WebFace datasets, offering a practical remedy to unlearning's blind spots.

Machine unlearning (MU) aims to remove sensitive information from trained AI models without costly retraining. Current approaches, however, can forget more than they should—and what has been forgotten can sometimes be learned again. 

To address both challenges, a joint research team led by Professor Sung Whan Yoon of the Graduate School of Artificial Intelligence and Professor Saerom Park of the Department of Industrial Engineering developed  Spotter , a new MU method that removes targeted information without affecting similar information that should be retained. The method also makes deleted knowledge far from difficult to recover through a relearning attack.

Current MU methods often forget more than intended. Removing one category can unintentionally affect similar categories that were meant to be retained. For example, teaching an AI model to forget cats may also weaken its ability to recognize tigers or leopards.

The team identified a second weakness. Even after information has been removed, a model can recover much of that knowledge from only a handful of examples. In other words, information that was supposed to be forgotten may not stay forgotten.

Spotter addresses both problems at once. It preserves the model's ability to distinguish similar information that should be retained while making deleted knowledge far more difficult to recover. Because it is designed as a plug-and-play method, Spotter can be integrated into existing MU frameworks without extensive modification.

"Machine unlearning has traditionally been evaluated by asking two questions: Has the target information been removed, and does the model still perform well overall?" said SeungBum Ha, the study's first author. "Our work shows those measures do not tell the whole story. Models can still lose information they were supposed to retain—or recover information that was meant to be erased."

In experiments on the CIFAR-10 dataset, Spotter completely removed recognition of the target class, reducing its accuracy to 0%. Even after a relearning attack using just five images, recognition accuracy remained at 0.24%. Under the same attack, existing methods recovered between 71.10% and 99.98% of their original recognition performance. Spotter also maintained 99.96% accuracy on the categories that were meant to be retained.

Spotter performed similarly on the larger TinyImageNet dataset and the real-world face recognition dataset CASIA-WebFace, demonstrating that the method remains effective across different image recognition tasks. These findings suggest Spotter could strengthen AI applications where reliable data removal is essential, including face recognition and identity-level content moderation.

The research was accepted to the International Conference on Machine Learning (ICML) 2026, held in Seoul from July 6 to 11, 2026. The study was supported by the Mid-Career Research Program of the National Research Foundation of Korea (NRF), as well as the Hyper-scale Industrial AI Research Support (R&D) Program, the Artificial Intelligence Graduate School Program at UNIST, the AI Star Fellowship Program, the Innovative Human Resource Development for Local Intellectualization Program, and the Information Technology Research Center (ITRC), supported by the Ministry of Science and ICT (MSIT) and the Institute of Information & Communications Technology Planning & Evaluation (IITP).

Journal Reference

SeungBum Ha, Saerom Park, and Sung Whan Yoon, "Unlearning's Blind Spots: Over-Unlearning and Prototypical Relearning Attack," ICML'26, (2026).