Privacy-Preserving Diffusion Against Identity Inference Attack via Semantic Destruction Recovery and Segmentation

Abstract

Facial image generation or generative editing using diffusion models presents risks of identity leakage, potentially enabling unauthorized re-identification in forensic and content moderation applications. Diffusion editing is vulnerable to privacy-preserving facial identity obfuscation through diffusion-based semantic transformation, which alters identity-sensitive facial attributes so that an identity inference adversary, such as a recognition or classification model, cannot reliably recover the original identity from the protected image. This work introduces DIFP (Diffusion-based Identity-preserving Facial Privacy Protection), a reversible privacy framework that unifies semantic overwrite and key-based image recovery. DIFP protects identity by intentionally overwriting facial semantics during diffusion using identity-altering prompts and adaptive guidance, while preserving background information through a FAPM that localizes transformations to identity-sensitive regions. To enable authorized recovery, DIFP injects a key derived via SHA-256 into the diffusion latent representation through deterministic, seed-controlled perturbations applied during the scheduler-driven denoising process. Decoding the keyed latent produces a privacy-protected image, whereas removing the same key-dependent perturbation enables key-conditioned image recovery to protect generative recovery alignment. This paper further presents a privacy-aware denoising method that illustrates the generative editing and filling of semantic destruction and authorized recoverability across diffusion steps under varying privacy strengths. Privacy effectiveness is assessed within an identity inference threat model, employing a ResNet-50 classifier as a proxy adversary. At the same time, SSIM (Structural Similarity Index Measure) and Facial IoU (Intersection over Union) are used to quantify the visual fidelity of the recovered image in generative identity editing, along with a privacy score. Experimental results demonstrate consistent identity obfuscation through semantic destruction, while preserving high perceptual quality and enabling reliable, authenticated content recovery for legitimate privacy linkage examination.