Physics-Guided Prompt Adaptation for Optically Robust Image Classification and Object Detection

Abstract

Optical systems in the real world often create image problems, such as Gaussian blur from defocus or atmospheric turbulence, and radial vignetting caused by lens shape. Standard mixed-data fine-tuning helps the task head handle these issues, but it does not actually fix them. We introduce the Iterative Correction of Optical Perturbations ICOP framework, which corrects encoder feature representations before they reach the task head using a physics-aware plug-in module. ICOP does this by modeling blur as an isotropic-Gaussian point-spread function (PSF) and uses gradient-based, self-supervised optimization (Adam) to discover feature-space corrections that steer degraded representations toward their clean-data distribution. It includes a BlurEstimator that builds a degradation descriptor using fixed Laplacian and Sobel operators, and a PromptGenerator that turns this descriptor into modulation parameters for the frozen encoder output. The framework comes in two versions based on the task: an additive correction (ICOP-Add) for image classification, and a Feature-wise Linear Modulation correction (ICOP-FiLM) for object detection. We observe a convergence between clean-task performance and blur-induced degradation across datasets, consistent with greater reliance on high-frequency features in stronger backbones; we treat this as an empirical, cross-dataset observation rather than a demonstrated causal claim (task difficulty, category structure, texture, and object scale also differ across datasets). Independently of this, ICOP-FiLM’s corrective benefit does not scale with degradation severity, revealing a more nuanced relationship between backbone quality and robustness. For classification, ICOP-Add improves distorted-condition accuracy over strong mixed fine-tuning by +2.4, +9.1, and +10.7 percentage points on MNIST, FashionMNIST, and CIFAR-10, respectively (McNemar’s test, p<0.001 on all three, 5000 paired predictions per dataset). On three object detection datasets, ICOP-FiLM improves distorted-condition mAP over a mixed-fine-tuning null hypothesis by +0.026, −0.005, and +0.003 mAP, respectively (all values mean over 3 seeds). Against a matched-blur-ratio control that isolates the correction module’s own contribution, ICOP-FiLM wins by a consistent margin on two of the three datasets (+0.037 and +0.024 mAP, winning in every one of 3/3 seeds on each) and loses on the third (−0.039 mAP, losing in 3/3 seeds); it outperforms parameter-efficient (VPT, Adapter) baselines trained on identical data on the same two datasets. This dataset-dependent pattern is discussed in detail in the main text. ICOP-FiLM adds only 82,672 parameters to a 42-million-parameter Real-Time DEtection TRansformer (RT-DETR) detector. All reported results are obtained under synthetic Gaussian blur and radial vignetting applied to clean images from the six benchmark datasets studied.