Complex-Plane-Inspired Magnitude Rectifier: An Exploratory Transformer Activation Study

Abstract

This exploratory preprint studies a complex-plane-inspired magnitude rectifier (CIMR), a real-valued activation that retains negative-input magnitude through a beta-scaled orthogonal construction. The scalar form reduces to a smoothed asymmetric absolute-value rectifier; it is not a genuinely complex-valued neural network. Local experiments provide mixed evidence. Fixed and scalar-learned variants do not consistently outperform ReLU, while an input-conditioned AttentionBeta variant performs best on one modular synthetic task but uses about 24.6% more parameters, so the result is capacity-confounded. The report identifies the experiments as exploratory and specifies parameter-matched, held-out follow-up tests. Status: non-peer-reviewed exploratory technical report. AI assistance disclosure: ChatGPT and Sarvam AI assisted with brainstorming and language development. ChatGPT also assisted with reconstructing project history, auditing local code and results, literature triage, and drafting. AI systems are not authors; Chaman Prakash Kanth remains responsible for the manuscript and public metadata.