昊辰 姜

Projected Primal–Dual Gradient Preference Optimization Under Category-Wise Split-Conformal Safety Constraints for Large Language Model Post-Training

Preference optimisation improves helpfulness, but a single aggregate safety penalty lets frequent harm categories dominate the objective while rare ones bear the residual risk. This paper models post-training as a vector-constrained programme: the pairwise preference loss is mini...