The recent advancements in preference optimization for large language models (LLMs) have significantly enhanced the alignment of these models with human preferences and specific goals. A notable trend is the shift towards more sophisticated data generation techniques, such as iterative pairwise ranking mechanisms, to ensure high-quality preference data. This approach addresses the limitations of existing scoring-based reward models, which often produce unsatisfactory data and perform poorly on out-of-distribution tasks. Additionally, there is a growing emphasis on novel training regularization techniques, such as budget-controlled regularization, which improve convergence and alignment performance. These innovations are leading to models that surpass state-of-the-art benchmarks, demonstrating the effectiveness of these methods. Furthermore, the introduction of dynamic rewarding mechanisms and prompt optimization enables tuning-free self-alignment, reducing the reliance on costly training and human preference annotations. This approach leverages search-based optimization frameworks to iteratively self-improve, showcasing the potential of LLMs to achieve adaptive self-alignment through inference-time optimization. Notably, these advancements are not limited to text-based models; preference optimization is also being explored for contrastive learning models in vision tasks, enhancing robustness and fairness. Overall, the field is moving towards more efficient, robust, and self-adaptive alignment techniques, paving the way for broader applications and improved performance in various domains.