The recent advancements in reinforcement learning (RL) have seen a significant shift towards leveraging large language models (LLMs) for reward design and human feedback integration. A notable trend is the development of frameworks that automate and optimize the reward function generation process, reducing the dependency on human intervention and repetitive RL training. These frameworks often incorporate dynamic feedback mechanisms, such as trajectory preference evaluation, to refine reward functions iteratively without the need for extensive RL training cycles. Additionally, there is a growing interest in real-time human-guided RL, where continuous human feedback is integrated into dense rewards to accelerate policy learning, sometimes supplemented by simulated feedback modules that mimic human patterns. Theoretical contributions are also emerging, focusing on optimal dataset selection for reward modeling in RL from human feedback (RLHF), providing offline approaches and worst-case guarantees. Furthermore, the use of LLMs in few-shot in-context preference learning is demonstrating efficiency in converting human preferences into reward functions, significantly reducing the query inefficiency associated with traditional RLHF methods. Lastly, there is a move towards enhancing RL with error-prone language models, proposing methods to navigate noisy feedback and improve convergence speed and policy returns, even in the presence of significant ranking errors.