The recent advancements in parameter-efficient fine-tuning (PEFT) methods for large-scale models are significantly reshaping the landscape of multi-modal and vision-language models. Researchers are increasingly focusing on strategies that preserve the pre-trained representation space while efficiently adapting models to new tasks. This trend is evident in the development of techniques like prefix-tuning, which excels at maintaining the original model's representation space, and dual low-rank adaptation, which addresses the challenge of catastrophic forgetting in continual learning scenarios. Additionally, innovations in sparse tuning and visual Fourier prompt tuning are providing solutions to the memory and performance degradation issues associated with large-scale model adaptation. These methods not only enhance the adaptability of models but also ensure that the models retain their generalizability and efficiency. Notably, the integration of Fourier transforms into prompt tuning and the use of sparse orthogonal parameters in continual learning are particularly groundbreaking, offering new paradigms for model adaptation and knowledge retention. Overall, the field is moving towards more sophisticated, yet parameter-efficient, methods that balance performance, memory usage, and the preservation of pre-trained knowledge.