The current research landscape in software engineering and binary code analysis is witnessing significant advancements, particularly in the integration of transformer-based models and parameter-efficient fine-tuning techniques. Recent studies are focusing on enhancing the understanding of binary code semantics by incorporating domain-specific features, such as control flow graphs and dynamic execution traces, into transformer architectures. This approach aims to improve model generalization and performance across various binary analysis tasks, albeit with challenges related to feature engineering and handling obfuscated code. Additionally, there is a growing emphasis on automating the detection of design smells in multi-language deep learning frameworks, which is crucial for maintaining code quality and performance. Parameter-efficient fine-tuning methods are also gaining traction, offering scalable and resource-efficient solutions for adapting large language models to specific software engineering tasks, such as code smell detection. These methods demonstrate competitive performance compared to full fine-tuning while significantly reducing memory requirements. Furthermore, research on model compression strategies for language models applied to code is providing valuable insights into balancing efficiency and effectiveness, enabling more practical adoption of these models in real-world scenarios.