The recent developments in the field of privacy-preserving machine learning and secure computation frameworks have shown significant progress, particularly in the areas of binary neural networks (BNNs), large language models (LLMs), and transformer-based models (TBMs). Innovations are primarily focused on enhancing the efficiency and security of model inference and fine-tuning processes, addressing the challenges of communication costs, accuracy, and privacy risks. Techniques such as knowledge distillation, separable convolutions, homomorphic encryption, and secret sharing are being optimized to create more efficient and secure computation protocols. Additionally, there is a growing emphasis on developing methods to ensure differential privacy in model training and fine-tuning, with novel approaches to data valuation and gradient encoding being introduced to mitigate privacy risks without compromising model utility.

Noteworthy papers include:

• A three-party secure computation framework for BNNs that maintains high utility through customized binarization and security measures.
• A comprehensive survey on privacy challenges in fine-tuning LLMs, proposing directions for advancing privacy-preserving methods.
• An approach to accelerate private TBM inference through fine-grained computation optimization, significantly reducing runtime and communication costs.
• A method for data value estimation on private gradients that addresses the paradox of estimation uncertainty scaling with the budget.
• A solution for end-to-end privacy guarantee in LLM fine-tuning by encoding low-rank gradients with a random prior.