Advancements in Federated Learning: Privacy, Efficiency, and Heterogeneity

The recent developments in the field of federated learning (FL) and its applications demonstrate a strong focus on enhancing privacy, reducing communication overhead, and addressing data heterogeneity. Innovations are particularly notable in the integration of FL with other learning paradigms such as split learning and continual learning, aiming to leverage their strengths while mitigating inherent challenges. Privacy-preserving techniques are evolving, with new algorithms designed to protect sensitive information during the training process, especially in vertical federated learning scenarios. Communication efficiency is another critical area of advancement, with novel methods employing learned compression and knowledge condensation to minimize data transfer without compromising model accuracy. Addressing data heterogeneity, especially in non-IID settings, remains a priority, with approaches that enhance local and global knowledge distillation to improve model robustness and performance. The field is also witnessing the exploration of FL's vulnerabilities, such as gradient inversion attacks, and the development of countermeasures tailored to specific data types like graph-structured data. These trends underscore the field's move towards more secure, efficient, and adaptable federated learning frameworks capable of handling the complexities of real-world applications.

Noteworthy Papers

  • FedGAT: Introduces a federated approximation algorithm for Graph Attention Networks, significantly reducing communication overhead while maintaining accuracy.
  • P$^3$EFT: Proposes a multi-party split learning algorithm for parameter-efficient fine-tuning, ensuring label privacy with competitive accuracy.
  • SplitFedZip: Employs learned compression to reduce data transfer in Split-Federated learning, maintaining model accuracy.
  • FedDP: Enhances privacy-preserving cross-project defect prediction with a novel knowledge enhancement approach.
  • FedTA: Addresses spatial-temporal data heterogeneity in federated continual learning, improving model adaptability.
  • GeFL: Incorporates generative models to facilitate federated learning with heterogeneous models, enhancing performance and privacy.
  • FedGIG: Introduces a gradient inversion attack method for graph-structured data, highlighting FL vulnerabilities.
  • FedVCK: Tackles non-IID data challenges in federated learning with valuable condensed knowledge, improving robustness and communication efficiency.

Sources

FedGAT: A Privacy-Preserving Federated Approximation Algorithm for Graph Attention Networks

Label Privacy in Split Learning for Large Models with Parameter-Efficient Training

SplitFedZip: Learned Compression for Data Transfer Reduction in Split-Federated Learning

Better Knowledge Enhancement for Privacy-Preserving Cross-Project Defect Prediction

Addressing Spatial-Temporal Data Heterogeneity in Federated Continual Learning via Tail Anchor

GeFL: Model-Agnostic Federated Learning with Generative Models

FedGIG: Graph Inversion from Gradient in Federated Learning

FedVCK: Non-IID Robust and Communication-Efficient Federated Learning via Valuable Condensed Knowledge for Medical Image Analysis

Built with on top of