The current research in neural network compression and on-device recommendation systems is significantly advancing the field by focusing on innovative methods to reduce model size and memory consumption while maintaining or even improving performance. Key developments include the integration of tensor decompositions and sparsity techniques for parameter sharing in large transformer models, which allows for efficient compression without significant loss in accuracy. Additionally, there is a growing emphasis on dynamic sparse training methods that optimize embedding tables for recommender systems, addressing the challenges of memory-constrained environments. These methods often leverage cooperative game theory and efficient gradient computation to maintain sparsity during both forward and backward passes. Furthermore, the exploration of extreme pruning techniques is pushing the boundaries of model sparsity, enabling continuous learning and performance preservation at unprecedented sparsity levels. These advancements collectively contribute to making deep learning models more deployable on resource-constrained devices, thereby broadening their practical applications.