Advances in Computational Methods and Neural Network Optimization

Recent developments in computational mechanics and neural network optimization have shown significant progress in enhancing the robustness, efficiency, and accuracy of various methods. In computational mechanics, there is a strong focus on mixed finite element methods that are parameter-robust, addressing issues such as locking phenomena and nearly incompressible regimes. These methods are being applied to a variety of problems, demonstrating their versatility and effectiveness. Additionally, entropy-stable and non-oscillatory schemes for hyperbolic conservation laws are being pursued to control entropy production and suppress spurious oscillations, crucial for accurate simulations of fluid dynamics and structural mechanics.

In the realm of neural network optimization, there is a growing interest in training techniques that enhance the utilization of model layers. Improved training regimes and self-supervised learning methods are increasing the importance of early layers while under-utilizing deeper layers, contrasting with adversarial training methods. The impact of data diversity on the weight landscape of neural networks is also being investigated, revealing that diverse data can significantly improve out-of-distribution performance. Novel training schemes, such as SGD jittering for model-based architectures, are being proposed to balance accuracy and robustness in inverse problems.

Noteworthy Developments

• Computational Mechanics: New variational formulations for the Stokes problem using T-coercivity stabilize unstable finite element pairs and improve numerical approximations. Mimetic approaches for computing divergence-free metric terms in DGSEMs are essential for free-stream preservation and entropy stability on curvilinear grids.
• Neural Network Optimization: Training methods influence layer utilization, with improved regimes significantly impacting layer importance. Data diversity shapes the weight landscape, enhancing out-of-distribution performance. SGD jittering demonstrates enhanced robustness to adversarial attacks, and fine-tuning maintains neural network robustness through adaptive pruning ratios.

These advancements collectively push the boundaries of computational methods and neural network optimization, offering more nuanced and effective strategies for model selection and evaluation.