A naukri.com initiative
ML News
Arxiv
4d
186
Image Credit: Arxiv
Thermodynamically Consistent Latent Dynamics Identification for Parametric Systems
- A new thermodynamics-informed latent space dynamics identification framework, tLaSDI, has been proposed for modeling parametric nonlinear dynamical systems.
- The framework combines autoencoders for dimensionality reduction with parametric GENERIC formalism-informed neural networks (pGFINNs) to efficiently learn parametric latent dynamics while upholding thermodynamic principles like free energy conservation and entropy generation.
- A physics-informed active learning strategy is included to improve model performance through adaptive sampling of training data based on a residual-based error indicator, resulting in better outcomes than uniform sampling at the same computational cost.
- Numerical experiments on different equations demonstrate that the proposed method achieves significant speed-up, reduced relative errors, and lower training and inference costs, while also providing insights into the thermodynamic behavior of the system through learned latent space dynamics.
Read Full Article
11 Likes
Arxiv
4d
8
Image Credit: Arxiv
Explaining, Fast and Slow: Abstraction and Refinement of Provable Explanations
- Recent advancements have shown the possibility of obtaining explanations with formal guarantees for neural networks using verification techniques.
- A novel abstraction-refinement technique has been proposed to efficiently compute provably sufficient explanations of neural network predictions.
- The method involves abstracting the original neural network into a reduced network to speed up the verification process.
- Experiments show that this approach improves the efficiency of obtaining provably sufficient explanations for neural network predictions.
Read Full Article
Like
Arxiv
4d
292
Image Credit: Arxiv
DiffGradCAM: A Universal Class Activation Map Resistant to Adversarial Training
- Class Activation Mapping and its variants are commonly used for explaining Convolutional Neural Network predictions.
- Standard CAMs are vulnerable to adversarial manipulation like passive fooling, leading to misleading CAMs without affecting decision performance.
- To address this vulnerability, Salience-Hoax Activation Maps (SHAMs) are introduced as a benchmark for CAM robustness under adversarial conditions.
- DiffGradCAM, a novel approach to class activation mapping, is proposed to be resistant to passive fooling and matches standard CAM methods in non-adversarial cases.
Read Full Article
17 Likes
Arxiv
4d
284
Image Credit: Arxiv
NeurIPS 2024 ML4CFD Competition: Results and Retrospective Analysis
- The integration of machine learning (ML) into physical sciences is transforming computational paradigms, aiming to enhance simulations like computational fluid dynamics (CFD).
- ML4CFD competition was organized to improve accuracy, generalization, and physical consistency of ML models for aerodynamic simulations over two-dimensional airfoils.
- Over 240 teams participated in the competition, utilizing a dataset from OpenFOAM and evaluated based on predictive accuracy, physical fidelity, computational efficiency, and generalization.
- Retrospective analysis of the competition revealed top-performing approaches surpassing traditional solvers, showcasing the potential of ML-based surrogates in scientific simulations.
Read Full Article
17 Likes
Arxiv
4d
170
Image Credit: Arxiv
Leveraging chaos in the training of artificial neural networks
- Researchers have explored the dynamics of artificial neural network trajectory during training with unconventional large learning rates.
- For certain learning rate values, the optimization shifts from exploitation-like to exploration-exploitation balance, leading to sensitive dependence on initial conditions.
- Training time to achieve acceptable accuracy in the test set reduces to a minimum in this regime, indicating accelerated training of neural networks near the onset of chaos.
- The study, initially demonstrated on the MNIST classification task, shows the constructive role of transient chaotic dynamics in training artificial neural networks across various learning tasks and architectures.
Read Full Article
10 Likes
Arxiv
4d
45
Image Credit: Arxiv
Robust Evolutionary Multi-Objective Network Architecture Search for Reinforcement Learning (EMNAS-RL)
- Evolutionary Multi-Objective Network Architecture Search (EMNAS) introduced for optimizing neural network architectures in large-scale Reinforcement Learning (RL) for Autonomous Driving.
- EMNAS uses genetic algorithms to automate network design to enhance rewards and reduce model size without performance compromise.
- Parallelization techniques and teacher-student methodologies are employed to accelerate the search and ensure scalable optimization.
- Experimental results show EMNAS outperforms manually designed models, achieving higher rewards with fewer parameters, contributing to better-performing networks for real-world autonomous driving scenarios.
Read Full Article
2 Likes
Arxiv
4d
207
Image Credit: Arxiv
DeepForm: Reasoning Large Language Model for Communication System Formulation
- DeepForm is introduced as the first reasoning Large Language Model (LLM) specifically designed for automated communication system formulation.
- A large-scale, open-source dataset named Communication System Formulation Reasoning Corpus (CSFRC) is presented for training the DeepForm model in this domain.
- DeepForm utilizes a two-stage training approach: Supervised Fine-Tuning with Chain-of-Thought (CoT) data for domain knowledge distillation, followed by a rule-based Reinforcement Learning (RL) algorithm, C-ReMax, for advanced modeling capabilities and reasoning patterns.
- Extensive experiments show that DeepForm achieves state-of-the-art performance, surpassing larger proprietary LLMs in various scenarios, and related resources will be released to encourage future research in this field.
Read Full Article
12 Likes
Arxiv
4d
357
Image Credit: Arxiv
The Geometries of Truth Are Orthogonal Across Tasks
- Recent works have proposed examining the activations produced by Large Language Models (LLMs) at inference time to assess the correctness of their answers.
- These works suggest that a 'geometry of truth' can be learned, where activations for correct answers differ from those producing mistakes.
- However, a limitation highlighted is that these 'geometries of truth' are task-dependent and do not transfer across different tasks.
- Linear classifiers trained across distinct tasks show little similarity, even with more sophisticated approaches, as activation vectors used to classify answers form separate clusters when examined across tasks.
Read Full Article
21 Likes
Arxiv
4d
350
Image Credit: Arxiv
SLEEPYLAND: trust begins with fair evaluation of automatic sleep staging models
- SLEEPYLAND is an open-source sleep staging evaluation framework designed to address challenges in model evaluation, generalization, bias, and human annotations.
- It includes over 22,000 hours of in-domain (ID) sleep recordings and 84,000 hours of out-of-domain (OOD) sleep recordings.
- SOMNUS, an ensemble combining models across architectures and channel setups, achieves robust performance across twenty-four different datasets, outperforming individual models in 94.9% of cases.
- In evaluations on multi-annotated datasets, SOMNUS exceeds the best human scorer, better reproducing the scorer consensus than any individual expert.
Read Full Article
21 Likes
Arxiv
4d
170
Image Credit: Arxiv
Flow Matching Meets PDEs: A Unified Framework for Physics-Constrained Generation
- A new generative framework called Physics-Based Flow Matching (PBFM) has been introduced to embed physical constraints into flow matching objectives.
- PBFM jointly minimizes flow matching loss and physics-based residual loss without requiring hyperparameter tuning of their relative weights.
- Temporal unrolling during training time is utilized to improve the accuracy of noise-free sample prediction.
- Extensive benchmarks on three PDE problems show that PBFM yields up to an 8 times more accurate physical residuals compared to existing algorithms, making it efficient for surrogate modeling and accelerated simulation in physics and engineering applications.
Read Full Article
10 Likes
Arxiv
4d
158
Image Credit: Arxiv
Sample Efficient Demonstration Selection for In-Context Learning
- The in-context learning paradigm with Large Language Models (LLMs) has been crucial for advancing various natural language processing tasks.
- Exemplar selection is important for constructing effective prompts within context-length budget constraints, and it is formulated as a top-m best arms identification problem.
- A new sample-efficient selective exploration strategy called Challenger Arm Sampling for Exemplar selection (CASE) is proposed to reduce sample complexity in exemplar selection tasks.
- CASE achieves up to a 7x speedup in runtime, requires 7x fewer LLM calls, and provides an 87% reduction compared to existing exemplar selection methods, without compromising performance.
Read Full Article
9 Likes
Arxiv
4d
28
Image Credit: Arxiv
HSG-12M: A Large-Scale Spatial Multigraph Dataset
- Introduction of HSG-12M, a large-scale dataset of spatial multigraphs embedded in a metric space.
- HSG-12M contains 11.6 million static and 5.1 million dynamic Hamiltonian spectral graphs across 1401 characteristic-polynomial classes.
- The dataset is derived from 177 TB of spectral potential data, encoding the full geometry of a 1-D crystal's energy spectrum on the complex plane.
- Release of Poly2Graph pipeline allows mapping of arbitrary 1-D crystal Hamiltonians to spectral graphs, presenting new challenges and opportunities in graph learning.
Read Full Article
1 Like
Arxiv
4d
85
Image Credit: Arxiv
Time Series Representations for Classification Lie Hidden in Pretrained Vision Transformers
- Time series classification is a crucial task in healthcare and industry, hindered by limited time series foundation models (TSFMs) due to lack of datasets.
- A new framework called Time Vision Transformer (TiViT) is introduced, converting time series data into images to utilize pretrained Vision Transformers (ViTs) from image datasets.
- Theoretical analysis shows that patching ViTs for time series can enhance label-relevant tokens and decrease sample complexity.
- TiViT achieves top performance on time series benchmarks by leveraging hidden representations from large OpenCLIP models, emphasizing the effectiveness of intermediate layers for classification.
Read Full Article
5 Likes
Arxiv
4d
73
Image Credit: Arxiv
Semi-gradient DICE for Offline Constrained Reinforcement Learning
- Stationary Distribution Correction Estimation (DICE) helps address the mismatch between stationary distribution induced by a policy and the target distribution required for reliable off-policy evaluation and policy optimization.
- Recent approaches to enhance offline reinforcement learning performance inadvertently hinder DICE's ability for off-policy evaluation, especially in constrained reinforcement learning scenarios.
- The limitation in recent approaches is attributed to their dependence on semi-gradient optimization, leading to failures in cost estimation in the DICE framework.
- A novel method called semi-gradient DICE is proposed to overcome limitations and improve off-policy evaluation and performance in offline constrained reinforcement learning, achieving state-of-the-art results on the DSRL benchmark.
Read Full Article
4 Likes
Arxiv
4d
69
Image Credit: Arxiv
Fusing Cross-modal and Uni-modal Representations: A Kronecker Product Approach
- Cross-modal embeddings like CLIP, BLIP have shown promise in aligning representations across modalities but may underperform on modality-specific tasks.
- Single-modality embeddings excel within their domains but lack cross-modal alignment capabilities.
- RP-KrossFuse is proposed as a method to unify cross-modality and single-modality embeddings by integrating them using a random projection-based Kronecker product.
- RP-KrossFuse aims to achieve competitive modality-specific performance while preserving cross-modal alignment, demonstrated through numerical experiments combining CLIP embeddings with uni-modal image and text embeddings.
Read Full Article
4 Likes
For uninterrupted reading, download the app