A naukri.com initiative
ML News
Arxiv
1w
418
Image Credit: Arxiv
Adaptive Locally Linear Embedding
- Manifold learning techniques, such as Locally linear embedding (LLE), aim to preserve local neighborhood structures of high-dimensional data during dimensionality reduction.
- Adaptive locally linear embedding (ALLE) is introduced as a novel approach to address the limitations of traditional LLE by incorporating a dynamic, data-driven metric for enhanced topological preservation.
- ALLE redefines the concept of proximity by focusing on topological neighborhood inclusion rather than fixed distances, resulting in superior neighborhood preservation and accurate embeddings.
- Experimental results demonstrate that ALLE improves the alignment between neighborhoods in input and feature spaces, providing a robust solution for capturing intricate relationships in high-dimensional datasets.
Read Full Article
25 Likes
Arxiv
1w
105
Image Credit: Arxiv
An Analysis of Temporal Dropout in Earth Observation Time Series for Regression Tasks
- Deep learning models in regression tasks face challenges due to missing instances in time series data.
- Monte Carlo Temporal Dropout (MC-TD) is introduced to address input-level uncertainty in Earth Observation time series data.
- Monte Carlo Concrete Temporal Dropout (MC-ConcTD) learns the optimal dropout distribution to improve predictive performance.
- Experiments demonstrate that MC-ConcTD enhances predictive performance and uncertainty calibration for EO applications.
Read Full Article
6 Likes
Arxiv
1w
245
Image Credit: Arxiv
RO-FIGS: Efficient and Expressive Tree-Based Ensembles for Tabular Data
- Tree-based ensembles are efficient and expressive models for tabular data.
- Traditional tree-based ensembles have limitations in expressing complex relationships.
- RO-FIGS (Random Oblique Fast Interpretable Greedy-Tree Sums) improves the efficiency and expressiveness of tree-based ensembles.
- RO-FIGS outperforms other tree- and neural network-based methods, while providing enhanced interpretability.
Read Full Article
14 Likes
Arxiv
1w
93
Image Credit: Arxiv
Adaptive Computation Pruning for the Forgetting Transformer
- The recently proposed Forgetting Transformer (FoX) incorporates a forget gate into softmax attention and has shown consistently better or on-par performance compared to the standard RoPE-based Transformer.
- Adaptive Computation Pruning (ACP) is introduced for FoX, a method that dynamically prunes computations involving input-output dependencies strongly decayed by the forget gate.
- ACP reduces the number of FLOPs in softmax attention by around 70% across different model sizes and context lengths, resulting in a 10-35% improvement in training throughput.
- The computational savings are greater with longer context lengths, and the performance of FoX is not affected by ACP.
Read Full Article
5 Likes
Arxiv
1w
186
Image Credit: Arxiv
Enhancing Metabolic Syndrome Prediction with Hybrid Data Balancing and Counterfactuals
- Researchers have developed a hybrid framework for enhancing metabolic syndrome (MetS) prediction.
- The framework leverages advanced data balancing techniques and counterfactual analysis.
- Multiple machine learning models were trained and compared under various data balancing techniques.
- The study provides actionable insights for clinicians and researchers in mitigating the public health burden of MetS.
Read Full Article
11 Likes
Arxiv
1w
266
Image Credit: Arxiv
FAME: Introducing Fuzzy Additive Models for Explainable AI
- Researchers introduce Fuzzy Additive Models (FAM) and FAM with Explainability (FAME) as a solution for Explainable Artificial Intelligence (XAI).
- FAM family has three layers: Projection Layer, Fuzzy Layer based on Single Input-Single Output Fuzzy Logic Systems (SFLS), and Aggregation Layer.
- FAME combines the interpretability of SFLS with the explainability of input-output relationships by sculpting the antecedent space within FAM.
- Comparative results demonstrate the potential of FAME in reducing model complexity while retaining interpretability for XAI.
Read Full Article
16 Likes
Arxiv
1w
291
Image Credit: Arxiv
Adapting GT2-FLS for Uncertainty Quantification: A Blueprint Calibration Strategy
- Uncertainty Quantification (UQ) is crucial for deploying reliable Deep Learning (DL) models in high-stakes applications.
- A blueprint calibration strategy for General Type-2 Fuzzy Logic Systems (GT2-FLSs) is proposed to improve efficiency and adaptability for generating Prediction Intervals (PIs) in new coverage levels.
- Two calibration methods, a lookup table-based approach and a derivative-free optimization algorithm, are developed to achieve accurate and reliable PIs while reducing computational overhead.
- Experimental results demonstrate the superior performance of the calibrated GT2-FLS in Uncertainty Quantification, emphasizing its potential for practical applications.
Read Full Article
17 Likes
Arxiv
1w
105
Image Credit: Arxiv
Identifying Key Challenges of Hardness-Based Resampling
- Performance gap across classes remains a persistent challenge in machine learning, often attributed to variations in class hardness.
- Hardness-based resampling is a promising approach to mitigate performance disparities.
- Resampling does not meaningfully affect class-wise performance disparities, contrary to theoretical expectations.
- Detailed analyses help identify key challenges unique to hardness-based imbalance and provide guidelines for future research.
Read Full Article
6 Likes
Arxiv
1w
372
Image Credit: Arxiv
To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoning
- Recent advancements in large language models have improved reasoning abilities through search and backtracking techniques.
- Sequential search, enabled by backtracking, allows linearized exploration via long chain-of-thought generation.
- Parallel sampling with best-of-n selection is an alternative approach to scaling test-time compute.
- Comparative analysis on reasoning tasks shows that while sequential search outperforms parallel sampling on Sudoku, it underperforms on CountDown, indicating the limitations of backtracking.
Read Full Article
22 Likes
Arxiv
1w
355
Image Credit: Arxiv
Identifying Unknown Stochastic Dynamics via Finite expression methods
- Modeling stochastic differential equations (SDEs) is crucial for understanding complex dynamical systems in various scientific fields.
- The Finite Expression Method (FEX) is introduced as a symbolic learning approach to derive interpretable mathematical representations of the deterministic component of SDEs.
- FEX integrates with generative modeling techniques to provide a comprehensive representation of SDEs, improving long-term predictions and generalization beyond training domain.
- FEX not only enhances prediction accuracy but also offers valuable scientific insights into the underlying dynamics of the systems, opening new possibilities for new discoveries.
Read Full Article
21 Likes
Arxiv
1w
359
Image Credit: Arxiv
A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibility
- Reasoning has become a significant focus for language models, but the progress often lacks methodological rigor and robust evaluation practices.
- Current mathematical reasoning benchmarks are sensitive to various implementation choices, leading to unclear comparisons and unreported sources of variance.
- A standardized evaluation framework with clear best practices and reporting standards is proposed to address these issues.
- Reassessment of recent methods reveals that reinforcement learning approaches show modest improvements and are prone to overfitting, while supervised fine-tuning methods demonstrate stronger generalization.
Read Full Article
21 Likes
Arxiv
1w
63
Image Credit: Arxiv
Neural Motion Simulator: Pushing the Limit of World Models in Reinforcement Learning
- Neural Motion Simulator (MoSim) is a world model that predicts the future physical state of an embodied system based on current observations and actions.
- MoSim achieves state-of-the-art performance in physical state prediction and provides competitive performance across a range of downstream tasks.
- Accurate world models with precise long-horizon predictions can facilitate efficient skill acquisition and enable zero-shot reinforcement learning.
- MoSim decouples physical environment modeling from RL algorithm development, leading to improved sample efficiency and generalization capabilities.
Read Full Article
3 Likes
Arxiv
1w
71
Image Credit: Arxiv
Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning
- Continual learning in large language models (LLMs) is prone to catastrophic forgetting.
- A novel continual full fine-tuning approach leveraging adaptive singular value decomposition (SVD) is proposed.
- The method identifies task-specific low-rank parameter subspaces and constrains updates to minimize interference without additional parameters or storing previous task gradients.
- Empirically, the approach achieves state-of-the-art results, maintaining model capabilities and reducing forgetting to near-negligible levels.
Read Full Article
4 Likes
Arxiv
1w
8
Image Credit: Arxiv
Information-Theoretic Reward Decomposition for Generalizable RLHF
- A new approach for generalizable reward model in Reinforcement Learning from Human Feedback (RLHF) is proposed.
- Existing reward models lack the ability to correctly evaluate unseen prompt-response pairs.
- The proposed approach decomposes the reward value into prompt-free reward and prompt-related reward.
- The new reward learning algorithm prioritizes data samples based on their prompt-free reward values.
Read Full Article
Like
Arxiv
1w
353
Image Credit: Arxiv
Joint Group Profiling and Recommendation via Deep Neural Network-based Multi-Task Learning
- Joint Group Profiling and Recommendation via Deep Neural Network-based Multi-Task Learning is a framework that unifies group profiling and recommendation tasks in a single model.
- The model improves recommendation accuracy by jointly learning these tasks, leading to a deeper understanding of group dynamics.
- Shared representations between the two tasks result in richer and more informative group embeddings.
- Experiments and evaluations on real-world datasets demonstrate that the multi-task learning approach consistently outperforms baseline models.
Read Full Article
21 Likes
For uninterrupted reading, download the app