A naukri.com initiative
ML News
Arxiv
1w
67
Image Credit: Arxiv
Benign Overfitting in Out-of-Distribution Generalization of Linear Models
- Benign overfitting refers to the phenomenon where an over-parameterized model fits the training data perfectly, including noise in the data, but still generalizes well to the unseen test data.
- This study focuses on understanding benign overfitting in the Out-of-Distribution (OOD) regime for over-parameterized linear models under covariate shift.
- The authors provide non-asymptotic guarantees that benign overfitting occurs in standard ridge regression, even in the OOD regime under certain structural conditions of the target covariance.
- Theoretical results show that Principal Component Regression (PCR) achieves a faster rate of O(1/n) for the excess risk compared to standard ridge regression's slower rate of O(1/√n) for a more general family of target covariance matrices.
Read Full Article
4 Likes
Arxiv
1w
54
Image Credit: Arxiv
Graph-Structured Topic Modeling for Documents with Spatial or Covariate Dependencies
- Researchers have addressed the challenge of incorporating document-level metadata into topic modeling to improve topic mixture estimation.
- They propose a graph-structured topic modeling approach that incorporates document-level covariates or known similarities between documents.
- The approach is based on a fast graph-regularized iterative singular value decomposition (SVD) that encourages similar documents to share similar topic mixture proportions.
- Experiments on synthetic datasets and real-world corpora validate the model, showing improved performance and faster inference compared to existing Bayesian methods.
Read Full Article
3 Likes
Arxiv
1w
213
Image Credit: Arxiv
Treatment Effects Estimation on Networked Observational Data using Disentangled Variational Graph Autoencoder
- Estimating individual treatment effect (ITE) from observational data has gained attention across various domains.
- A key challenge is identifying latent confounders affecting both treatment and outcome.
- Existing approaches fail in practice as they assume observed variables and network information are only proxy variables for latent confounders.
- A novel disentangled variational graph autoencoder is proposed for ITE estimation on networked observational data, achieving state-of-the-art performance.
Read Full Article
12 Likes
Arxiv
1w
297
Image Credit: Arxiv
A hybrid framework for effective and efficient machine unlearning
- A hybrid framework for effective and efficient machine unlearning has been proposed.
- The framework combines exact machine unlearning and approximate machine unlearning techniques.
- It aims to achieve an overall success by implementing unlearning with acceptable computation cost and improving accuracy.
- Experiments on real datasets show that the proposed framework improves unlearning efficiency by 1.5x to 8x while maintaining comparable accuracy.
Read Full Article
17 Likes
Arxiv
1w
163
Image Credit: Arxiv
Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignment
- Cal-DPO is a new algorithm proposed for aligning large language models (LLMs) with human preference data.
- It addresses the limitation of the contrastive preference optimization by calibrating the implicit reward to ensure comparability with ground-truth rewards.
- Cal-DPO demonstrates theoretical advantages and significantly improves off-the-shelf methods in aligning LLMs with given preferences.
Read Full Article
9 Likes
Arxiv
1w
50
Image Credit: Arxiv
CAE-T: A Channelwise AutoEncoder with Transformer for EEG Abnormality Detection
- CAE-T is a novel framework for EEG abnormality detection.
- It combines a channelwise CNN-based autoencoder with a single-head transformer classifier.
- CAE-T achieves high accuracy, sensitivity, and specificity on the TUH Abnormal EEG Corpus.
- The framework is more efficient than other transformer-based alternatives and retains interpretability.
Read Full Article
3 Likes
Arxiv
1w
33
Image Credit: Arxiv
Knowledge Distillation in RNN-Attention Models for Early Prediction of Student Performance
- Educational data mining focuses on analyzing data from learning contexts and predicting at-risk students.
- An RNN-Attention-KD framework is introduced to predict at-risk students early throughout a course.
- The framework utilizes Recurrent Neural Networks (RNNs) and attention mechanism for improved predictive accuracy.
- Empirical evaluation shows that RNN-Attention-KD outperforms traditional neural network models in terms of recall and F1-measure.
Read Full Article
2 Likes
Arxiv
1w
20
Image Credit: Arxiv
Leveraging Time Series Categorization and Temporal Fusion Transformers to Improve Cryptocurrency Price Forecasting
- The accuracy of cryptocurrency price prediction is vital for organizing and managing portfolios and making transactions.
- To improve accuracy, this study categorizes financial time series into similar subseries.
- Deep learning models with attention mechanisms are created for each subseries category to predict future steps.
- To overcome limited training data, the study suggests combining time series data from multiple cryptocurrencies.
Read Full Article
1 Like
Arxiv
1w
318
Image Credit: Arxiv
ST-ReP: Learning Predictive Representations Efficiently for Spatial-Temporal Forecasting
- Spatial-temporal forecasting is crucial in various domains.
- Challenges in self-supervised learning for spatial-temporal forecasting include selecting reliable negative pairs, overlooking spatial correlations, and limitations of efficiency and scalability.
- ST-ReP is a lightweight representation-learning model that integrates current value reconstruction and future value prediction.
- ST-ReP surpasses pre-training-based baselines and exhibits superior scalability.
Read Full Article
19 Likes
Arxiv
1w
310
Image Credit: Arxiv
Downscaling Precipitation with Bias-informed Conditional Diffusion Model
- The proposed model achieves highly accurate results in an 8 times downscaling setting, outperforming previous deterministic methods.
Read Full Article
18 Likes
Arxiv
1w
67
Image Credit: Arxiv
Single-Loop Federated Actor-Critic across Heterogeneous Environments
- Federated reinforcement learning (FRL) allows multiple agents to collaborate and learn a shared policy in different environments.
- The actor-critic (AC) algorithm is known for its low variance and high sample efficiency in RL.
- However, theoretical understanding of AC in a federated manner with different environments is limited.
- The Single-loop Federated Actor Critic (SFAC) algorithm is proposed, showing convergence to a near-stationary point and linear speed-up in sample complexity.
Read Full Article
4 Likes
Arxiv
1w
339
Image Credit: Arxiv
Global Spatio-Temporal Fusion-based Traffic Prediction Algorithm with Anomaly Aware
- Traffic prediction is crucial for urban planning and traffic management, but existing methods overlook long-term relationships and anomaly influences.
- A global spatio-temporal fusion-based algorithm with anomaly awareness is proposed to address these issues.
- The algorithm incorporates an anomaly detection network to evaluate the impact of unexpected events on traffic prediction.
- Experiments using real-scenario datasets show that the proposed approach achieves state-of-the-art performance.
Read Full Article
20 Likes
Arxiv
1w
406
Image Credit: Arxiv
MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Design
- MixLLM is a new optimization approach for quantization of LLMs.
- MixLLM explores mixed-precision quantization between output features based on their salience in the global view.
- By assigning larger bit-width to output features that need it most, MixLLM achieves good accuracy with low memory consumption.
- MixLLM demonstrates superior accuracy and state-of-the-art system efficiency compared to existing quantization solutions.
Read Full Article
24 Likes
Arxiv
1w
54
Image Credit: Arxiv
Towards Scalable and Deep Graph Neural Networks via Noise Masking
- Graph Neural Networks (GNNs) have achieved remarkable success in graph mining tasks.
- Scaling GNNs to large graphs is challenging due to high computational and storage costs.
- Proposed random walk with noise masking (RMask) module to enable deeper GNN exploration while preserving scalability.
- Experimental results show improved performance and trade-off between accuracy and efficiency.
Read Full Article
3 Likes
Arxiv
1w
184
Image Credit: Arxiv
Robust PCA Based on Adaptive Weighted Least Squares and Low-Rank Matrix Factorization
- Researchers propose a novel Robust Principal Component Analysis (RPCA) model for decomposing data into low-rank and sparse components.
- The model integrates adaptive weighted least squares (AWLS) and low-rank matrix factorization (LRMF).
- It employs a self-attention-inspired mechanism to dynamically adjust and emphasize significant components.
- The proposed method outperforms existing non-convex regularization approaches in terms of performance, stability, accuracy, and robustness.
Read Full Article
11 Likes
For uninterrupted reading, download the app