Machine Learning (ML) Latest News and Trending articles from all top sources only on Techminis

A naukri.com initiative

New

Home

ML News

Arxiv

280

Image Credit: Arxiv

Link Prediction with Physics-Inspired Graph Neural Networks

The article introduces GRAFF-LP, an extension of GRAFF for link prediction in heterophilic datasets.
GRAFF-LP discriminates existing from non-existing edges by implicitly learning to separate the edge gradients.
A new readout function inspired by physics is proposed, improving performance of GRAFF-LP and other baseline models.
Heterophily measures specifically tailored for link prediction are suggested, different from those used in node classification.

Read Full Article

16 Likes

Arxiv

Image Credit: Arxiv

A Hierarchical Federated Learning Approach for the Internet of Things

This paper presents a novel federated learning solution, QHetFed, suitable for large-scale Internet of Things deployments.
QHetFed addresses the challenges of large geographic span, communication resource limitation, and data heterogeneity.
The approach is based on hierarchical federated learning over multiple device sets and integrates quantization and data heterogeneity into the learning process.
QHetFed consistently achieves high learning accuracy and outperforms other hierarchical algorithms in scenarios with heterogeneous data distributions.

Read Full Article

Arxiv

332

Image Credit: Arxiv

Towards Efficient Risk-Sensitive Policy Gradient: An Iteration Complexity Analysis

Reinforcement Learning (RL) frameworks often face challenges in terms of iteration efficiency and robustness.
Risk-sensitive policy gradient methods aim to yield more robust policies, but their iteration complexity is not well understood.
A rigorous analysis of the risk-sensitive policy gradient method reveals an iteration complexity of O(ε^-2) to reach an ε-approximate first-order stationary point.
Empirical evaluation shows that risk-averse cases can converge and stabilize faster compared to risk-neutral counterparts.

Read Full Article

20 Likes

Arxiv

164

Image Credit: Arxiv

Contextualized Messages Boost Graph Representations

Graph neural networks (GNNs) are being used to process data represented as graphs.
A new perspective on the representational capability of GNNs is explored.
A soft-injective function and a soft-isomorphic relational graph convolution network (SIR-GCN) are proposed.
Experiments show that SIR-GCN outperforms comparable models in prediction tasks.

Read Full Article

9 Likes

Discover more

Arxiv

240

Image Credit: Arxiv

Collaborative Pareto Set Learning in Multiple Multi-Objective Optimization Problems

Pareto Set Learning (PSL) is an emerging research area in multi-objective optimization.
Existing PSL methods are limited to addressing a single Multi-objective Optimization Problem (MOP) at a time.
A Collaborative Pareto Set Learning (CoPSL) framework is proposed to learn the Pareto sets of multiple MOPs simultaneously.
CoPSL efficiently learns the Pareto sets of multiple MOPs, leveraging shared representations and improving approximation capabilities.

Read Full Article

14 Likes

Arxiv

168

Image Credit: Arxiv

A Causal Framework for Evaluating Deferring Systems

Deferring systems extend supervised Machine Learning (ML) models with the possibility to defer predictions to human experts.
This paper evaluates the impact of deferring strategies on system accuracy through a causal lens.
The potential outcomes framework for causal inference is linked with deferring systems to identify the causal impact of the deferring strategy on predictive accuracy.
The approach is evaluated on synthetic and real datasets for seven deferring systems from the literature.

Read Full Article

10 Likes

Arxiv

120

Image Credit: Arxiv

CONFINE: Conformal Prediction for Interpretable Neural Networks

Deep neural networks often lack interpretability, which limits their utility in fields like healthcare where transparency is crucial.
A new framework called Conformal Prediction for Interpretable Neural Networks (CONFINE) generates prediction sets with statistically robust uncertainty estimates.
CONFINE provides example-based explanations, confidence estimates, and improves accuracy by up to 3.6%.
CONFINE achieves a correct efficiency that is 3.3% higher than the original accuracy, demonstrating its validity across different tasks and surpassing previous methods.

Read Full Article

7 Likes

Arxiv

260

Image Credit: Arxiv

AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs

Domain reweighting is an emerging research area aimed at adjusting the relative weights of different data sources to improve the effectiveness and efficiency of LLM pre-training.
The existing practice of determining competitive data mixtures in small-scale experiments and directly applying them at larger scales may not retain their advantage.
AutoScale is a two-stage, scale-aware data composition framework that fits a parametric model to predict the loss under different data compositions, finding an approximate best allocation at smaller budgets and extrapolating that composition to larger budgets.
Empirical results show that AutoScale accelerates convergence, improves downstream performance, and achieves a 28% faster perplexity reduction than baselines when pre-training GPT-2 Large.

Read Full Article

15 Likes

Arxiv

236

Image Credit: Arxiv

Invariant deep neural networks under the finite group for solving partial differential equations

Utilizing physics-informed neural networks (PINN) to solve partial differential equations (PDEs) becomes a hot issue and also shows its great powers.
In this paper, a symmetry-enhanced deep neural network (sDNN) is proposed.
sDNN makes the architecture of neural networks invariant under the finite group.
Numerical results show that the sDNN outperforms the vanilla PINN with fewer training points and simpler architecture.

Read Full Article

14 Likes

Arxiv

Image Credit: Arxiv

A Self-Supervised Paradigm for Data-Efficient Medical Foundation Model Pre-training: V-information Optimization Framework

Self-supervised pre-training of medical foundation models on large-scale datasets is a common approach for achieving good performance.
Current methods for increasing pre-training data volume do not necessarily improve model performance.
The introduction of V-information in self-supervised pre-training provides a theoretical foundation for sample selection.
OptiDEL, an optimized data-effective learning method, outperforms existing approaches on multiple datasets by using 20x less training data.

Read Full Article

1 Like

Arxiv

Image Credit: Arxiv

FedQUIT: On-Device Federated Unlearning via a Quasi-Competent Virtual Teacher

FedQUIT is a novel algorithm that enables on-device federated unlearning.
It uses knowledge distillation to scrub the contribution of data from an FL global model.
FedQUIT tailors the teacher's output on local data to induce forgetting.
Experimental results show that FedQUIT outperforms existing methods in forgetting data.

Read Full Article

3 Likes

Arxiv

228

Image Credit: Arxiv

Biased Dueling Bandits with Stochastic Delayed Feedback

The dueling bandit problem is gaining popularity in various fields due to its applications in online advertising, recommendation systems, and more.
Delays in feedback pose a challenge to existing dueling bandit literature, affecting the agent's ability to update their policy quickly and accurately.
A new problem called biased dueling bandit problem with stochastic delayed feedback is introduced, involving preference bias between selections.
Two algorithms are presented to handle delayed feedback, one requiring complete delay distribution information and the other only the expected value of delay.

Read Full Article

13 Likes

Arxiv

200

Image Credit: Arxiv

UNA: Unifying Alignments of RLHF/PPO, DPO and KTO by a Generalized Implicit Reward Function

Alignment techniques in reinforcement learning have limitations such as being complex, time-consuming, memory intensive, and unstable during training processes.
A proposed solution, UNA (Unified Alignment), unifies RLHF/PPO, DPO, and KTO techniques and can accommodate different feedback types.
UNA aims to minimize the difference between an implicit reward and an explicit reward, outperforming RLHF/PPO while simplifying and speeding up the RL fine-tuning process.
In experiments, UNA performs better than DPO, KTO, and RLHF.

Read Full Article

12 Likes

Arxiv

332

Image Credit: Arxiv

Large-Scale Targeted Cause Discovery with Data-Driven Learning

A new machine learning approach has been proposed for inferring causal variables of a target variable from observations.
The approach directly infers a set of causal factors without requiring full causal graph reconstruction, making it computationally efficient for large-scale systems.
The identified causal set includes potential regulators of the target variable, enabling efficient regulation in varying intervention costs and feasibility.
Empirical results demonstrate superior performance in identifying causal relationships within large-scale gene regulatory networks.

Read Full Article

20 Likes

Arxiv

232

Image Credit: Arxiv

Preserving Diversity in Supervised Fine-Tuning of Large Language Models

Large Language Models (LLMs) typically rely on Supervised Fine-Tuning (SFT) to specialize in downstream tasks.
Cross Entropy (CE) loss leads to reduced diversity in the model's outputs, hindering further development.
A new game-theoretic formulation for SFT is introduced to address output diversity limitations.
The proposed approach enhances output diversity without compromising downstream performance.

Read Full Article

13 Likes

For uninterrupted reading, download the app