Machine Learning (ML) Latest News and Trending articles from all top sources only on Techminis

A naukri.com initiative

New

Home

ML News

Arxiv

302

Image Credit: Arxiv

The impact of internal variability on benchmarking deep learning climate emulators

Full-complexity Earth system models (ESMs) are computationally expensive, limiting their use in exploring climate outcomes.
Efficient emulators that approximate ESMs are being used to map emissions onto climate outcomes.
A comparison between deep learning emulators and a linear regression-based emulator was conducted on ClimateBench, a popular benchmark for data-driven climate emulation.
The linear regression-based emulator outperformed the deep learning foundation model on 3 out of 4 regionally-resolved climate variables.

Read Full Article

18 Likes

Arxiv

128

Image Credit: Arxiv

Monge-Kantorovich Fitting With Sobolev Budgets

The paper discusses the problem of approximating an n-dimensional probability measure with an m-dimensional measure.
The approach involves the use of Monge-Kantorovich (Wasserstein) p-cost to quantify the performance of the approximation.
The complexity is constrained by bounding the Sobolev norm of the coverable support by an 'f' function.
The study also presents a gradient analysis of the functional and proposes interpretations for regularization in improvement of training.

Read Full Article

7 Likes

Arxiv

256

Image Credit: Arxiv

On the Implicit Relation Between Low-Rank Adaptation and Differential Privacy

Low-rank adaptation methods in natural language processing, such as LoRA and FLoRA, involve keeping pre-trained model weights fixed and incorporating trainable low-rank decomposition matrices into some layers of the transformer architecture, called adapters.
Researchers have found that the low-rank adaptation used in LoRA and FLoRA introduces random noise into the batch gradients with respect to the adapter parameters, leading to a variance in the injected noise that increases as the adaptation rank decreases.
The study establishes a relationship between low-rank adaptation and differential privacy, showing that the dynamics of low-rank adaptation is similar to differentially private fine-tuning of the adapters.
The researchers suggest that low-rank adaptation offers privacy protection without the high space complexity of differentially private stochastic gradient descent (DPSGD), providing an efficient alternative for privacy-preserving fine-tuning in NLP models.

Read Full Article

15 Likes

Arxiv

Image Credit: Arxiv

Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization

Large Language Models (LLMs), built on Transformer architectures, exhibit remarkable generalization across a wide range of tasks.
Fine-tuning LLMs for specific tasks remains resource-intensive due to extensive parameterization.
Two remarkable phenomena related to the attention mechanism during fine-tuning of LLMs are investigated.
Insights from the investigation lead to a new strategy to improve fine-tuning efficiency in terms of storage and time.

Read Full Article

Discover more

Arxiv

236

Image Credit: Arxiv

RelChaNet: Neural Network Feature Selection using Relative Change Scores

RelChaNet is a supervised feature selection algorithm that utilizes neuron pruning and regrowth in a dense neural network's input layer.
For pruning, a relative change metric is used to measure the impact a feature has on the network.
In addition, an extension is proposed to dynamically adapt the size of the input layer during runtime.
Experimental results on 13 datasets demonstrate that RelChaNet outperforms existing methods, with a 2% increase in accuracy on the MNIST dataset.

Read Full Article

14 Likes

Arxiv

Image Credit: Arxiv

Improving Neural Optimal Transport via Displacement Interpolation

Optimal Transport (OT) theory investigates the cost-minimizing transport map that moves a source distribution to a target distribution.
Existing methods for learning the optimal transport map using neural networks often experience training instability and sensitivity to hyperparameters.
A novel method called Displacement Interpolation Optimal Transport Model (DIOTM) is proposed to improve stability and achieve a better approximation of the OT Map.
DIOTM outperforms existing OT-based models on image-to-image translation tasks.

Read Full Article

4 Likes

Arxiv

182

Image Credit: Arxiv

DDIL: Diversity Enhancing Diffusion Distillation With Imitation Learning

Diffusion models excel at generative modeling but have limitations in sampling due to multiple denoising network passes.
Co-variate shift is identified as a reason for poor performance of multi-step distilled models.
To address co-variate shift, the researchers propose a diffusion distillation within an imitation learning framework (DDIL).
DDIL enhances training distribution for distilling diffusion models, improving performance and stability.

Read Full Article

10 Likes

Arxiv

Image Credit: Arxiv

MazeNet: An Accurate, Fast, and Scalable Deep Learning Solution for Steiner Minimum Trees

MazeNet is a deep learning-based method for solving the Obstacle Avoiding Rectilinear Steiner Minimum Tree (OARSMT) problem.
MazeNet reframes OARSMT as a maze-solving task and utilizes a recurrent convolutional neural network (RCNN).
MazeNet achieves perfect OARSMT-solving accuracy, reduces runtime compared to classical exact algorithms, and can handle more terminals than approximate algorithms.
The scalability of MazeNet allows for training on small mazes and solving larger mazes by replicating pre-trained blocks.

Read Full Article

Arxiv

232

Image Credit: Arxiv

Trojan Cleansing with Neural Collapse

Trojan Cleansing with Neural Collapse
Trojan attacks are sophisticated training-time attacks on neural networks that embed backdoor triggers.
The researchers connect trojan attacks to Neural Collapse, a phenomenon that affects over-parameterized neural networks.
They designed a mechanism to cleanse trojan attacks from different network architectures and demonstrated its efficacy.

Read Full Article

14 Likes

Arxiv

Image Credit: Arxiv

Machine Learning Analysis of Anomalous Diffusion

Machine learning techniques are increasingly being used for analyzing anomalous diffusion.
The review focuses on single trajectory characterization and representation learning.
Various machine learning methods, including classical and deep learning, are compared.
The study offers valuable perspectives for future research in the field.

Read Full Article

2 Likes

Arxiv

349

Image Credit: Arxiv

COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation

Retrieval augmentation, the practice of retrieving additional data from large auxiliary pools, has emerged as an effective technique for enhancing model performance in the low-data regime.
Prior approaches have employed only nearest-neighbor based strategies for data selection, which retrieve auxiliary samples with high similarity to instances in the target task.
COBRA (COmBinatorial Retrieval Augmentation) is a new approach that employs an alternative CMI measure that considers both diversity and similarity to a target dataset for retrieval augmentation.
COBRA consistently outperforms previous retrieval approaches, providing significant gains in downstream model performance without incurring significant computational overhead.

Read Full Article

21 Likes

Arxiv

Image Credit: Arxiv

ML Mule: Mobile-Driven Context-Aware Collaborative Learning

Artificial intelligence has been integrated into nearly every aspect of daily life, but often detached from individual users.
ML Mule proposes a mobile-driven approach to address privacy concerns and provide real-time, personalized experiences.
ML Mule utilizes individual mobile devices as 'mules' to train and transport model snapshots through physical spaces.
ML Mule converges faster and achieves higher model accuracy compared to existing methods.

Read Full Article

1 Like

Arxiv

267

Image Credit: Arxiv

Cross-Cluster Weighted Forests

Adapting machine learning algorithms to handle clusters or batch effects within training datasets is important in various biological applications.
Ensembling Random Forest learners trained on clusters in datasets with heterogeneous feature distributions improves accuracy and generalizability.
The Cross-Cluster Weighted Forest approach shows significant benefits over the traditional Random Forest algorithm.
The approach outperforms classic Random Forest in cancer molecular profiling and gene expression datasets.

Read Full Article

16 Likes

Arxiv

310

Image Credit: Arxiv

Benchmarking the Robustness of Instance Segmentation Models

This paper presents a comprehensive evaluation of instance segmentation models with respect to real-world image corruptions and out-of-domain image collections.
The evaluation shows the generalization capability of models, an essential aspect of real-world applications and domain adaptation.
The study includes analysis of network architectures, normalization layers, pretrained networks, and the effect of multi-task training on robustness and generalization.
Insights from the study indicate the impact of group normalization, batch normalization, and image resolution on the performance of instance segmentation models.

Read Full Article

18 Likes

Arxiv

201

Image Credit: Arxiv

Minerva: A File-Based Ransomware Detector

Minerva is a file-based ransomware detector.
It is engineered to be robust against evasion attacks.
Minerva accurately identifies different types of ransomware and can generalize to unseen threats.
It has a quick detection time of 0.52 seconds, enabling the use of data loss prevention techniques with minimal overhead.

Read Full Article

12 Likes

For uninterrupted reading, download the app