menu
techminis

A naukri.com initiative

google-web-stories
Home

>

ML News

ML News

source image

Arxiv

1d

read

302

img
dot

Image Credit: Arxiv

The impact of internal variability on benchmarking deep learning climate emulators

  • Full-complexity Earth system models (ESMs) are computationally expensive, limiting their use in exploring climate outcomes.
  • Efficient emulators that approximate ESMs are being used to map emissions onto climate outcomes.
  • A comparison between deep learning emulators and a linear regression-based emulator was conducted on ClimateBench, a popular benchmark for data-driven climate emulation.
  • The linear regression-based emulator outperformed the deep learning foundation model on 3 out of 4 regionally-resolved climate variables.

Read Full Article

like

18 Likes

source image

Arxiv

1d

read

128

img
dot

Image Credit: Arxiv

Monge-Kantorovich Fitting With Sobolev Budgets

  • The paper discusses the problem of approximating an n-dimensional probability measure with an m-dimensional measure.
  • The approach involves the use of Monge-Kantorovich (Wasserstein) p-cost to quantify the performance of the approximation.
  • The complexity is constrained by bounding the Sobolev norm of the coverable support by an 'f' function.
  • The study also presents a gradient analysis of the functional and proposes interpretations for regularization in improvement of training.

Read Full Article

like

7 Likes

source image

Arxiv

1d

read

256

img
dot

Image Credit: Arxiv

On the Implicit Relation Between Low-Rank Adaptation and Differential Privacy

  • Low-rank adaptation methods in natural language processing, such as LoRA and FLoRA, involve keeping pre-trained model weights fixed and incorporating trainable low-rank decomposition matrices into some layers of the transformer architecture, called adapters.
  • Researchers have found that the low-rank adaptation used in LoRA and FLoRA introduces random noise into the batch gradients with respect to the adapter parameters, leading to a variance in the injected noise that increases as the adaptation rank decreases.
  • The study establishes a relationship between low-rank adaptation and differential privacy, showing that the dynamics of low-rank adaptation is similar to differentially private fine-tuning of the adapters.
  • The researchers suggest that low-rank adaptation offers privacy protection without the high space complexity of differentially private stochastic gradient descent (DPSGD), providing an efficient alternative for privacy-preserving fine-tuning in NLP models.

Read Full Article

like

15 Likes

source image

Arxiv

1d

read

11

img
dot

Image Credit: Arxiv

Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization

  • Large Language Models (LLMs), built on Transformer architectures, exhibit remarkable generalization across a wide range of tasks.
  • Fine-tuning LLMs for specific tasks remains resource-intensive due to extensive parameterization.
  • Two remarkable phenomena related to the attention mechanism during fine-tuning of LLMs are investigated.
  • Insights from the investigation lead to a new strategy to improve fine-tuning efficiency in terms of storage and time.

Read Full Article

like

Like

source image

Arxiv

1d

read

236

img
dot

Image Credit: Arxiv

RelChaNet: Neural Network Feature Selection using Relative Change Scores

  • RelChaNet is a supervised feature selection algorithm that utilizes neuron pruning and regrowth in a dense neural network's input layer.
  • For pruning, a relative change metric is used to measure the impact a feature has on the network.
  • In addition, an extension is proposed to dynamically adapt the size of the input layer during runtime.
  • Experimental results on 13 datasets demonstrate that RelChaNet outperforms existing methods, with a 2% increase in accuracy on the MNIST dataset.

Read Full Article

like

14 Likes

source image

Arxiv

1d

read

73

img
dot

Image Credit: Arxiv

Improving Neural Optimal Transport via Displacement Interpolation

  • Optimal Transport (OT) theory investigates the cost-minimizing transport map that moves a source distribution to a target distribution.
  • Existing methods for learning the optimal transport map using neural networks often experience training instability and sensitivity to hyperparameters.
  • A novel method called Displacement Interpolation Optimal Transport Model (DIOTM) is proposed to improve stability and achieve a better approximation of the OT Map.
  • DIOTM outperforms existing OT-based models on image-to-image translation tasks.

Read Full Article

like

4 Likes

source image

Arxiv

1d

read

182

img
dot

Image Credit: Arxiv

DDIL: Diversity Enhancing Diffusion Distillation With Imitation Learning

  • Diffusion models excel at generative modeling but have limitations in sampling due to multiple denoising network passes.
  • Co-variate shift is identified as a reason for poor performance of multi-step distilled models.
  • To address co-variate shift, the researchers propose a diffusion distillation within an imitation learning framework (DDIL).
  • DDIL enhances training distribution for distilling diffusion models, improving performance and stability.

Read Full Article

like

10 Likes

source image

Arxiv

1d

read

0

img
dot

Image Credit: Arxiv

MazeNet: An Accurate, Fast, and Scalable Deep Learning Solution for Steiner Minimum Trees

  • MazeNet is a deep learning-based method for solving the Obstacle Avoiding Rectilinear Steiner Minimum Tree (OARSMT) problem.
  • MazeNet reframes OARSMT as a maze-solving task and utilizes a recurrent convolutional neural network (RCNN).
  • MazeNet achieves perfect OARSMT-solving accuracy, reduces runtime compared to classical exact algorithms, and can handle more terminals than approximate algorithms.
  • The scalability of MazeNet allows for training on small mazes and solving larger mazes by replicating pre-trained blocks.

Read Full Article

like

Like

source image

Arxiv

1d

read

232

img
dot

Image Credit: Arxiv

Trojan Cleansing with Neural Collapse

  • Trojan Cleansing with Neural Collapse
  • Trojan attacks are sophisticated training-time attacks on neural networks that embed backdoor triggers.
  • The researchers connect trojan attacks to Neural Collapse, a phenomenon that affects over-parameterized neural networks.
  • They designed a mechanism to cleanse trojan attacks from different network architectures and demonstrated its efficacy.

Read Full Article

like

14 Likes

source image

Arxiv

1d

read

38

img
dot

Image Credit: Arxiv

Machine Learning Analysis of Anomalous Diffusion

  • Machine learning techniques are increasingly being used for analyzing anomalous diffusion.
  • The review focuses on single trajectory characterization and representation learning.
  • Various machine learning methods, including classical and deep learning, are compared.
  • The study offers valuable perspectives for future research in the field.

Read Full Article

like

2 Likes

source image

Arxiv

1d

read

349

img
dot

Image Credit: Arxiv

COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation

  • Retrieval augmentation, the practice of retrieving additional data from large auxiliary pools, has emerged as an effective technique for enhancing model performance in the low-data regime.
  • Prior approaches have employed only nearest-neighbor based strategies for data selection, which retrieve auxiliary samples with high similarity to instances in the target task.
  • COBRA (COmBinatorial Retrieval Augmentation) is a new approach that employs an alternative CMI measure that considers both diversity and similarity to a target dataset for retrieval augmentation.
  • COBRA consistently outperforms previous retrieval approaches, providing significant gains in downstream model performance without incurring significant computational overhead.

Read Full Article

like

21 Likes

source image

Arxiv

1d

read

31

img
dot

Image Credit: Arxiv

ML Mule: Mobile-Driven Context-Aware Collaborative Learning

  • Artificial intelligence has been integrated into nearly every aspect of daily life, but often detached from individual users.
  • ML Mule proposes a mobile-driven approach to address privacy concerns and provide real-time, personalized experiences.
  • ML Mule utilizes individual mobile devices as 'mules' to train and transport model snapshots through physical spaces.
  • ML Mule converges faster and achieves higher model accuracy compared to existing methods.

Read Full Article

like

1 Like

source image

Arxiv

1d

read

267

img
dot

Image Credit: Arxiv

Cross-Cluster Weighted Forests

  • Adapting machine learning algorithms to handle clusters or batch effects within training datasets is important in various biological applications.
  • Ensembling Random Forest learners trained on clusters in datasets with heterogeneous feature distributions improves accuracy and generalizability.
  • The Cross-Cluster Weighted Forest approach shows significant benefits over the traditional Random Forest algorithm.
  • The approach outperforms classic Random Forest in cancer molecular profiling and gene expression datasets.

Read Full Article

like

16 Likes

source image

Arxiv

1d

read

310

img
dot

Image Credit: Arxiv

Benchmarking the Robustness of Instance Segmentation Models

  • This paper presents a comprehensive evaluation of instance segmentation models with respect to real-world image corruptions and out-of-domain image collections.
  • The evaluation shows the generalization capability of models, an essential aspect of real-world applications and domain adaptation.
  • The study includes analysis of network architectures, normalization layers, pretrained networks, and the effect of multi-task training on robustness and generalization.
  • Insights from the study indicate the impact of group normalization, batch normalization, and image resolution on the performance of instance segmentation models.

Read Full Article

like

18 Likes

source image

Arxiv

1d

read

201

img
dot

Image Credit: Arxiv

Minerva: A File-Based Ransomware Detector

  • Minerva is a file-based ransomware detector.
  • It is engineered to be robust against evasion attacks.
  • Minerva accurately identifies different types of ransomware and can generalize to unseen threats.
  • It has a quick detection time of 0.52 seconds, enabling the use of data loss prevention techniques with minimal overhead.

Read Full Article

like

12 Likes

For uninterrupted reading, download the app