A naukri.com initiative
ML News
Arxiv
15h
43
Image Credit: Arxiv
TransMamba: Flexibly Switching between Transformer and Mamba
- TransMamba is a framework that combines Transformer and Mamba models for efficient long-sequence processing.
- TransMamba uses shared parameter matrices to switch between attention and state space model (SSM) mechanisms.
- The framework includes a Memory converter to bridge Transformer and Mamba models for seamless information flow.
- Experimental results demonstrate that TransMamba achieves superior training efficiency and performance compared to baselines.
Read Full Article
2 Likes
Arxiv
15h
25
Image Credit: Arxiv
Level the Level: Balancing Game Levels for Asymmetric Player Archetypes With Reinforcement Learning
- This work focuses on generating balanced levels tailored to asymmetric player archetypes in games.
- The goal is to balance the disparity in abilities through the level design.
- A method using reinforcement learning is used to balance tile-based game levels.
- The evaluation shows that the method can balance a larger proportion of levels compared to two baseline approaches.
Read Full Article
1 Like
Arxiv
15h
164
Image Credit: Arxiv
CTSketch: Compositional Tensor Sketching for Scalable Neurosymbolic Learning
- CTSketch is a novel, scalable neurosymbolic learning algorithm for training neural networks using end-to-end input-output labels.
- CTSketch decomposes the symbolic program into sub-programs and summarizes each sub-program with a sketched tensor to improve scalability.
- The algorithm approximates the output distribution of the program using simple tensor operations over input distributions and summaries.
- CTSketch achieves high accuracy on tasks involving over one thousand inputs, pushing neurosymbolic learning to new scales.
Read Full Article
9 Likes
Arxiv
15h
127
Image Credit: Arxiv
Learning a Canonical Basis of Human Preferences from Binary Ratings
- Recent advances in generative AI have been driven by alignment techniques such as reinforcement learning from human feedback (RLHF).
- This paper focuses on understanding the preferences encoded in datasets used for RLHF and identifying common human preferences.
- A small subset of 21 preference categories captures over 89% of preference variation across individuals, serving as a canonical basis of human preferences.
- The identified preference basis proves useful for model evaluation and training, offering insights into model alignment and successful fine-tuning.
Read Full Article
7 Likes
Arxiv
15h
259
Image Credit: Arxiv
Predicting Targeted Therapy Resistance in Non-Small Cell Lung Cancer Using Multimodal Machine Learning
- Lung cancer is the primary cause of cancer death globally, with non-small cell lung cancer (NSCLC) being the most common subtype.
- A new study has developed a multimodal machine learning model to predict patient resistance to osimertinib, a third-generation EGFR-tyrosine kinase inhibitor, in late-stage NSCLC patients with activating EGFR mutations.
- The model achieved a c-index of 0.82 on a multi-institutional dataset by integrating various data types such as histology images, next-generation sequencing (NGS) data, demographics data, and clinical records.
- The multimodal model demonstrated superior performance over single modality models, highlighting the importance of combining multiple data types for accurate patient outcome prediction.
Read Full Article
15 Likes
Arxiv
15h
113
Image Credit: Arxiv
Ride-Sourcing Vehicle Rebalancing with Service Accessibility Guarantees via Constrained Mean-Field Reinforcement Learning
- The rapid expansion of ride-sourcing services presents operational challenges, such as vehicle rebalancing.
- A scalable mean-field control and reinforcement learning model is proposed for precise vehicle repositioning.
- An accessibility constraint is integrated to ensure equitable service distribution.
- Empirical evaluation using real-world data-driven simulation demonstrates the efficiency and robustness of the approach.
Read Full Article
6 Likes
Arxiv
15h
164
Image Credit: Arxiv
Many-to-Many Matching via Sparsity Controlled Optimal Transport
- Many-to-many matching seeks to match multiple points in one set and multiple points in another set.
- This paper proposes a novel many-to-many matching method that explicitly encodes many-to-many constraints while preventing one-to-one matching.
- The method includes matching budget constraints and a deformed $q$-entropy regularization to maximize the matching budget.
- Experimental results show that the proposed method achieves good performance in generating meaningful many-to-many matchings.
Read Full Article
9 Likes
Arxiv
15h
149
Image Credit: Arxiv
Spatio-temporal Prediction of Fine-Grained Origin-Destination Matrices with Applications in Ridesharing
- Accurate spatial-temporal prediction of network-based travelers' requests is crucial for the effective policy design of ridesharing platforms.
- This paper introduces a novel prediction model, OD-CED, for fine-grained Origin-Destination (OD) demand prediction in ridesharing platforms.
- OD-CED combines an unsupervised space coarsening technique and an encoder-decoder architecture to capture both semantic and geographic dependencies.
- Experimental results show that OD-CED outperforms traditional statistical methods, achieving significant reductions in root-mean-square error and weighted mean absolute percentage error.
Read Full Article
9 Likes
Arxiv
15h
18
Image Credit: Arxiv
Advances in Continual Graph Learning for Anti-Money Laundering Systems: A Comprehensive Review
- Financial institutions are required to monitor vast amounts of transactions for money laundering.
- Traditional machine learning models have limitations in adapting to dynamic environments for AML detection.
- Continual graph learning approaches can enhance AML practices by incorporating new information while retaining prior knowledge.
- Experimental evaluations show that continual learning improves model adaptability and robustness in detecting money laundering.
Read Full Article
1 Like
Arxiv
15h
237
Image Credit: Arxiv
Evaluating and Designing Sparse Autoencoders by Approximating Quasi-Orthogonality
- Researchers propose a new evaluation metric called Approximate Feature Activation (AFA) for assessing alignment between inputs and activations in Sparse Autoencoders (SAEs).
- The study introduces a novel SAE architecture called top-AFA SAE, which eliminates the need to tune SAE sparsity hyperparameters.
- The top-AFA SAEs achieve reconstruction loss comparable to state-of-the-art top-k SAEs without requiring the hyperparameter k to be tuned.
- The proposed method also introduces the ZF plot, revealing a relationship between large language model hidden embeddings and SAE feature vectors.
Read Full Article
14 Likes
Arxiv
15h
340
Image Credit: Arxiv
Value of Information-based Deceptive Path Planning Under Adversarial Interventions
- Existing methods for deceptive path planning (DPP) do not address the problem of adversarial interventions.
- A novel Markov decision process (MDP)-based model is proposed for DPP under adversarial interventions.
- New value of information (VoI) objectives are developed to guide DPP policy design.
- Computationally efficient methods are derived for synthesizing policies for DPP under adversarial interventions.
Read Full Article
20 Likes
Arxiv
15h
25
Image Credit: Arxiv
Evaluating machine learning models for predicting pesticides toxicity to honey bees
- Small molecules play a critical role in the biomedical, environmental, and agrochemical domains.
- This work focuses on ApisTox, the most comprehensive dataset of experimentally validated chemical toxicity to the honey bee (Apis mellifera).
- The evaluation of ApisTox using various machine learning approaches reveals that it represents a distinct chemical space.
- The limited generalizability of current state-of-the-art algorithms trained solely on biomedical data highlights the need for targeted model development in the agrochemical domain.
Read Full Article
1 Like
Arxiv
15h
241
Image Credit: Arxiv
NoProp: Training Neural Networks without Back-propagation or Forward-propagation
- The paper introduces a new learning method named NoProp, which does not rely on either forward or backward propagation in deep learning.
- NoProp takes inspiration from diffusion and flow matching methods to independently learn to denoise a noisy target at each layer.
- The method demonstrates superior accuracy, ease of use, and computational efficiency compared to other back-propagation-free methods on image classification benchmarks such as MNIST, CIFAR-10, and CIFAR-100.
- NoProp alters the traditional gradient-based learning paradigm, enabling more efficient distributed learning and potentially impacting other characteristics of the learning process.
Read Full Article
14 Likes
Arxiv
15h
223
Image Credit: Arxiv
ORAL: Prompting Your Large-Scale LoRAs via Conditional Recurrent Diffusion
- Parameter generation has emerged as a novel paradigm for neural network development, offering an alternative to traditional neural network training by synthesizing high-quality model weights directly.
- In this paper, a novel conditional recurrent diffusion framework called ORAL is introduced, which addresses the limitations of existing methods in achieving scalability and controllability.
- ORAL incorporates a novel conditioning mechanism to generate task-specific Low-Rank Adaptation (LoRA) parameters that can seamlessly transfer across evolving language models.
- Extensive experiments show that ORAL generates high-quality LoRA parameters, achieving comparable or superior performance to vanilla trained counterparts across various language, vision, and multimodal tasks.
Read Full Article
13 Likes
Arxiv
15h
237
Image Credit: Arxiv
SQuat: Subspace-orthogonal KV Cache Quantization
- Researchers propose SQuat (Subspace-orthogonal KV cache quantization) to reduce memory usage in key-value (KV) cache used for LLMs decoding.
- SQuat constructs a subspace spanned by query tensors to capture critical task-related information.
- SQuat enforces orthogonality between (de)quantized and original keys in the subspace, minimizing the impact of quantization errors.
- The method achieves reduced memory usage, improved throughput, and better benchmark scores compared to existing KV cache quantization algorithms.
Read Full Article
14 Likes
For uninterrupted reading, download the app