A naukri.com initiative
ML News
Marktechpost
1w
183
How AI Models Learn to Solve Problems That Humans Can’t
- Researchers have developed the Easy-to-Hard Generalization (E2H) methodology to tackle alignment issues in complex tasks without relying on human feedback.
- The methodology involves Process-Supervised Reward Models (PRMs), Easy-to-Hard generalization, and Iterative Refinement.
- The E2H methodology enables AI models to shift from human-feedback-dependent to reduced human annotations.
- The method demonstrates significant improvements in performance and reduces the need for human-labeled data on complex tasks.
Read Full Article
11 Likes
Marktechpost
1w
271
Scaling Language Model Evaluation: From Thousands to Millions of Tokens with BABILong
- BABILong is a new benchmark designed to evaluate language models’ reasoning capabilities across long documents. The benchmark employs a new methodology for testing long-term reasoning abilities. The benchmark’s flexibility allows for testing sequences of up to 50 million tokens, making it uniquely suited for evaluating next-generation models. Initial testing reveals significant limitations in current models, with popular LLMs effectively utilizing only 10-20% of available context.
- Testing across various question-answering tasks demonstrates that most current LLMs efficiently use only 10-20% of their advertised context window. While models like GPT-4 and Llama-3.1-70b maintain effectiveness up to 16K tokens, most models struggle beyond 4K tokens. Fine-tuning experiments proved particularly revealing, showing that even relatively small models like RMT and ARMT (137M parameters) can effectively handle BABILong tasks.
- BABILong encompasses 20 distinct reasoning tasks and utilizes books from the PG19 corpora as source material. Notably, this synthetic approach ensures immunity to training data contamination. Testing revealed that even advanced language models like GPT-4 and Gemini 1.5 Pro, utilize only 5-25% of their input context effectively.
- The LRA, L-Eval, and LongBench workbook handles sequences up to 16,000- 60,000 tokens. However, recent developments like InfinityBench and ChapterBreak can handle up to 636,000 tokens. The BABILong benchmark allows unlimited scaling of context length, enabling the evaluation of models with context windows of millions of tokens.
- The new methodology for evaluating language models is unique and helps evaluate long-term reasoning abilities. Fine-tuned recurrent memory models, particularly ARMT, show remarkable capabilities, processing sequences up to 50 million tokens with consistent performance.
- Recently developed benchmarks like Long Align, LongICLBench, InfinityBench, ChapterBreak, and BABILong focus on in-context learning and instruction and push further boundaries by handling sequences up to 636K tokens.
- Due to advancements in Large Language Models (LLMs) and neural architectures have significantly advanced capabilities, particularly by improving processing longer contexts. These improvements have profound implications for various applications.
- The BABILong benchmark is a significant advancement in evaluating language models’ long-context capabilities through its unique combination of scalability and diverse reasoning tasks.
- Recent developments show promising improvements, with Qwen-2.5 models leading among open LLMs. The evaluation also explored alternative approaches, including Retrieval-Augmented Generation (RAG) and fine-tuned models. While RAG demonstrates limited success, fine-tuned recurrent memory models, particularly ARMT, show remarkable capabilities.
- Current evaluation benchmarks remains limited to 40,000 tokens, creating a significant gap between model capabilities and evaluation methods.
Read Full Article
16 Likes
Medium
1w
396
Exploring Real-World Use Cases of Machine Learning Development Across Industries
- Machine learning has revolutionized healthcare, enabling early disease detection and personalized treatment plans.
- Retailers use machine learning to analyze customer behavior and improve personalization.
- Manufacturers employ machine learning to streamline production processes and reduce downtime.
- The financial sector benefits from AI-driven solutions for fraud detection and customer relationship management.
Read Full Article
23 Likes
Arxiv
1w
83
Image Credit: Arxiv
Heterogeneous Multi-Agent Reinforcement Learning for Distributed Channel Access in WLANs
- This paper explores the application of multi-agent reinforcement learning (MARL) to address distributed channel access in wireless local area networks (WLANs).
- The study focuses on the practical scenario where agents adopt heterogeneously value-based or policy-based reinforcement learning algorithms to train the model.
- The researchers propose a heterogeneous MARL training framework called QPMIX, which enables collaboration among heterogeneous agents through centralized training and distributed execution.
- Simulation results demonstrate that the QPMIX algorithm improves network throughput, mean delay, delay jitter, and collision rates compared to conventional carrier-sense multiple access with collision avoidance in saturated traffic scenarios.
Read Full Article
5 Likes
Arxiv
1w
79
Image Credit: Arxiv
A Survey on Inference Optimization Techniques for Mixture of Experts Models
- A new survey analyzes inference optimization techniques for Mixture of Experts (MoE) models.
- The survey categorizes optimization approaches into model-level, system-level, and hardware-level optimizations.
- Model-level optimizations include architectural innovations, compression techniques, and algorithm improvements.
- System-level optimizations investigate distributed computing approaches, load balancing mechanisms, and efficient scheduling algorithms.
Read Full Article
4 Likes
Arxiv
1w
392
Image Credit: Arxiv
Towards Precise Prediction Uncertainty in GNNs: Refining GNNs with Topology-grouping Strategy
- Recent advancements in graph neural networks (GNNs) have highlighted the critical need of calibrating model predictions, with neighborhood prediction similarity recognized as a pivotal component.
- Existing approaches incorporate neighborhood similarity into node-wise temperature scaling techniques, but this assumption does not hold universally and can lead to sub-optimal calibration.
- The new approach called Simi-Mailbox categorizes nodes by both neighborhood similarity and their own confidence, allowing fine-grained calibration using group-specific temperature scaling.
- Extensive experiments demonstrate the effectiveness of Simi-Mailbox, achieving up to 13.79% error reduction compared to uncalibrated GNN predictions.
Read Full Article
23 Likes
Arxiv
1w
304
Image Credit: Arxiv
Distributionally Robust Policy Learning under Concept Drifts
- Distributionally robust policy learning aims to find a policy that performs well under the worst-case distributional shift.
- Existing methods for robust policy learning consider the worst-case joint distribution of the covariate and the outcome, which can be unnecessarily conservative.
- This paper focuses on robust policy learning under concept drift, where only the conditional relationship between the outcome and the covariate changes.
- The paper proposes a learning algorithm that maximizes the estimated policy value within a given policy class, with an optimal sub-optimality gap.
Read Full Article
18 Likes
Arxiv
1w
296
Image Credit: Arxiv
The Multiplex Classification Framework: optimizing multi-label classifiers through problem transformation, ontology engineering, and model ensembling
- A new approach called the Multiplex Classification Framework has been introduced to address the complexities of classification problems through problem transformation, ontology engineering, and model ensembling.
- The framework offers adaptability to any number of classes and logical constraints, a method for managing class imbalance, elimination of confidence threshold selection, and a modular structure.
- Experiments comparing the Multiplex approach with conventional classification models showed significant improvement in classification performance, especially in problems with a large number of classes and class imbalances.
- However, the Multiplex approach requires understanding of the problem domain and experience with ontology engineering, and involves training multiple models.
Read Full Article
17 Likes
Arxiv
1w
271
Image Credit: Arxiv
Stealing That Free Lunch: Exposing the Limits of Dyna-Style Reinforcement Learning
- Dyna-style off-policy model-based reinforcement learning (DMBRL) algorithms are facing a performance gap when applied across different benchmark environments.
- While DMBRL algorithms perform well in OpenAI Gym, their performance drops significantly in DeepMind Control Suite (DMC) with proprioceptive observations.
- Modern techniques designed to address issues in these settings do not consistently improve performance across all environments.
- Adding synthetic rollouts to the training process, which is the backbone of Dyna-style algorithms, significantly degrades performance in most DMC environments.
Read Full Article
16 Likes
Arxiv
1w
125
Image Credit: Arxiv
Covariances for Free: Exploiting Mean Distributions for Federated Learning with Pre-Trained Models
- This research proposes a training-free method for federated learning with pre-trained models.
- The method utilizes an unbiased estimator of class covariance matrices.
- It only requires the communication of class means, reducing communication costs.
- The approach improves performance by 4-26% compared to existing methods with the same communication cost.
Read Full Article
7 Likes
Arxiv
1w
308
Image Credit: Arxiv
A Unifying Information-theoretic Perspective on Evaluating Generative Models
- There is significant current research focused on determining meaningful evaluation metrics for generative models.
- A unifying perspective is needed to allow for easier comparison and clearer explanation of metric benefits and drawbacks.
- A class of kth-nearest-neighbors (kNN)-based metrics is unified under an information-theoretic lens.
- A tri-dimensional metric composed of Precision Cross-Entropy (PCE), Recall Cross-Entropy (RCE), and Recall Entropy (RE) is proposed to measure fidelity and diversity.
Read Full Article
18 Likes
Arxiv
1w
158
Image Credit: Arxiv
Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inference
- Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inference
- Realtime environments change as agents perform action inference and learning, requiring high interaction frequencies to minimize regret.
- Recent advances in machine learning involve larger neural networks with longer inference times, raising concerns about their applicability in realtime systems.
- Proposed algorithms for staggering asynchronous inference processes ensure consistent time intervals for actions, enabling use of models with high inference times.
Read Full Article
9 Likes
Arxiv
1w
37
Image Credit: Arxiv
ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals
- ResQ is a post-training quantization (PTQ) method for large language models (LLMs).
- ResQ uses principal component analysis (PCA) to identify a low-rank subspace with high activation variances.
- Within this subspace, ResQ keeps the coefficients in high precision while quantizing the rest to 4-bit.
- ResQ outperforms recent PTQ methods, achieving lower perplexity and faster inference on benchmarks.
Read Full Article
2 Likes
Arxiv
1w
192
Image Credit: Arxiv
I0T: Embedding Standardization Method Towards Zero Modality Gap
- Contrastive Language-Image Pretraining (CLIP) enables zero-shot inference in downstream tasks such as image-text retrieval and classification.
- Recent works extending CLIP suffer from the issue of modality gap, which arises when the image and text embeddings are projected to disparate manifolds, deviating from the intended objective of image-text contrastive learning.
- Researchers propose two methods to address the modality gap: (1) a post-hoc embedding standardization method, I0T_post, that reduces the modality gap to zero and (2) a trainable method, I0T_async, that adds two normalization layers for each encoder to alleviate the modality gap.
- The I0T framework significantly reduces the modality gap while preserving the original embedding representations of trained models with their locked parameters and can serve as an alternative evaluation metric for CLIPScore.
Read Full Article
11 Likes
Arxiv
1w
183
Image Credit: Arxiv
Balanced Gradient Sample Retrieval for Enhanced Knowledge Retention in Proxy-based Continual Learning
- Continual learning in deep neural networks often suffers from catastrophic forgetting, where representations for previous tasks are overwritten during subsequent training.
- A novel sample retrieval strategy is proposed that leverages both gradient-conflicting and gradient-aligned samples to retain knowledge about past tasks.
- Gradient-conflicting samples are selected to reduce interference and re-align gradients, preserving past task knowledge.
- Experiments validate the method's state-of-the-art performance in mitigating forgetting and maintaining competitive accuracy on new tasks.
Read Full Article
11 Likes
For uninterrupted reading, download the app