A naukri.com initiative
ML News
Arxiv
1w
323
Image Credit: Arxiv
Distilling Knowledge from Heterogeneous Architectures for Semantic Segmentation
- Current knowledge distillation methods for semantic segmentation focus on guiding the student to imitate the teacher's knowledge within homogeneous architectures.
- A generic knowledge distillation method for semantic segmentation from a heterogeneous perspective called HeteroAKD is proposed.
- HeteroAKD eliminates the influence of architecture-specific information by projecting intermediate features of the teacher and student into an aligned logits space.
- HeteroAKD outperforms state-of-the-art KD methods in facilitating distillation between heterogeneous architectures.
Read Full Article
19 Likes
Arxiv
1w
411
Image Credit: Arxiv
Counting Hours, Counting Losses: The Toll of Unpredictable Work Schedules on Financial Security
- Financial instability has become a significant issue, but the impact of unstable work schedules is often overlooked.
- Unpredictable work schedules lead to burnout, work-family conflicts, and financial shocks affecting workers' income and assets.
- Workers in sectors with frequently changing schedules are particularly vulnerable, exacerbating their financial fragility.
- A simulation framework shows how workers' ability to anticipate schedule changes enhances their long-term utility.
Read Full Article
24 Likes
Arxiv
1w
348
Image Credit: Arxiv
DG-STMTL: A Novel Graph Convolutional Network for Multi-Task Spatio-Temporal Traffic Forecasting
- Spatio-temporal traffic prediction is important in intelligent transportation systems.
- Traditional Graph Convolutional Networks (GCNs) face challenges with static adjacency matrices and learnable matrices.
- The study introduces a novel Multi-Task Learning (MTL) framework called DG-STMTL.
- DG-STMTL combines static and dynamic adjacency matrices and includes a group-wise GCN module.
Read Full Article
20 Likes
Arxiv
1w
71
Image Credit: Arxiv
Pychop: Emulating Low-Precision Arithmetic in Numerical Methods and Neural Networks
- A new library, Pychop, has been developed to enable low-precision emulation in numerical methods and neural networks using Python programming language.
- Low-precision training in deep learning has been successful in reducing memory and energy consumption while maintaining model fidelity.
- Pychop provides customizable floating-point formats and a comprehensive set of rounding modes for fast, low-precision emulation in various applications.
- The library includes interfaces for PyTorch and JAX, allowing efficient low-precision emulation on GPUs for neural network training and inference.
Read Full Article
4 Likes
Arxiv
1w
33
Image Credit: Arxiv
Robust Hallucination Detection in LLMs via Adaptive Token Selection
- Recent research in hallucination detection in large language models (LLMs) has shown that LLMs' internal representations contain truthfulness hints that can be used for detector training.
- However, the performance of these detectors is heavily dependent on predetermined tokens and fluctuates when working on free-form generations with varying lengths and sparse distributions of hallucinated entities.
- To address this, a novel approach called HaMI is proposed, which enables robust detection of hallucinations through adaptive selection and learning of critical tokens that are most indicative of hallucinations.
- Experimental results on four hallucination benchmarks demonstrate that HaMI outperforms existing state-of-the-art approaches.
Read Full Article
2 Likes
Arxiv
1w
415
Image Credit: Arxiv
SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning
- Recent advances in inference-time compute have improved performance on complex tasks using Large Reasoning Models (LRMs).
- The high inference latency is a trade-off for improved accuracy due to the length of generated reasoning sequences and autoregressive decoding.
- SpecReason is a system that accelerates LRM inference by using a lightweight model to carry out simpler intermediate reasoning steps.
- SpecReason achieves 1.5-2.5x speedup over vanilla LRM inference while improving accuracy by 1.0-9.9%.
Read Full Article
25 Likes
Arxiv
1w
8
Image Credit: Arxiv
DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flows
- DiverseFlow is a training-free approach to improve the diversity of flow models.
- It uses a determinantal point process to induce a coupling between samples, driving diversity within a fixed sampling budget.
- DiverseFlow allows exploration of more variations in a learned flow model with fewer samples, making it sample-efficient.
- The method demonstrates efficacy in tasks such as text-guided image generation, large-hole inpainting, and class-conditional image synthesis.
Read Full Article
Like
Arxiv
1w
16
Image Credit: Arxiv
Fast Adaptation with Behavioral Foundation Models
- Unsupervised zero-shot reinforcement learning (RL) is a powerful paradigm for pretraining behavioral foundation models (BFMs).
- The BFMs solve downstream tasks specified via reward functions in a zero-shot fashion without additional test-time learning or planning.
- This paper focuses on devising fast adaptation strategies to improve the zero-shot performance of BFMs in a few steps of online interaction with the environment.
- The proposed strategies achieve 10-40% improvement over the zero-shot performance in a few tens of episodes, outperforming existing baselines.
Read Full Article
1 Like
Arxiv
1w
54
Image Credit: Arxiv
Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining
- Reinforcement learning (RL) based fine-tuning is important for post-training language models for advanced mathematical reasoning and coding.
- RL fine-tuning consistently improves performance, even in smaller-scale models, but the underlying mechanisms are not well-understood.
- RL fine-tuning amplifies patterns in the pretraining data and converges towards a dominant output distribution.
- RL post-training on simpler questions can lead to performance gains on harder ones, indicating generalization of reasoning capabilities.
Read Full Article
3 Likes
Arxiv
1w
71
Image Credit: Arxiv
Semantically Encoding Activity Labels for Context-Aware Human Activity Recognition
- Prior work primarily formulates Context-Aware Human Activity Recognition (CA-HAR) as a multi-label classification problem.
- Existing CA-HAR methods struggle to capture the semantic relationships between activity labels, limiting their accuracy.
- To address this limitation, researchers propose SEAL, which leverages Language Models (LMs) to encode CA-HAR activity labels.
- SEAL uses LMs to generate vector embeddings that preserve rich semantic information from natural language, improving CA-HAR accuracy.
Read Full Article
4 Likes
Arxiv
1w
296
Image Credit: Arxiv
C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing
- Mixture-of-Experts (MoE) Large Language Models (LLMs) suffer from sub-optimal expert pathways resulting in lower accuracy.
- A novel class of test-time optimization methods, called C3PO, is developed to re-weight or 're-mix' the experts in different layers for each test sample.
- C3PO applies optimization only to the core experts' mixing weights in critical layers, resulting in improved accuracy while saving computation.
- C3PO consistently improves the accuracy of MoE LLMs by 7-15% and outperforms other test-time learning methods.
Read Full Article
17 Likes
Arxiv
1w
16
Image Credit: Arxiv
DOMAC: Differentiable Optimization for High-Speed Multipliers and Multiply-Accumulators
- DOMAC is a novel approach that employs differentiable optimization for designing multipliers and multiply-accumulators (MACs) at specific technology nodes.
- DOMAC reformulates the discrete optimization challenge into a continuous problem by incorporating differentiable timing and area objectives.
- Experimental results show that DOMAC achieves significant enhancements in both performance and area efficiency compared to state-of-the-art baselines and commercial IPs in multiplier and MAC designs.
Read Full Article
1 Like
Arxiv
1w
395
Image Credit: Arxiv
Behavior Importance-Aware Graph Neural Architecture Search for Cross-Domain Recommendation
- Cross-domain recommendation (CDR) addresses data sparsity and cold-start challenges in recommendation systems.
- Behavior importance-aware Graph Neural Architecture Search (BiGNAS) is proposed to optimize GNN architecture and data importance for CDR.
- BiGNAS uses a Cross-Domain Customized Supernetwork to search the optimal GNN architecture without retraining.
- The Graph-Based Behavior Importance Perceptron dynamically assesses the importance of source domain behaviors for improved recommendations.
Read Full Article
23 Likes
Arxiv
1w
403
Image Credit: Arxiv
Relevance Isn't All You Need: Scaling RAG Systems With Inference-Time Compute Via Multi-Criteria Reranking
- Modern Large Language Model (LLM) systems rely on Retrieval Augmented Generation (RAG) to gather useful context for response generation.
- Maximizing context relevance alone in RAG systems can result in degraded downstream response quality.
- The evaluation of existing RAG methods shows that they scale poorly with inference time compute usage.
- Introducing "RErank BEyond reLevance (REBEL)" enables RAG systems to scale by using multi-criteria optimization for higher relevance and superior answer quality.
Read Full Article
24 Likes
Arxiv
1w
415
Image Credit: Arxiv
Guarding Digital Privacy: Exploring User Profiling and Security Enhancements
- User profiling, the practice of collecting user information for personalized recommendations, has become widespread, driving progress in technology.
- This article aims to consolidate knowledge on user profiling, exploring various approaches and associated challenges.
- The article unveils privacy vulnerabilities in two companies sharing user data and in 18 popular Android applications in India across various categories.
- The paper suggests research directions to strengthen digital security measures.
Read Full Article
25 Likes
For uninterrupted reading, download the app