A naukri.com initiative
ML News
Arxiv
1d
166
Image Credit: Arxiv
RILQ: Rank-Insensitive LoRA-based Quantization Error Compensation for Boosting 2-bit Large Language Model Accuracy
- Researchers propose Rank-Insensitive LoRA-based Quantization Error Compensation (RILQ) to understand and address limitations in sub-4-bit scenarios of LoRA-based quantization error compensation (LQEC).
- RILQ employs model-wise activation discrepancy loss to adjust adapters cooperatively across layers, enabling robust error compensation with low-rank adapters.
- Evaluations on LLaMA-2 and LLaMA-3 demonstrate RILQ's consistent improvements in 2-bit quantized inference across various quantizers and enhanced accuracy in task-specific fine-tuning.
- RILQ enables adapter-merged weight-quantized Large Language Model (LLM) inference with significantly enhanced accuracy, making it a promising approach for boosting 2-bit LLM performance.
Read Full Article
10 Likes
Arxiv
1d
197
Image Credit: Arxiv
Multimodal Learning with Uncertainty Quantification based on Discounted Belief Fusion
- Multimodal Learning with Uncertainty Quantification based on Discounted Belief Fusion
- Multimodal AI models are used in various fields such as healthcare, finance, and autonomous driving.
- Managing uncertainty is important for reliable decision-making in multimodal learning.
- A novel multimodal learning method with order-invariant evidence fusion and conflict-based discounting is proposed.
Read Full Article
11 Likes
Arxiv
1d
112
Image Credit: Arxiv
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference
- DANCE is a proposed co-optimization approach for efficient segmentation model training and inference.
- Current segmentation models suffer from expensive computation due to high-resolution images and multi-scale aggregation.
- DANCE focuses on data-network co-optimization through input data manipulation and network architecture slimming.
- Experiments show that DANCE achieves reduced training cost, less expensive inference, and improved mean Intersection-over-Union (mIoU).
Read Full Article
6 Likes
Arxiv
1d
302
Image Credit: Arxiv
Policy Learning with Competing Agents
- Policy Learning with Competing Agents
- Decision makers often aim to learn a treatment assignment policy under a capacity constraint on the number of agents that they can treat.
- This paper studies capacity-constrained treatment assignment in the presence of strategic interference.
- The authors present a consistent estimator for policy gradient, allowing for learning capacity-constrained policies in the presence of strategic behavior.
Read Full Article
18 Likes
Arxiv
1d
213
Image Credit: Arxiv
Adversarially Robust Topological Inference
- Persistent homology is a key tool in topological data analysis, but it is highly sensitive to outliers.
- In response, researchers have developed a framework of statistical inference for persistent homology in the presence of outliers.
- They propose a extit{median-of-means} variant of the distance function and establish its statistical properties.
- Simulations and applications demonstrate the advantages of the proposed methodology.
Read Full Article
12 Likes
Arxiv
1d
7
Image Credit: Arxiv
USC: Uncompromising Spatial Constraints for Safety-Oriented 3D Object Detectors in Autonomous Driving
- This work focuses on the safety-oriented performance of 3D object detectors in autonomous driving contexts.
- The lack of safety-oriented metrics in these perception models makes it challenging to ensure safe deployment.
- The authors introduce uncompromising spatial constraints (USC) to demand predictions that fully cover objects from the vehicle's perspective and bird's-eye views.
- The USC constraints enable quantitative evaluation, improving model performance and providing a more direct link to system safety.
Read Full Article
Like
Arxiv
1d
364
Image Credit: Arxiv
Compress Then Test: Powerful Kernel Testing in Near-linear Time
- Kernel two-sample testing provides a powerful framework for distinguishing any pair of distributions based on $n$ sample points.
- Compress Then Test (CTT) is a new framework for high-powered kernel testing based on sample compression.
- CTT approximates an expensive test by compressing each $n$ point sample into a small but high-fidelity coreset.
- CTT and its extensions provide significant speed-ups over state-of-the-art approximate MMD tests with no loss of power.
Read Full Article
21 Likes
Arxiv
1d
372
Image Credit: Arxiv
Tomography of Quantum States from Structured Measurements via quantum-aware transformer
- Researchers propose a quantum-aware transformer (QAT) model for quantum state tomography.
- The QAT model leverages the intrinsic quantum characteristics involved in quantum state tomography.
- It captures the complex relationship between measured frequencies and density matrices.
- Extensive simulations and experiments demonstrate the QAT's superiority in reconstructing quantum states with robustness against experimental noise.
Read Full Article
22 Likes
Arxiv
1d
357
Image Credit: Arxiv
Personalized Privacy Amplification via Importance Sampling
- For scalable machine learning on large data sets, importance sampling is commonly used to subsample a representative subset for efficient model training.
- This paper examines the privacy properties of importance sampling, specifically focusing on individualized privacy analysis.
- The study finds that privacy in importance sampling is aligned with utility but conflicts with sample size.
- The paper proposes two approaches for constructing sampling distributions that optimize privacy-efficiency trade-off and provide utility guarantees through coresets.
Read Full Article
21 Likes
Arxiv
1d
97
Image Credit: Arxiv
High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curve
- Variational autoencoders (VAEs) often experience posterior collapse, leading to poor representation learning quality.
- An adjustable hyperparameter beta has been introduced in VAEs to address posterior collapse.
- This study examines the conditions under which posterior collapse occurs, as determined by beta and dataset size.
- The findings reveal the inevitable posterior collapse beyond a certain beta threshold, regardless of dataset size, and the dataset size dependence of the rate-distortion curve in VAEs.
Read Full Article
5 Likes
Arxiv
1d
318
Image Credit: Arxiv
Manifold learning in Wasserstein space
- This paper discusses the theoretical foundations for manifold learning algorithms in the space of absolutely continuous probability measures.
- The focus is on building submanifolds in the absolute continuous probability measure space, using the Wasserstein-2 distance as the metric.
- These submanifolds allow for local linearizations, similar to Riemannian submanifolds of Euclidean space.
- The paper also presents methods for learning the latent manifold structure and recovering tangent spaces using pairwise extrinsic Wasserstein distances and spectral analysis.
Read Full Article
19 Likes
Arxiv
1d
279
Image Credit: Arxiv
Dynamics-Guided Diffusion Model for Sensor-less Robot Manipulator Design
- A data-driven framework called Dynamics-Guided Diffusion Model (DGDM) has been developed for generating task-specific manipulator designs without task-specific training.
- DGDM generates sensor-less manipulator designs that can blindly manipulate objects towards desired motions and poses using an open-loop parallel motion.
- The framework represents manipulation tasks as interaction profiles and the design space using a geometric diffusion model.
- DGDM outperforms optimization-based and unguided diffusion baselines in terms of the average success rate.
Read Full Article
16 Likes
Arxiv
1d
124
Image Credit: Arxiv
Approximate Nullspace Augmented Finetuning for Robust Vision Transformers
- Researchers propose an approach to enhance the robustness of vision transformers (ViTs) inspired by the concept of nullspace from linear algebra.
- The investigation focuses on whether ViTs can exhibit resilience to input variations akin to the nullspace property in linear mappings.
- The researchers extend the notion of nullspace to nonlinear settings and demonstrate the synthesis of approximate nullspace elements for ViT's encoder blocks through optimization.
- A finetuning strategy is proposed for ViTs by augmenting the training data with synthesized approximate nullspace noise, leading to improved robustness.
Read Full Article
7 Likes
Arxiv
1d
271
Image Credit: Arxiv
Convergence analysis of controlled particle systems arising in deep learning: from finite to infinite sample size
- This paper studies the convergence analysis of a class of neural stochastic differential equations (SDEs) and their associated optimal control problems.
- The focus is on understanding the limiting behavior of the sampled optimal control problems as the sample size grows to infinity.
- The paper analyzes the N-particle systems with centralized control and establishes uniform regularity estimates by solving the Hamilton-Jacobi-Bellman equation.
- The research shows the convergence of both the objective functionals and optimal parameters of the neural SDEs as the sample size tends to infinity, with the limiting objects identified as suitable functions defined on the Wasserstein space of Borel probability measures.
Read Full Article
16 Likes
Arxiv
1d
201
Image Credit: Arxiv
Asymptotic Unbiased Sample Sampling to Speed Up Sharpness-Aware Minimization
- Researchers propose Asymptotic Unbiased Sampling with respect to iterations to accelerate Sharpness-Aware Minimization (SAM).
- AUSAM probabilistically samples a subset of data points based on the Gradient Norm of each Sample (GNS).
- The method consistently accelerates SAM across various tasks and networks, achieving a speedup of over 70%.
- AUSAM is better suited for SAM and excels in maintaining performance compared to recent dynamic data pruning methods.
Read Full Article
12 Likes
For uninterrupted reading, download the app