A naukri.com initiative
ML News
Arxiv
1d
315
Image Credit: Arxiv
Rethinking Optimization and Architecture for Tiny Language Models
- The application of language models on mobile devices is facing a challenge in terms of computation and memory costs.
- A study has analyzed the effect of various components on tiny language models with 1B parameters.
- Optimization strategies such as tokenizer compression, architecture tweaking, and parameter inheritance have shown effective results.
- Experimental results demonstrate improved optimization and architecture of tiny language models, yielding notable performance improvements.
Read Full Article
18 Likes
Arxiv
1d
82
Image Credit: Arxiv
Decentralized Bilevel Optimization: A Perspective from Transient Iteration Complexity
- Stochastic bilevel optimization (SBO) is increasingly important in machine learning and nested structures.
- Decentralized approaches, like D-SOBA, improve communication efficiency and algorithmic robustness.
- D-SOBA framework has two variants: D-SOBA-SO with second-order matrices, and D-SOBA-FO with first-order gradients.
- Comprehensive non-asymptotic convergence analysis of D-SOBA reveals the impact of network topology and data heterogeneity.
Read Full Article
4 Likes
Arxiv
1d
319
Image Credit: Arxiv
Distributed Fractional Bayesian Learning for Adaptive Optimization
- This paper introduces a distributed adaptive optimization problem in which agents collaboratively estimate an unknown parameter while finding the optimal solution.
- The proposed Prediction while Optimization scheme utilizes distributed fractional Bayesian learning and distributed gradient descent.
- Under suitable assumptions, the paper proves the convergence of agents' beliefs and decision variables towards the true parameter and optimal solution.
- Numerical experiments are conducted to validate the theoretical analysis.
Read Full Article
19 Likes
Arxiv
1d
349
Image Credit: Arxiv
Modeling Caption Diversity in Contrastive Vision-Language Pretraining
- Llip, Latent Language Image Pretraining, is introduced to model the diversity of captions that could match an image.
- Llip's vision encoder outputs a set of visual features that are mixed into a final representation by conditioning on information derived from the text.
- Llip outperforms non-contextualized baselines like CLIP and SigLIP on various tasks, including zero-shot classification and retrieval.
- Llip achieves a zero-shot top-1 accuracy of 83.5% on ImageNet, outperforming similarly sized CLIP by 1.4%.
Read Full Article
21 Likes
Arxiv
1d
3
Image Credit: Arxiv
Configurable Holography: Towards Display and Scene Adaptation
- Researchers have developed a highly configurable learned model structure for synthesizing 3D holograms.
- The models can be conditioned for varying display-scene parameters, including input images, propagation distances, volume depths, and more.
- The correlation between depth estimation and hologram synthesis tasks has led to an accurate model for generating 3D holograms from 2D images.
- The models have been validated through simulations and two different holographic display prototypes.
Read Full Article
Like
Arxiv
1d
210
Image Credit: Arxiv
A distance for mixed-variable and hierarchical domains with meta variables
- A modeling framework is introduced to generalize hierarchical and mixed-variable domains with meta variables.
- The framework allows comparison of mixed-variable points that don't share the same variables.
- The methodology is applied to regression and classification experiments using distance-based models.
- Experiments utilize datasets of hyperparameters and performance scores.
Read Full Article
12 Likes
Arxiv
1d
26
Image Credit: Arxiv
Lusifer: LLM-based User SImulated Feedback Environment for online Recommender systems
- Lusifer is an LLM-based simulation environment designed to generate dynamic, realistic user feedback for RL-based recommender training.
- Lusifer updates user profiles at each interaction step using Large Language Models (LLMs) and provides transparent explanations of how and why preferences evolve.
- By processing textual metadata, Lusifer creates context-aware user states and simulates feedback on new items, reducing reliance on extensive historical data and facilitating adaptation to out of distribution cases.
- Lusifer excels in capturing dynamic user responses and yielding explainable results, making it a scalable and ethically sound alternative to live user experiments in RL-based recommender systems.
Read Full Article
1 Like
Arxiv
1d
322
Image Credit: Arxiv
Is Algorithmic Stability Testable? A Unified Framework under Computational Constraints
- Algorithmic stability is a central notion in learning theory that quantifies the sensitivity of an algorithm to small changes in the training data.
- Recent results establish that testing the stability of a black-box algorithm is impossible, given limited data from an unknown distribution.
- This work examines the hardness of testing algorithmic stability in a broad range of settings, including categorical data.
- The study finds that if the available data is limited, exhaustive search is essentially the only universally valid mechanism for certifying algorithmic stability, implying fundamental limits on stability testing.
Read Full Article
19 Likes
Arxiv
1d
161
Image Credit: Arxiv
Boost Your Human Image Generation Model via Direct Preference Optimization
- Human image generation is a key focus in image synthesis with broad applications.
- Direct Preference Optimization (DPO) is explored to improve the realism of generated images.
- An enhanced DPO approach is proposed, incorporating high-quality real images as winning images.
- The HG-DPO approach employs a curriculum learning framework for gradual improvements and personalized text-to-image tasks.
Read Full Article
9 Likes
Arxiv
1d
82
Image Credit: Arxiv
LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs
- Large language models (LLMs) show impressive capabilities in mathematical reasoning.
- A new benchmark called Mathematical Topics Tree (MaTT) is introduced to evaluate LLMs on comprehensive mathematical subjects.
- GPT-4, the most advanced LLM, achieved only 54% accuracy in the multiple-choice scenario of the MaTT benchmark.
- LLMs' performance varied significantly across different mathematical topics, and their explanations were deemed incomplete or inaccurate in many instances.
Read Full Article
4 Likes
Arxiv
1d
334
Image Credit: Arxiv
VELOCITI: Benchmarking Video-Language Compositional Reasoning with Strict Entailment
- VELOCITI is a benchmark created to study Video-LLMs and assess compositional reasoning in short videos.
- It disentangles and evaluates the comprehension of agents, actions, and their associations across multiple events.
- Current video models like LLaVA-OneVision and Gemini-1.5-Pro perform far from human accuracy in classifying positive and negative captions.
- The benchmark highlights challenges with ClassicVLE and multiple-choice evaluation, emphasizing the preference for StrictVLE.
Read Full Article
20 Likes
Arxiv
1d
105
Image Credit: Arxiv
Coupled Input-Output Dimension Reduction: Application to Goal-oriented Bayesian Experimental Design and Global Sensitivity Analysis
- A new method for joint dimension reduction of input and output spaces of a function is introduced.
- Conventional methods focus on reducing either the input or output space, while this coupled approach supports simultaneous reduction of both.
- The method is suitable for goal-oriented dimension reduction, where input or output quantities of interest are prescribed.
- Applications include goal-oriented sensor placement and goal-oriented sensitivity analysis, solving combinatorial optimization problems by optimizing gradient-based bounds.
Read Full Article
6 Likes
Arxiv
1d
296
Image Credit: Arxiv
Cascade Reward Sampling for Efficient Decoding-Time Alignment
- Cascade Reward Sampling (CARDS) is introduced to address efficiency bottlenecks in decoding-time alignment of large language models (LLMs).
- CARDS utilizes a segment-level rejection sampling algorithm to minimize redundant computations of LLMs and reward models (RMs).
- An uncertainty-based segmentation mechanism ensures accurate evaluation of RMs on incomplete segments.
- Experimental results demonstrate that CARDS significantly improves decoding efficiency, alignment quality, and general utility.
Read Full Article
17 Likes
Arxiv
1d
303
Image Credit: Arxiv
ShapG: new feature importance method based on the Shapley value
- A new Explainable Artificial Intelligence (XAI) method called ShapG (Explanations based on Shapley value for Graphs) has been developed for measuring feature importance.
- ShapG is a model-agnostic global explanation method that defines an undirected graph based on the dataset and calculates feature importance using an approximated Shapley value.
- Comparisons with existing XAI methods demonstrate that ShapG provides more accurate explanations and exhibits advantages in terms of computational efficiency.
- The ShapG method has wide applicability and can improve the explainability and transparency of AI systems in various fields.
Read Full Article
18 Likes
Arxiv
1d
146
Image Credit: Arxiv
PQCache: Product Quantization-based KVCache for Long Context LLM Inference
- A new method called PQCache is proposed to address the memory bottleneck in Large Language Models (LLMs) inference.
- PQCache employs Product Quantization (PQ) to manage the Key-Value Cache (KVCache) in LLMs, maintaining model quality while ensuring low serving latency.
- PQCache applies PQ to tokens' keys during the prefilling phase and uses PQ codes and centroids to fetch key-value pairs during the autoregressive decoding phase.
- Extensive experiments show that PQCache achieves improved model effectiveness and efficiency, with a 4.60% score improvement over existing methods.
Read Full Article
8 Likes
For uninterrupted reading, download the app