Best Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models alternatives.
Live source-backed alternatives to Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models for Vision-language. Alternatives are selected from the same task category and update whenever the best-of index rebuilds.
Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models
Recently, self-evolving large multimodal models (LMMs) have received attention for improving visual reasoning in a purely unsupervised setting. However, multi-role self-play and self-consistency reward schemes in existing self-evolving LMMs optimize answer agreement without ensuring the decoder attends to visual content, relying instead on statistical language priors to produce self consistent outputs. This leads to a persistent failure mode we term visual under-conditioning, where the decoder relies on language priors rather than the image during generation, manifesting as insufficient attention to visual tokens. As a result, current self-evolving LMMs struggle on vision--language understanding tasks such as image captioning and visual question answering. To address this, we propose VISE (Visual Invariance Self-Evolution), a purely unsupervised self-evolving framework that directly regularizes the model's visual conditioning policy through two complementary invariance-based rewards: a geometric invariance reward that enforces spatial consistency under known transformations, and a semantic invariance reward that penalizes evidence-agnostic generation by requiring the model to recognize the absence of evidence when predicted regions are perturbed. VISE operates within a single model without specialist roles, external reward models, or annotations, and is trained on raw unlabeled images. Experiments on 18 benchmarks demonstrate the efficacy of our approach. Using Qwen3-VL-2B as the base model, VISE achieves gains of $+16.85$ CIDEr on COCO and $+19.66$ CIDEr on TextCaps, reduces object hallucination by $5.0$ Chair-I points, and generalizes across four model families and scales. Our code and models are available at https://mbzuai-oryx.github.io/VISE cs.CV Recently, self-evolving large multimodal models (LMMs) have received attention for improving visual reasoning in a purely unsupervised setting. However, multi-role self-play and self-consistency reward schemes in existing self-evolving LMMs optimize answer agreement without ensuring the decoder attends to visual content, relying instead on statistical language priors to produce self consistent outputs. This leads to a persistent failure mode we term visual under-conditioning, where the decoder relies on language priors rather than the image during generation, manifesting as insufficient attention to visual tokens. As a result, current self-evolving LMMs struggle on vision--language understanding tasks such as image captioning and visual question answering. To address this, we propose VISE (Visual Invariance Self-Evolution), a purely unsupervised self-evolving framework that directly regularizes the model's visual conditioning policy through two complementary invariance-based rewards: a geometric invariance reward that enforces spatial consistency under known transformations, and a semantic invariance reward that penalizes evidence-agnostic generation by requiring the model to recognize the absence of evidence when predicted regions are perturbed. VISE operates within a single model without specialist roles, external reward models, or annotations, and is trained on raw unlabeled images. Experiments on 18 benchmarks demonstrate the efficacy of our approach. Using Qwen3-VL-2B as the base model, VISE achieves gains of $+16.85$ CIDEr on COCO and $+19.66$ CIDEr on TextCaps, reduces object hallucination by $5.0$ Chair-I points, and generalizes across four model families and scales. Our code and models are available at https://mbzuai-oryx.github.io/VISE Research signal collected from arXiv metadata; Gemini enrichment can add a clearer summary. cs.CV benchmark
NVIDIA NIM Model Catalog
Matched vision-language, vision language, multimodal; 3 source links; official inference catalog signal; access model: Free endpoint
Hugging Face Inference Providers
Matched vision-language, vision language, multimodal; 2 source links; official inference catalog signal; access model: Paid API
| # | Alternative | Kind | Access | Fit | Why it appears | Source |
|---|---|---|---|---|---|---|
| 01 | NVIDIA NIM Model Catalog | service | Free endpoint | RDR84 | Matched vision-language, vision language, multimodal; 3 source links; official inference catalog signal; access model: Free endpoint | build.nvidia.com |
| 02 | Hugging Face Inference Providers | service | Paid API | RDR81 | Matched vision-language, vision language, multimodal; 2 source links; official inference catalog signal; access model: Paid API | huggingface.co |
| 03 | A Unified Framework for Efficient Remote Sensing Visual Question Answering: Adapting Dual, Hybrid, and Encoder-Decoder Architectures | paper | Research-only | RDR80 | Matched vision-language, vision language, vlm; 1 source link; access model: Research-only | arxiv.org |
| 04 | Fireworks AI Serverless Models | service | Paid API | RDR80 | Matched vision-language, vision language, multimodal; 2 source links; official inference catalog signal; access model: Paid API | docs.fireworks.ai |
| 05 | Together AI Serverless Models | service | Paid API | RDR80 | Matched vision-language, vision language, multimodal; 2 source links; official inference catalog signal; access model: Paid API | docs.together.ai |
| 06 | SARLO-80: Worldwide Slant SAR Language Optic Dataset 80cm | paper | Research-only | RDR75 | Matched vision-language, vision language, multimodal; 2 source links; access model: Research-only | arxiv.org |
| 07 | RSICCLLM: A Multimodal Large Language Model for Remote Sensing Image Change Captioning | paper | Research-only | RDR75 | Matched vision-language, vision language, multimodal; 2 source links; access model: Research-only; freshly updated | arxiv.org |