pith. sign in

arxiv: 2403.13372 · v4 · submitted 2024-03-20 · 💻 cs.CL · cs.AI

LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models

Pith reviewed 2026-05-11 12:35 UTC · model grok-4.3

classification 💻 cs.CL cs.AI
keywords fine-tuninglarge language modelsunified frameworkefficient trainingweb interfacelanguage modelingtext generationmodel adaptation
0
0 comments X

The pith

A unified framework lets users fine-tune over 100 language models efficiently using only a web interface and no code.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents a framework that combines multiple efficient fine-tuning techniques into one system for adapting large language models to new tasks. The system includes a web interface that lets users adjust settings for many different models without writing any code. Such a tool addresses the problem that each model usually needs its own implementation of training methods, which takes expert effort. Experiments confirm the framework works for basic language tasks and text generation.

Core claim

The framework integrates a range of efficient training methods to support the fine-tuning of more than one hundred language models in a flexible way. Customization happens entirely through the accompanying web user interface, removing any requirement for coding. Validation experiments on language modeling and text generation tasks establish both the efficiency and the effectiveness of this approach.

What carries the argument

The unified framework that merges efficient training methods with a web-based interface to manage fine-tuning across many models.

Load-bearing premise

That the efficient methods integrate without conflicts or performance drops when applied uniformly to many different language models.

What would settle it

A demonstration that fine-tuning performance or speed for some models falls below what direct per-model implementations achieve.

read the original abstract

Efficient fine-tuning is vital for adapting large language models (LLMs) to downstream tasks. However, it requires non-trivial efforts to implement these methods on different models. We present LlamaFactory, a unified framework that integrates a suite of cutting-edge efficient training methods. It provides a solution for flexibly customizing the fine-tuning of 100+ LLMs without the need for coding through the built-in web UI LlamaBoard. We empirically validate the efficiency and effectiveness of our framework on language modeling and text generation tasks. It has been released at https://github.com/hiyouga/LLaMA-Factory and received over 25,000 stars and 3,000 forks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript presents LlamaFactory, a unified open-source framework integrating a suite of efficient fine-tuning methods for over 100 language models. It features a web-based UI (LlamaBoard) enabling no-code customization of fine-tuning workflows. The authors state that they empirically validate the framework's efficiency and effectiveness on language modeling and text generation tasks, and report its public release on GitHub with over 25,000 stars and 3,000 forks.

Significance. If the integration claims hold, the work provides a practical, accessible tool that reduces implementation barriers for efficient LLM adaptation across many architectures. The high GitHub adoption offers evidence of real-world utility and community value. The open release of the artifact itself constitutes a reproducible contribution that can support further research in NLP fine-tuning.

major comments (1)
  1. Abstract: the claim of empirical validation on language modeling and text generation tasks is not accompanied by any metrics, baselines, or quantitative results. This detail is load-bearing for the effectiveness and efficiency assertions and should be expanded with concrete numbers and comparisons.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback and positive recommendation. We address the single major comment below.

read point-by-point responses
  1. Referee: [—] Abstract: the claim of empirical validation on language modeling and text generation tasks is not accompanied by any metrics, baselines, or quantitative results. This detail is load-bearing for the effectiveness and efficiency assertions and should be expanded with concrete numbers and comparisons.

    Authors: We agree that the abstract would benefit from greater specificity to support the stated claims. The full manuscript (Section 4) contains the detailed experiments, including quantitative results on language modeling (e.g., perplexity) and text generation tasks with comparisons to baselines. We will revise the abstract to incorporate a concise summary of key metrics and efficiency gains, making the validation claims more concrete while preserving the abstract's brevity. revision: yes

Circularity Check

0 steps flagged

No circularity: framework release with external verifiability

full rationale

The paper presents LlamaFactory as an open-source software artifact integrating existing efficient fine-tuning methods (LoRA, QLoRA, etc.) for 100+ LLMs, with a no-code web UI. No mathematical derivations, fitted parameters, predictions, or uniqueness theorems are claimed. The central contribution is the released codebase (GitHub link provided, with reported stars/forks as external evidence of adoption). Empirical validation is described at a high level on standard tasks but does not involve any internal reduction to self-defined inputs or self-citations that bear the load of a derivation. The work is self-contained as an engineering deliverable whose functionality is directly testable outside the paper.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is an applied software engineering paper; no free parameters are fitted to data, no additional axioms beyond standard computing are invoked, and no new scientific entities are postulated. The contribution rests on engineering integration and release.

pith-pipeline@v0.9.0 · 5427 in / 1146 out tokens · 89645 ms · 2026-05-11T12:35:12.761726+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SARL: Label-Free Reinforcement Learning by Rewarding Reasoning Topology

    cs.AI 2026-03 conditional novelty 8.0

    SARL rewards reasoning topology to improve label-free RL, outperforming baselines with gains up to 44.7% on math and 34.6% on open-ended tasks while maintaining more stable training.

  2. FLEX: A Largescale Multimodal, Multiview Dataset for Learning Structured Representations for Fitness Action Quality Assessment

    cs.CV 2025-06 conditional novelty 8.0

    FLEX is the first large-scale multimodal multiview dataset for fitness AQA, featuring RGB, 3D pose, sEMG and physiological data plus a Fitness Knowledge Graph for structured annotations and a VideoQA benchmark.

  3. CASS: Nvidia to AMD Transpilation with Data, Models, and Benchmark

    cs.AR 2025-05 unverdicted novelty 8.0

    CASS supplies the first large verified dataset and trained models for cross-vendor GPU code translation between CUDA and HIP at source and assembly levels, with reported accuracies of 88.2% and 69.1% that beat commerc...

  4. When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizon

    cs.LG 2026-06 unverdicted novelty 7.0

    Online IL overcomes an information-theoretic bottleneck that offline IL faces in non-realizable settings even at horizon 1, under a new structural characterization of reward-relative misspecification.

  5. Scaling LLM Knowledge Boundaries via Distribution-Optimized Synthesis

    cs.CL 2026-06 unverdicted novelty 7.0

    KDoS optimizes synthetic data distributions via a three-stage knowledge density feedback mechanism to maximize LLM knowledge boundary expansion, with the optimal distribution stable across model sizes and data scales.

  6. RedAct: Redacting Agent Capability Traces for Procedural Skill Protection

    cs.CR 2026-06 unverdicted novelty 7.0

    RedAct redacts agent traces to drop normalized skill transfer below the no-skill baseline on CapTraceBench while preserving audit evidence and adding detectable behavioral watermarks.

  7. VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understanding

    cs.CV 2026-06 unverdicted novelty 7.0

    VideoKR supplies 315K knowledge-intensive video reasoning examples and a dedicated benchmark, with experiments indicating post-training gains on reasoning tasks that require both video content and external knowledge.

  8. RogueMerge: Robust and Unified Attacks against LLM Model Merging

    cs.CR 2026-06 unverdicted novelty 7.0

    RogueMerge is a unified attack method that jointly optimizes task vectors to succeed after merging, using stochastic min-max simulation for unknown merging settings and a Taylor-approximated DRO for prompt generalizat...

  9. Synthetic Hallucinations, Real Gains: Hard Negatives from Frontier Models for FIM Hallucination Mitigation

    cs.LG 2026-06 unverdicted novelty 7.0

    Using frontier models to synthesize plausible-but-wrong FIM completions as hard negatives for SFT improves Delulu exact match by +18.8 and edit similarity by +0.22 on Qwen2.5-Coder-7B while also lifting HumanEval-Infi...

  10. Pause and Think: A Dataset and Benchmark for Video-Grounded Assistive Action Suggestion

    cs.CV 2026-05 unverdicted novelty 7.0

    Introduces pause-and-think-T dataset and pause-and-think-B benchmark; fine-tunes 4B VLM to 58% accuracy matching 235B model while generalizing out-of-distribution.

  11. GGBound: A Genome-Grounded Agent for Microbial Life-Boundary Prediction

    cs.CY 2026-05 unverdicted novelty 7.0

    A genome-conditioned 4B LLM agent predicts microbial life boundaries and matches larger frontier models via token fusion, tool use, and a counterfactual gene-grounding reward.

  12. Teaching Language Models to Think in Code

    cs.CL 2026-05 unverdicted novelty 7.0

    ThinC trains small models to reason primarily in code rather than natural language, outperforming tool-integrated baselines and even larger models on competition math benchmarks.

  13. DiagramNet: An End-to-End Recognition Framework and Dataset for Non-Standard System-Level Diagrams

    cs.AI 2026-05 unverdicted novelty 7.0

    DiagramNet supplies a new multimodal dataset and progressive training pipeline with decoupled multi-agent workflow, allowing a 3B model to outperform GPT-5, Claude-Sonnet-4, and Gemini-2.5-Pro by over 2x on system-lev...

  14. World2Minecraft: Occupancy-Driven Simulated Scenes Construction

    cs.CV 2026-04 unverdicted novelty 7.0

    World2Minecraft turns real scenes into Minecraft worlds via occupancy prediction and releases a large indoor occupancy dataset to improve such models.

  15. BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering

    cs.CL 2026-04 unverdicted novelty 7.0

    BERAG applies Bayesian ensemble weighting of individual documents via token-by-token posterior updates in retrieval-augmented generation, yielding gains on knowledge-based visual QA tasks.

  16. EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training

    cs.CV 2026-04 unverdicted novelty 7.0

    EmbodiedMidtrain mid-trains VLMs on curated VLA-aligned data subsets to improve downstream performance on robot manipulation benchmarks.

  17. S-GRPO: Unified Post-Training for Large Vision-Language Models

    cs.LG 2026-04 unverdicted novelty 7.0

    S-GRPO unifies SFT and RL for LVLMs via conditional ground-truth injection that supplies a maximal-reward anchor when group exploration fails completely.

  18. C-Mining: Unsupervised Discovery of Seeds for Cultural Data Synthesis via Geometric Misalignment

    cs.CL 2026-04 unverdicted novelty 7.0

    C-Mining automatically mines high-fidelity Culture Points from raw multilingual text by treating cross-lingual geometric isolation in embeddings as a quantifiable signal for cultural specificity, then uses them to syn...

  19. RLSpoofer: A Lightweight Evaluator for LLM Watermark Spoofing Resilience

    cs.CR 2026-04 unverdicted novelty 7.0

    RLSpoofer trains a 4B model on 100 watermarked paraphrase pairs to spoof PF watermarks at 62% success rate, far exceeding baselines trained on up to 10,000 samples.

  20. DeonticBench: A Benchmark for Reasoning over Rules

    cs.CL 2026-04 unverdicted novelty 7.0

    DEONTICBENCH is a new benchmark of 6,232 deontic reasoning tasks from U.S. legal domains where frontier LLMs reach only ~45% accuracy and symbolic Prolog assistance plus RL training still fail to solve tasks reliably.

  21. ChatSVA: Bridging SVA Generation for Hardware Verification via Task-Specific LLMs

    cs.AR 2026-04 unverdicted novelty 7.0

    ChatSVA achieves 96.12% functional pass rate and 82.5% coverage in SVA generation on 24 RTL designs, delivering 33 percentage point gains and 11x better coverage than prior state-of-the-art.

  22. PR-CAD: Progressive Refinement for Unified Controllable and Faithful Text-to-CAD Generation with Large Language Models

    cs.CL 2026-03 unverdicted novelty 7.0

    PR-CAD unifies text-to-CAD generation and editing via progressive refinement with LLMs, a new interaction dataset, and RL-enhanced reasoning to achieve better controllability and faithfulness.

  23. Scaling Latent Reasoning via Looped Language Models

    cs.CL 2025-10 unverdicted novelty 7.0

    Looped language models with latent iterative computation and entropy-regularized depth allocation achieve performance matching up to 12B standard LLMs through superior knowledge manipulation.

  24. MMSearch-R1: Incentivizing LMMs to Search

    cs.CV 2025-06 unverdicted novelty 7.0

    MMSearch-R1 uses reinforcement learning to train multimodal models for on-demand multi-turn internet search with image and text tools, outperforming same-size RAG baselines and matching larger ones while cutting searc...

  25. PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehunts

    cs.CL 2025-06 conditional novelty 7.0

    PuzzleWorld benchmark reveals state-of-the-art AI models solve only 18% of complex puzzlehunt problems with 40% stepwise accuracy, matching novices but trailing enthusiasts, while fine-tuning on traces yields modest gains.

  26. ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering

    cs.CV 2026-06 unverdicted novelty 6.0

    ProMSA is a progressive multimodal search agent for KB-VQA that iteratively selects search tools under budgets, trained via rejection-sampling SFT then TN-GSPO RL, reporting gains on E-VQA and InfoSeek over RAG baselines.

  27. Empowering GUI Agents via Autonomous Experience Exploration and Hindsight Experience Utilization for Task Planning

    cs.CL 2026-06 unverdicted novelty 6.0

    PEEU enables a 7B MLLM to reach 30.6% accuracy on GUI task planning by autonomous exploration and hindsight experience synthesis, outperforming a 32B model through stronger high-level OOD generalization.

  28. AutoBG: A Board Game Design Assistant with Interactive Ideation, Iterative Rulebook Generation, and Individualized Feedback

    cs.HC 2026-06 unverdicted novelty 6.0

    AutoBG is a multi-module AI assistant that uses critic-driven iterative refinement on LLM-generated rulebooks, trained on 2.2K rulebooks and 180K reviews, to produce audience-tested designs that outperform GPT-5.4 baselines.

  29. JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions

    cs.SD 2026-06 unverdicted novelty 6.0

    JenBridge pretrains a flow-matching Transformer on text-audio data then adapts it with video conditioning and an LLM director to select transitions, claiming better coherence than prior methods on a new LVS benchmark.

  30. Pause and Think: A Dataset and Benchmark for Video-Grounded Assistive Action Suggestion

    cs.CV 2026-05 unverdicted novelty 6.0

    Introduces pause-and-think-T dataset and pause-and-think-B benchmark for video-grounded assistive action suggestion, enabling a 4B VLM to match larger models on reasoning tasks and generalize to EgoThink and TempCompass.

  31. Enhancing the Code Reasoning Capabilities of LLMs via Consistency-based Reinforcement Learning

    cs.LG 2026-05 unverdicted novelty 6.0

    CodeThinker improves LLM code reasoning via consistency-based RL with stepwise training data, dynamic beam sampling, and consistency rewards, reaching SOTA on benchmarks with 4.3% gains on Qwen2.5-Coder-7B.

  32. Contextualized Code Pretraining for Code Generation

    cs.SE 2026-05 unverdicted novelty 6.0

    Introduces contextualized code pretraining with caller-callee pairs from static analysis to train CallerGen models that outperform baselines on the new CallerEval benchmark.

  33. GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents

    cs.LG 2026-05 conditional novelty 6.0

    GROW decomposes trajectories into state-action samples for GRPO training of VLM agents and reports state-of-the-art results on over 800 Minecraft tasks.

  34. Chain-of-Procedure: Hierarchical Visual-Language Reasoning for Procedural QA

    cs.CL 2026-05 unverdicted novelty 6.0

    Introduces ProcedureVQA benchmark and Chain-of-Procedure framework that improves VLM next-step prediction in procedures by up to 13% over baselines.

  35. Speculative Interaction Agents: Building Real-Time Agents with Asynchronous I/O and Speculative Tool Calling

    cs.LG 2026-05 unverdicted novelty 6.0

    Asynchronous I/O and Speculative Tool Calling cut latency in tool-calling LLM agents by 1.3-2.2x with only minor accuracy loss on cloud and edge models.

  36. Speculative Interaction Agents: Building Real-Time Agents with Asynchronous I/O and Speculative Tool Calling

    cs.LG 2026-05 unverdicted novelty 6.0

    Speculative Interaction Agents achieve 1.3-2.2x speedups for real-time tool-calling agents via async I/O decoupling and speculative calls, with clock-based training for small edge models.

  37. Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning

    cs.LG 2026-05 unverdicted novelty 6.0

    For a fixed data budget in LLM supervised fine-tuning, optimal data difficulty shifts toward harder examples as the budget grows because of the tradeoff between in-distribution generalization gap and extrapolation gap.

  38. Teaching Language Models to Think in Code

    cs.CL 2026-05 unverdicted novelty 6.0

    ThinC trains smaller language models to reason entirely in code after minimal NL planning, outperforming tool-integrated baselines and even much larger models on competition math benchmarks.

  39. Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less

    cs.LG 2026-05 unverdicted novelty 6.0

    Full finetuning with the pretraining optimizer reduces forgetting compared to other optimizers or LoRA while achieving comparable new-task performance.

  40. Revealing Modular Gradient Noise Imbalance in LLMs: Calibrating Adam via Signal-to-Noise Ratio

    cs.LG 2026-05 unverdicted novelty 6.0

    MoLS scales Adam updates using module-level SNR estimates to correct gradient noise imbalance and improve LLM training convergence and generalization.

  41. Estimating the Black-box LLM Uncertainty with Distribution-Aligned Adversarial Distillation

    cs.CL 2026-05 unverdicted novelty 6.0

    DisAAD trains a 1%-sized proxy model via adversarial distillation to quantify uncertainty in black-box LLMs by aligning with their output distributions.

  42. ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting

    cs.AI 2026-05 unverdicted novelty 6.0

    ScrapMem introduces optical forgetting to compress multimodal memories for LLM agents on edge devices, cutting storage by up to 93% while reaching 51.0% Joint@10 and 70.3% Recall@10 on ATM-Bench.

  43. ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting

    cs.AI 2026-05 unverdicted novelty 6.0

    ScrapMem reports SOTA 51.0% Joint@10 on ATM-Bench with up to 93% memory reduction and 70.3% Recall@10 via optical forgetting and EM-Graph.

  44. Learn-To-Learn on Arbitrary Textual Conditioning: A Hypernetwork-Driven Meta-Gated LLM

    cs.CL 2026-05 unverdicted novelty 6.0

    A hypernetwork generates meta-gating parameters for SwiGLU blocks to let LLMs adapt their nonlinearity to arbitrary textual conditions, outperforming finetuning and meta-learning baselines with reasonable generalizati...

  45. When Model Editing Meets Service Evolution: A Knowledge-Update Perspective for Service Recommendation

    cs.SE 2026-04 unverdicted novelty 6.0

    EVOREC integrates locate-then-edit model editing with FA-constrained decoding to improve LLM-based service recommendation under evolution, reporting 25.9% average relative gain in Recall@5 over baselines and 22.3% ove...

  46. MCMit: Mid-Circuit Measurement Error Mitigation

    quant-ph 2026-04 unverdicted novelty 6.0

    MCMit proposes a constant-latency multi-control branch instruction, transformer and CNN discriminators, plus static MCM elimination and stochastic branching, evaluated on Qubic with QPU traces to cut latency by 70% an...

  47. SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring

    cs.CV 2026-04 unverdicted novelty 6.0

    SIEVES improves selective prediction coverage up to 3x on OOD VQA benchmarks by training a selector on visual localization quality, generalizing across datasets and proprietary reasoners without specific adaptation.

  48. SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring

    cs.CV 2026-04 conditional novelty 6.0

    SIEVES improves selective prediction coverage by up to 3x on OOD VQA benchmarks by training a selector to score the quality of visual evidence produced by reasoner models, generalizing across benchmarks and proprietar...

  49. Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA

    cs.IR 2026-04 unverdicted novelty 6.0

    Rabtriever distills a generative reranker into an efficient independent encoder using JEPA and auxiliary reverse KL loss to achieve linear complexity and strong performance on rationale-based retrieval tasks.

  50. Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA

    cs.IR 2026-04 unverdicted novelty 6.0

    Rabtriever distills a generative reranker into an efficient bi-encoder using on-policy JEPA to achieve near-reranker accuracy with linear complexity on rationale-based retrieval.

  51. CoDA: Towards Effective Cross-domain Knowledge Transfer via CoT-guided Domain Adaptation

    cs.AI 2026-04 unverdicted novelty 6.0

    CoDA aligns cross-domain latent reasoning representations in LLMs via CoT distillation and MMD to enable effective knowledge transfer without in-domain demonstrations.

  52. CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corpora

    cs.SE 2026-04 unverdicted novelty 6.0

    CodePivot uses Python as a pivot language plus an Aggressive-Partial-Functional RL reward to train a 7B model that outperforms much larger LLMs on multilingual code transpilation without parallel corpora.

  53. Characterizing Model-Native Skills

    cs.AI 2026-04 conditional novelty 6.0

    Recovering an orthogonal basis from model activations yields a model-native skill characterization that improves reasoning Pass@1 by up to 41% via targeted data selection and supports inference steering, outperforming...

  54. Chain-of-Glimpse: Search-Guided Progressive Object-Grounded Reasoning for Video Understanding

    cs.CV 2026-04 unverdicted novelty 6.0

    Chain-of-Glimpse is a reinforcement learning framework that builds progressive, spatially grounded reasoning traces around task-relevant objects in videos to enable more accurate and interpretable multi-step decisions.

  55. Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe

    cs.LG 2026-04 unverdicted novelty 6.0

    On-policy distillation works when student and teacher models share thinking patterns and the teacher adds new capabilities, with success tied to alignment on a small set of high-probability tokens.

  56. Pioneer Agent: Continual Improvement of Small Language Models in Production

    cs.AI 2026-04 unverdicted novelty 6.0

    Pioneer Agent automates the full lifecycle of adapting and continually improving small language models via diagnosis-driven data synthesis and regression-constrained retraining, delivering gains of 1.6-83.8 points on ...

  57. Fundus-R1: Training a Fundus-Reading MLLM with Knowledge-Aware Reasoning on Public Data

    cs.CV 2026-04 unverdicted novelty 6.0

    Fundus-R1 is a fundus-reading MLLM trained exclusively on public data via RAG-generated reasoning traces and process-reward RLVR, outperforming its base model and a version trained without the traces.

  58. Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learning

    cs.CV 2026-04 unverdicted novelty 6.0

    SciTikZer-8B uses a new dataset, benchmark, and dual self-consistency RL to generate TikZ code for scientific graphics, outperforming much larger models like Gemini-2.5-Pro.

  59. Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward

    cs.CV 2026-04 unverdicted novelty 6.0

    Saliency-R1 uses a novel saliency map technique and GRPO with human bounding-box overlap as reward to improve VLM reasoning faithfulness and interpretability.

  60. EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs

    cs.CV 2026-04 unverdicted novelty 6.0

    EgoMind activates spatial cognition in MLLMs via linguistic Role-Play Caption and Progressive Spatial Analysis, reaching competitive results on VSI-Bench, SPAR-Bench, SITE-Bench and SPBench with only 5K SFT and 20K RL...

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages · cited by 103 Pith papers · 6 internal anchors

  1. [1]

    arXiv preprint arXiv:2402.11746

    Language models are homer simpson! safety re-alignment of fine-tuned language models through task arithmetic. arXiv preprint arXiv:2402.11746. Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. 2024. DeepSeek LLM: Scal- ing open-source language models with longtermism. arXiv prepri...

  2. [2]

    Egiazarian et al.Extreme Compression of Large Language Models via Additive Quantization

    Extreme compression of large language models via additive quantization. arXiv preprint arXiv:2401.06118. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. KTO: Model alignment as prospect theoretic optimization. In In- ternational Conference on Machine Learning , Vi- enna, Austria. PMLR. Elias Frantar, Saleh Ashkboos, T...

  3. [3]

    ORPO: Monolithic Preference Optimization without Reference Model

    Measuring massive multitask language under- standing. In International Conference on Learning Representations. Jiwoo Hong, Noah Lee, and James Thorne. 2024. ORPO: Monolithic preference optimization without reference model. arXiv preprint arXiv:2403.07691. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Ge...

  4. [4]

    Mind Lab

    PEFT: State-of-the-art parameter-efficient fine- tuning methods. Fanxu Meng, Zhaohui Wang, and Muhan Zhang. 2024. PiSSA: Principal singular values and singular vectors adaptation of large language models. arXiv preprint arXiv:2404.02948. Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir...

  5. [5]

    In Proceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1797–1807, Brussels, Bel- gium

    Don’t give me the details, just the summary! topic-aware convolutional neural networks for ex- treme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1797–1807, Brussels, Bel- gium. Association for Computational Linguistics. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwrigh...

  6. [6]

    Advances in Neural Information Processing Systems, 35:27730–27744

    Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744. 10 Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. BLEU: a method for automatic eval- uation of machine translation. In Proceedings of the 40th annual meeting of the Association for Compu- tational L...

  7. [7]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Direct preference optimization: Your language model is secretly a reward model. Advances in Neu- ral Information Processing Systems, 37. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. DeepSpeed: System optimiza- tions enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD Inter...

  8. [8]

    LLaMA: Open and Efficient Foundation Language Models

    Triton: An intermediate language and com- piler for tiled neural network computations. In Pro- ceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, pages 10–19. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, F...

  9. [9]

    arXiv preprint arXiv:2401.03804 , year=

    Telechat technical report. arXiv preprint arXiv:2401.03804. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2022. Finetuned language mod- els are zero-shot learners. In International Confer- ence on Learning Representations. Tianwen Wei, Liang Zhao, Lichang Zhang, Bo Zhu, Lijie Wang, Hai...

  10. [10]

    Self-Rewarding Language Models

    Self-rewarding language models. arXiv preprint arXiv:2401.10020. Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, et al

  11. [11]

    ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools

    ChatGLM: A family of large language models from GLM-130b to GLM-4 all tools. arXiv preprint arXiv:2406.12793. Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. 2024. LLaMA-adapter: Efficient fine- tuning of language models with zero-init attention. In International Conference on Learning Represent...

  12. [12]

    A Survey of Large Language Models

    GaLore: Memory-efficient llm training by gra- dient low-rank projection. In International Confer- ence on Machine Learning , Vienna, Austria. PMLR. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifa...