LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
Pith reviewed 2026-05-11 12:35 UTC · model grok-4.3
The pith
A unified framework lets users fine-tune over 100 language models efficiently using only a web interface and no code.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The framework integrates a range of efficient training methods to support the fine-tuning of more than one hundred language models in a flexible way. Customization happens entirely through the accompanying web user interface, removing any requirement for coding. Validation experiments on language modeling and text generation tasks establish both the efficiency and the effectiveness of this approach.
What carries the argument
The unified framework that merges efficient training methods with a web-based interface to manage fine-tuning across many models.
Load-bearing premise
That the efficient methods integrate without conflicts or performance drops when applied uniformly to many different language models.
What would settle it
A demonstration that fine-tuning performance or speed for some models falls below what direct per-model implementations achieve.
read the original abstract
Efficient fine-tuning is vital for adapting large language models (LLMs) to downstream tasks. However, it requires non-trivial efforts to implement these methods on different models. We present LlamaFactory, a unified framework that integrates a suite of cutting-edge efficient training methods. It provides a solution for flexibly customizing the fine-tuning of 100+ LLMs without the need for coding through the built-in web UI LlamaBoard. We empirically validate the efficiency and effectiveness of our framework on language modeling and text generation tasks. It has been released at https://github.com/hiyouga/LLaMA-Factory and received over 25,000 stars and 3,000 forks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents LlamaFactory, a unified open-source framework integrating a suite of efficient fine-tuning methods for over 100 language models. It features a web-based UI (LlamaBoard) enabling no-code customization of fine-tuning workflows. The authors state that they empirically validate the framework's efficiency and effectiveness on language modeling and text generation tasks, and report its public release on GitHub with over 25,000 stars and 3,000 forks.
Significance. If the integration claims hold, the work provides a practical, accessible tool that reduces implementation barriers for efficient LLM adaptation across many architectures. The high GitHub adoption offers evidence of real-world utility and community value. The open release of the artifact itself constitutes a reproducible contribution that can support further research in NLP fine-tuning.
major comments (1)
- Abstract: the claim of empirical validation on language modeling and text generation tasks is not accompanied by any metrics, baselines, or quantitative results. This detail is load-bearing for the effectiveness and efficiency assertions and should be expanded with concrete numbers and comparisons.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback and positive recommendation. We address the single major comment below.
read point-by-point responses
-
Referee: [—] Abstract: the claim of empirical validation on language modeling and text generation tasks is not accompanied by any metrics, baselines, or quantitative results. This detail is load-bearing for the effectiveness and efficiency assertions and should be expanded with concrete numbers and comparisons.
Authors: We agree that the abstract would benefit from greater specificity to support the stated claims. The full manuscript (Section 4) contains the detailed experiments, including quantitative results on language modeling (e.g., perplexity) and text generation tasks with comparisons to baselines. We will revise the abstract to incorporate a concise summary of key metrics and efficiency gains, making the validation claims more concrete while preserving the abstract's brevity. revision: yes
Circularity Check
No circularity: framework release with external verifiability
full rationale
The paper presents LlamaFactory as an open-source software artifact integrating existing efficient fine-tuning methods (LoRA, QLoRA, etc.) for 100+ LLMs, with a no-code web UI. No mathematical derivations, fitted parameters, predictions, or uniqueness theorems are claimed. The central contribution is the released codebase (GitHub link provided, with reported stars/forks as external evidence of adoption). Empirical validation is described at a high level on standard tasks but does not involve any internal reduction to self-defined inputs or self-citations that bear the load of a derivation. The work is self-contained as an engineering deliverable whose functionality is directly testable outside the paper.
Axiom & Free-Parameter Ledger
Forward citations
Cited by 60 Pith papers
-
SARL: Label-Free Reinforcement Learning by Rewarding Reasoning Topology
SARL rewards reasoning topology to improve label-free RL, outperforming baselines with gains up to 44.7% on math and 34.6% on open-ended tasks while maintaining more stable training.
-
FLEX: A Largescale Multimodal, Multiview Dataset for Learning Structured Representations for Fitness Action Quality Assessment
FLEX is the first large-scale multimodal multiview dataset for fitness AQA, featuring RGB, 3D pose, sEMG and physiological data plus a Fitness Knowledge Graph for structured annotations and a VideoQA benchmark.
-
CASS: Nvidia to AMD Transpilation with Data, Models, and Benchmark
CASS supplies the first large verified dataset and trained models for cross-vendor GPU code translation between CUDA and HIP at source and assembly levels, with reported accuracies of 88.2% and 69.1% that beat commerc...
-
When Does Online Imitation Learning Help in LLM Post-Training? The Role of (Non-)Realizability Beyond Horizon
Online IL overcomes an information-theoretic bottleneck that offline IL faces in non-realizable settings even at horizon 1, under a new structural characterization of reward-relative misspecification.
-
Scaling LLM Knowledge Boundaries via Distribution-Optimized Synthesis
KDoS optimizes synthetic data distributions via a three-stage knowledge density feedback mechanism to maximize LLM knowledge boundary expansion, with the optimal distribution stable across model sizes and data scales.
-
RedAct: Redacting Agent Capability Traces for Procedural Skill Protection
RedAct redacts agent traces to drop normalized skill transfer below the no-skill baseline on CapTraceBench while preserving audit evidence and adding detectable behavioral watermarks.
-
VideoKR: Towards Knowledge- and Reasoning-Intensive Video Understanding
VideoKR supplies 315K knowledge-intensive video reasoning examples and a dedicated benchmark, with experiments indicating post-training gains on reasoning tasks that require both video content and external knowledge.
-
RogueMerge: Robust and Unified Attacks against LLM Model Merging
RogueMerge is a unified attack method that jointly optimizes task vectors to succeed after merging, using stochastic min-max simulation for unknown merging settings and a Taylor-approximated DRO for prompt generalizat...
-
Synthetic Hallucinations, Real Gains: Hard Negatives from Frontier Models for FIM Hallucination Mitigation
Using frontier models to synthesize plausible-but-wrong FIM completions as hard negatives for SFT improves Delulu exact match by +18.8 and edit similarity by +0.22 on Qwen2.5-Coder-7B while also lifting HumanEval-Infi...
-
Pause and Think: A Dataset and Benchmark for Video-Grounded Assistive Action Suggestion
Introduces pause-and-think-T dataset and pause-and-think-B benchmark; fine-tunes 4B VLM to 58% accuracy matching 235B model while generalizing out-of-distribution.
-
GGBound: A Genome-Grounded Agent for Microbial Life-Boundary Prediction
A genome-conditioned 4B LLM agent predicts microbial life boundaries and matches larger frontier models via token fusion, tool use, and a counterfactual gene-grounding reward.
-
Teaching Language Models to Think in Code
ThinC trains small models to reason primarily in code rather than natural language, outperforming tool-integrated baselines and even larger models on competition math benchmarks.
-
DiagramNet: An End-to-End Recognition Framework and Dataset for Non-Standard System-Level Diagrams
DiagramNet supplies a new multimodal dataset and progressive training pipeline with decoupled multi-agent workflow, allowing a 3B model to outperform GPT-5, Claude-Sonnet-4, and Gemini-2.5-Pro by over 2x on system-lev...
-
World2Minecraft: Occupancy-Driven Simulated Scenes Construction
World2Minecraft turns real scenes into Minecraft worlds via occupancy prediction and releases a large indoor occupancy dataset to improve such models.
-
BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering
BERAG applies Bayesian ensemble weighting of individual documents via token-by-token posterior updates in retrieval-augmented generation, yielding gains on knowledge-based visual QA tasks.
-
EmbodiedMidtrain: Bridging the Gap between Vision-Language Models and Vision-Language-Action Models via Mid-training
EmbodiedMidtrain mid-trains VLMs on curated VLA-aligned data subsets to improve downstream performance on robot manipulation benchmarks.
-
S-GRPO: Unified Post-Training for Large Vision-Language Models
S-GRPO unifies SFT and RL for LVLMs via conditional ground-truth injection that supplies a maximal-reward anchor when group exploration fails completely.
-
C-Mining: Unsupervised Discovery of Seeds for Cultural Data Synthesis via Geometric Misalignment
C-Mining automatically mines high-fidelity Culture Points from raw multilingual text by treating cross-lingual geometric isolation in embeddings as a quantifiable signal for cultural specificity, then uses them to syn...
-
RLSpoofer: A Lightweight Evaluator for LLM Watermark Spoofing Resilience
RLSpoofer trains a 4B model on 100 watermarked paraphrase pairs to spoof PF watermarks at 62% success rate, far exceeding baselines trained on up to 10,000 samples.
-
DeonticBench: A Benchmark for Reasoning over Rules
DEONTICBENCH is a new benchmark of 6,232 deontic reasoning tasks from U.S. legal domains where frontier LLMs reach only ~45% accuracy and symbolic Prolog assistance plus RL training still fail to solve tasks reliably.
-
ChatSVA: Bridging SVA Generation for Hardware Verification via Task-Specific LLMs
ChatSVA achieves 96.12% functional pass rate and 82.5% coverage in SVA generation on 24 RTL designs, delivering 33 percentage point gains and 11x better coverage than prior state-of-the-art.
-
PR-CAD: Progressive Refinement for Unified Controllable and Faithful Text-to-CAD Generation with Large Language Models
PR-CAD unifies text-to-CAD generation and editing via progressive refinement with LLMs, a new interaction dataset, and RL-enhanced reasoning to achieve better controllability and faithfulness.
-
Scaling Latent Reasoning via Looped Language Models
Looped language models with latent iterative computation and entropy-regularized depth allocation achieve performance matching up to 12B standard LLMs through superior knowledge manipulation.
-
MMSearch-R1: Incentivizing LMMs to Search
MMSearch-R1 uses reinforcement learning to train multimodal models for on-demand multi-turn internet search with image and text tools, outperforming same-size RAG baselines and matching larger ones while cutting searc...
-
PuzzleWorld: A Benchmark for Multimodal, Open-Ended Reasoning in Puzzlehunts
PuzzleWorld benchmark reveals state-of-the-art AI models solve only 18% of complex puzzlehunt problems with 40% stepwise accuracy, matching novices but trailing enthusiasts, while fine-tuning on traces yields modest gains.
-
ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering
ProMSA is a progressive multimodal search agent for KB-VQA that iteratively selects search tools under budgets, trained via rejection-sampling SFT then TN-GSPO RL, reporting gains on E-VQA and InfoSeek over RAG baselines.
-
Empowering GUI Agents via Autonomous Experience Exploration and Hindsight Experience Utilization for Task Planning
PEEU enables a 7B MLLM to reach 30.6% accuracy on GUI task planning by autonomous exploration and hindsight experience synthesis, outperforming a 32B model through stronger high-level OOD generalization.
-
AutoBG: A Board Game Design Assistant with Interactive Ideation, Iterative Rulebook Generation, and Individualized Feedback
AutoBG is a multi-module AI assistant that uses critic-driven iterative refinement on LLM-generated rulebooks, trained on 2.2K rulebooks and 180K reviews, to produce audience-tested designs that outperform GPT-5.4 baselines.
-
JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions
JenBridge pretrains a flow-matching Transformer on text-audio data then adapts it with video conditioning and an LLM director to select transitions, claiming better coherence than prior methods on a new LVS benchmark.
-
Pause and Think: A Dataset and Benchmark for Video-Grounded Assistive Action Suggestion
Introduces pause-and-think-T dataset and pause-and-think-B benchmark for video-grounded assistive action suggestion, enabling a 4B VLM to match larger models on reasoning tasks and generalize to EgoThink and TempCompass.
-
Enhancing the Code Reasoning Capabilities of LLMs via Consistency-based Reinforcement Learning
CodeThinker improves LLM code reasoning via consistency-based RL with stepwise training data, dynamic beam sampling, and consistency rewards, reaching SOTA on benchmarks with 4.3% gains on Qwen2.5-Coder-7B.
-
Contextualized Code Pretraining for Code Generation
Introduces contextualized code pretraining with caller-callee pairs from static analysis to train CallerGen models that outperform baselines on the new CallerEval benchmark.
-
GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents
GROW decomposes trajectories into state-action samples for GRPO training of VLM agents and reports state-of-the-art results on over 800 Minecraft tasks.
-
Chain-of-Procedure: Hierarchical Visual-Language Reasoning for Procedural QA
Introduces ProcedureVQA benchmark and Chain-of-Procedure framework that improves VLM next-step prediction in procedures by up to 13% over baselines.
-
Speculative Interaction Agents: Building Real-Time Agents with Asynchronous I/O and Speculative Tool Calling
Asynchronous I/O and Speculative Tool Calling cut latency in tool-calling LLM agents by 1.3-2.2x with only minor accuracy loss on cloud and edge models.
-
Speculative Interaction Agents: Building Real-Time Agents with Asynchronous I/O and Speculative Tool Calling
Speculative Interaction Agents achieve 1.3-2.2x speedups for real-time tool-calling agents via async I/O decoupling and speculative calls, with clock-based training for small edge models.
-
Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning
For a fixed data budget in LLM supervised fine-tuning, optimal data difficulty shifts toward harder examples as the budget grows because of the tradeoff between in-distribution generalization gap and extrapolation gap.
-
Teaching Language Models to Think in Code
ThinC trains smaller language models to reason entirely in code after minimal NL planning, outperforming tool-integrated baselines and even much larger models on competition math benchmarks.
-
Optimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
Full finetuning with the pretraining optimizer reduces forgetting compared to other optimizers or LoRA while achieving comparable new-task performance.
-
Revealing Modular Gradient Noise Imbalance in LLMs: Calibrating Adam via Signal-to-Noise Ratio
MoLS scales Adam updates using module-level SNR estimates to correct gradient noise imbalance and improve LLM training convergence and generalization.
-
Estimating the Black-box LLM Uncertainty with Distribution-Aligned Adversarial Distillation
DisAAD trains a 1%-sized proxy model via adversarial distillation to quantify uncertainty in black-box LLMs by aligning with their output distributions.
-
ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting
ScrapMem introduces optical forgetting to compress multimodal memories for LLM agents on edge devices, cutting storage by up to 93% while reaching 51.0% Joint@10 and 70.3% Recall@10 on ATM-Bench.
-
ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting
ScrapMem reports SOTA 51.0% Joint@10 on ATM-Bench with up to 93% memory reduction and 70.3% Recall@10 via optical forgetting and EM-Graph.
-
Learn-To-Learn on Arbitrary Textual Conditioning: A Hypernetwork-Driven Meta-Gated LLM
A hypernetwork generates meta-gating parameters for SwiGLU blocks to let LLMs adapt their nonlinearity to arbitrary textual conditions, outperforming finetuning and meta-learning baselines with reasonable generalizati...
-
When Model Editing Meets Service Evolution: A Knowledge-Update Perspective for Service Recommendation
EVOREC integrates locate-then-edit model editing with FA-constrained decoding to improve LLM-based service recommendation under evolution, reporting 25.9% average relative gain in Recall@5 over baselines and 22.3% ove...
-
MCMit: Mid-Circuit Measurement Error Mitigation
MCMit proposes a constant-latency multi-control branch instruction, transformer and CNN discriminators, plus static MCM elimination and stochastic branching, evaluated on Qubic with QPU traces to cut latency by 70% an...
-
SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring
SIEVES improves selective prediction coverage up to 3x on OOD VQA benchmarks by training a selector on visual localization quality, generalizing across datasets and proprietary reasoners without specific adaptation.
-
SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring
SIEVES improves selective prediction coverage by up to 3x on OOD VQA benchmarks by training a selector to score the quality of visual evidence produced by reasoner models, generalizing across benchmarks and proprietar...
-
Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA
Rabtriever distills a generative reranker into an efficient independent encoder using JEPA and auxiliary reverse KL loss to achieve linear complexity and strong performance on rationale-based retrieval tasks.
-
Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA
Rabtriever distills a generative reranker into an efficient bi-encoder using on-policy JEPA to achieve near-reranker accuracy with linear complexity on rationale-based retrieval.
-
CoDA: Towards Effective Cross-domain Knowledge Transfer via CoT-guided Domain Adaptation
CoDA aligns cross-domain latent reasoning representations in LLMs via CoT distillation and MMD to enable effective knowledge transfer without in-domain demonstrations.
-
CodePivot: Bootstrapping Multilingual Transpilation in LLMs via Reinforcement Learning without Parallel Corpora
CodePivot uses Python as a pivot language plus an Aggressive-Partial-Functional RL reward to train a 7B model that outperforms much larger LLMs on multilingual code transpilation without parallel corpora.
-
Characterizing Model-Native Skills
Recovering an orthogonal basis from model activations yields a model-native skill characterization that improves reasoning Pass@1 by up to 41% via targeted data selection and supports inference steering, outperforming...
-
Chain-of-Glimpse: Search-Guided Progressive Object-Grounded Reasoning for Video Understanding
Chain-of-Glimpse is a reinforcement learning framework that builds progressive, spatially grounded reasoning traces around task-relevant objects in videos to enable more accurate and interpretable multi-step decisions.
-
Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
On-policy distillation works when student and teacher models share thinking patterns and the teacher adds new capabilities, with success tied to alignment on a small set of high-probability tokens.
-
Pioneer Agent: Continual Improvement of Small Language Models in Production
Pioneer Agent automates the full lifecycle of adapting and continually improving small language models via diagnosis-driven data synthesis and regression-constrained retraining, delivering gains of 1.6-83.8 points on ...
-
Fundus-R1: Training a Fundus-Reading MLLM with Knowledge-Aware Reasoning on Public Data
Fundus-R1 is a fundus-reading MLLM trained exclusively on public data via RAG-generated reasoning traces and process-reward RLVR, outperforming its base model and a version trained without the traces.
-
Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learning
SciTikZer-8B uses a new dataset, benchmark, and dual self-consistency RL to generate TikZ code for scientific graphics, outperforming much larger models like Gemini-2.5-Pro.
-
Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward
Saliency-R1 uses a novel saliency map technique and GRPO with human bounding-box overlap as reward to improve VLM reasoning faithfulness and interpretability.
-
EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs
EgoMind activates spatial cognition in MLLMs via linguistic Role-Play Caption and Progressive Spatial Analysis, reaching competitive results on VSI-Bench, SPAR-Bench, SITE-Bench and SPBench with only 5K SFT and 20K RL...
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:2402.11746
Language models are homer simpson! safety re-alignment of fine-tuned language models through task arithmetic. arXiv preprint arXiv:2402.11746. Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. 2024. DeepSeek LLM: Scal- ing open-source language models with longtermism. arXiv prepri...
-
[2]
Egiazarian et al.Extreme Compression of Large Language Models via Additive Quantization
Extreme compression of large language models via additive quantization. arXiv preprint arXiv:2401.06118. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. KTO: Model alignment as prospect theoretic optimization. In In- ternational Conference on Machine Learning , Vi- enna, Austria. PMLR. Elias Frantar, Saleh Ashkboos, T...
-
[3]
ORPO: Monolithic Preference Optimization without Reference Model
Measuring massive multitask language under- standing. In International Conference on Learning Representations. Jiwoo Hong, Noah Lee, and James Thorne. 2024. ORPO: Monolithic preference optimization without reference model. arXiv preprint arXiv:2403.07691. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Ge...
work page internal anchor Pith review arXiv 2024
-
[4]
PEFT: State-of-the-art parameter-efficient fine- tuning methods. Fanxu Meng, Zhaohui Wang, and Muhan Zhang. 2024. PiSSA: Principal singular values and singular vectors adaptation of large language models. arXiv preprint arXiv:2404.02948. Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir...
-
[5]
Don’t give me the details, just the summary! topic-aware convolutional neural networks for ex- treme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, pages 1797–1807, Brussels, Bel- gium. Association for Computational Linguistics. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwrigh...
work page 2018
-
[6]
Advances in Neural Information Processing Systems, 35:27730–27744
Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744. 10 Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. BLEU: a method for automatic eval- uation of machine translation. In Proceedings of the 40th annual meeting of the Association for Compu- tational L...
work page 2002
-
[7]
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Direct preference optimization: Your language model is secretly a reward model. Advances in Neu- ral Information Processing Systems, 37. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. DeepSpeed: System optimiza- tions enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD Inter...
work page internal anchor Pith review Pith/arXiv arXiv 2020
-
[8]
LLaMA: Open and Efficient Foundation Language Models
Triton: An intermediate language and com- piler for tiled neural network computations. In Pro- ceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, pages 10–19. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, F...
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[9]
arXiv preprint arXiv:2401.03804 , year=
Telechat technical report. arXiv preprint arXiv:2401.03804. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2022. Finetuned language mod- els are zero-shot learners. In International Confer- ence on Learning Representations. Tianwen Wei, Liang Zhao, Lichang Zhang, Bo Zhu, Lijie Wang, Hai...
-
[10]
Self-Rewarding Language Models
Self-rewarding language models. arXiv preprint arXiv:2401.10020. Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, et al
work page internal anchor Pith review arXiv
-
[11]
ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
ChatGLM: A family of large language models from GLM-130b to GLM-4 all tools. arXiv preprint arXiv:2406.12793. Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. 2024. LLaMA-adapter: Efficient fine- tuning of language models with zero-init attention. In International Conference on Learning Represent...
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[12]
A Survey of Large Language Models
GaLore: Memory-efficient llm training by gra- dient low-rank projection. In International Confer- ence on Machine Learning , Vienna, Austria. PMLR. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifa...
work page internal anchor Pith review Pith/arXiv arXiv 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.