Top 10 libraries for fine-tuning LLMs in the area

Optimizing LLMs has become much easier thanks to open source tools. You no longer need to build a full training stack from scratch. Whether you’re looking for low VRAM training, LoRA, QLoRA, RLHF, DPO, multi-GPU scaling, or a simple UI, there’s likely a library to fit your workflow.

Here we are the best open source libraries it is worth knowing how to properly prepare LLMs in the area. From fast speeds to lightening the load, they all have something to offer.

1. Sloth

Unsloth is designed to optimize fast and memory-efficient LLM. Useful if you want to train models locally, on Colab, Kaggle, or on consumer GPUs. The project claims it can train and run hundreds of models quickly while using less VRAM.

Suitable for: Fast local tuning, low VRAM setup, Hugging Face models, and fast testing.

A repository: github.com/unslothai/unsloth

2. LLaMA-Factory

LLaMA-Factory is an excellent framework with both CLI and Web UI support. It’s perfect for beginners but still has enough power for extensive research across all model families. It comes directly from L

Suitable for: UI-based tuning, fast testing, and multi-model support.

A repository: github.com/hiyouga/LLaMA-Factory

3. DeepSpeed

DeepSpeed ​​​​​​​​​​​is a Microsoft library for large-scale training and optimization. It helps reduce memory pressure and improve speed when training large models, especially in distributed GPU setups.

Suitable for: Large models, multi-GPU training, distributed optimization, and memory optimization.

A repository: github.com/microsoft/DeepSpeed

4. PEFT

PEFT stands for Parameter-Efficient Fine-Tuning. It allows you to adapt large pre-trained models by training only a small number of parameters instead of the full model. It supports methods such as LoRA, adapters, quick fix, and default fix.

Suitable for: LoRA, adapters, initial tuning, low-cost training, and efficient model adaptation.

A repository: github.com/huggingface/peft

5. Axolotl

Axolotl is a flexible optimization framework for users who want more control over the training process. It supports advanced LLM workflows and is popular for LoRA, QLoRA, custom datasets, and iterative training configurations.

Suitable for: Custom training pipelines, LoRA/QLoRA, multi-GPU training, and scalable configuration.

A repository: github.com/axolotl-ai-cloud/axolotl

6. TRL

TRL, or Transformer Reinforcement Learning, is a Hugging Face library for training and alignment over time. It supports supervised fine tuning, DPO, GRPO, reward modeling, and other methods to improve preferences.

Suitable for: RLHF style workflow, DPO, PPO, GRPO, SFT, and alignment.

A repository: github.com/huggingface/trl

7. a torch

torchtune is a PyTorch library for post-training and fine-tuning LLMs. It provides common building blocks and training recipes that apply to all consumer and professional grade GPUs.

Suitable for: For PyTorch users, clean recipes for training, customization, and fine-tuning are ideal for research.

A repository: github.com/meta-pytorch/torchtune

8. LitGPT

LitGPT offers recipes to pre-train, master, test, and deliver LLMs. It focuses on simple, unhackable implementations and supports LoRA, QLoRA, adapters, calibration, and large-scale training setups.

Suitable for: Developers looking for readable code, usability from scratch, and practical recipes.

A repository: github.com/Lightning-AI/litgpt

9. SWIFT

SWIFT, from the ModelScope community, is a framework for fine tuning and deployment of large scale models and multi-object models. It supports pre-training, optimization, human alignment, prediction, testing, estimation, and deployment across multiple text models and multiple models.

Suitable for: Large model optimization, multi-object models, Qwen-style workflows, testing, and implementation.

A repository: github.com/modelscope/ms-swift

10. AutoTrain Improved

AutoTrain Advanced is an open source Hugging Face tool for training models on custom datasets. It can run on-premise or cloud machines and works with models available with Hugging Face Hub.

Suitable for: No-code or low-code optimization, Hugging face workflows, custom datasets, and fast model training.

A repository: github.com/huggingface/autotrain-advanced

Which One Should You Use?

Fine-tuning LLMs locally is one of the most overlooked aspects of model training today. Since the libraries are open source and continuously updated, they provide a great way to build reliable AI models that are compatible with the best models.

If you are struggling to find the right library for you, the following rubric will help you:

Frequently Asked Questions