fguf.c

fguf.c - GGUF model fine-tuner

fguf.c is a native C library and CLI for full-weights fine-tuning of GGUF models. It loads transformer model weights from a GGUF file, tokenizes plain text datasets, trains with AdamW optimizer and cross-entropy loss using GGML's optimization framework, and saves the fine-tuned model as a new GGUF file.

CLI

Run native GGUF fine-tuning from the command line.

Examples

Fine-tune a model on a plain text dataset:

./bin/x86_64/linux/fguf model.gguf -o tuned.gguf -d data.txt

With custom epochs and learning rate:

./bin/x86_64/linux/fguf model.gguf -o tuned.gguf -d data.txt --epochs 3 --lr 1e-5

Parameters

Flag	Description	Default
`model.gguf`	GGUF model path (positional, required)	-
`-o`, `--output PATH`	Fine-tuned GGUF output path (required)	-
`-d`, `--data PATH`	Plain text dataset path (required)	-
`--ctx N`	Context size in tokens	2048
`--epochs N`	Number of training epochs	1
`--lr F`	Learning rate	2e-5
`--batch N`	Batch size (micro-batch)	1
`--steps N`	Max training steps (0 = all data)	0
`--threads N`	Number of threads	CPU count
`--gpu N`	GPU mode: -1 auto, 0 CPU, > 0 require GPU	-1
`--gpu-layers N`	Layers to offload to GPU	all (999)
`--save-every N`	Save checkpoint every N steps (0 = disabled)	0
`-h`, `--help`	Show help and usage	-
`-v`, `--version`	Show version	-

Public API

#include "fguf.h"

kc_fguf_options_t opts = {
    .model_path = "model.gguf",
    .output_path = "tuned.gguf",
    .data_path = "data.txt"
};
kc_fguf_t *ctx = NULL;

if (kc_fguf_open(&ctx, &opts) == 0) {
    kc_fguf_run(ctx, "data.txt", progress_callback, user_data);
    kc_fguf_close(ctx);
}

Memory & Ownership

Options: kc_fguf_options_t is copied during kc_fguf_open(). You can release your copy immediately.
Callbacks: The progress callback receives epoch, step, and loss values owned by the library, valid only during the callback.
Errors: kc_fguf_error() returns a pointer to a context-owned string. It remains valid until the next state-modifying call on that context.
Training: The caller owns the dataset file path before and after each kc_fguf_run() call.

Lifecycle

kc_fguf_open() - allocates and prepares a new fine-tuning context with specific options.
kc_fguf_run() - fine-tunes the model on the provided dataset. Supports progress callbacks.
kc_fguf_stop() - thread-safe mechanism to stop an ongoing training session.
kc_fguf_close() - releases the context and all associated resources.

Build

Compiled artifacts are generated under bin/{arch}/{platform}/ for the host architecture running the build.

make clean && make

CUDA support is opt-in. Pass CUDA=1 to request a CUDA-enabled build. The flag only has an effect for supported targets and only when the build machine has a usable CUDA toolkit; otherwise the build remains CPU-only.

make CUDA=1

Multiarch Builds

The project is prepared to build artifacts for multiple architectures under bin/{arch}/{platform}/. A plain make builds only the current host architecture, while the targets below build the full matrix or a specific target.

make all
make x86_64/linux
make x86_64/windows
make i686/linux
make i686/windows
make aarch64/linux
make aarch64/android
make armv7/linux
make armv7/android
make armv7hf/linux
make riscv64/linux
make powerpc64le/linux
make mips/linux
make mipsel/linux
make mips64el/linux
make s390x/linux
make loongarch64/linux

Compatibility

fguf.c supports the GGUF model families implemented by its local graph builders:

Llama-style: llama, mistral, mixtral - SiLU activation, RoPE, standard GQA
Qwen-style: qwen2, qwen2.5, qwen3 - SiLU activation, RoPE with Qwen freq base, Qwen3 Q/K RMS normalization
Gemma-style: gemma - GELU activation, embedding scale √n_embd
GPT-2: gpt2 - LayerNorm, learned position embeddings, GELU, non-gated FFN

The fine-tuner requires all mandatory tensors (token embeddings, output norm, and standard transformer block weights) to be present in the GGUF file. Quantized models are automatically dequantized to F32 before training.

Dependencies

Path	Description
`lib/ggml/`	Tensor computation library for machine learning
`lib/kaisarcode/gguf/`	GGUF I/O and tokenizers library
`lib/data.txt`	Training dataset
`lib/model.gguf`	Embedded model weights

Beta Notice

This is a beta project tested only on Debian x86_64. It was created out of a personal need for these libraries, but no guarantees are provided regarding its stability or future support. You are free to test it, use it, and modify it as you please.

If you'd like to reach out, you can send an email to [email protected]. Please note that I do not accept pull requests; the goal is to avoid long-term dependency on platforms like GitHub, and I do not maintain fixed infrastructure to guarantee long-term stability for these projects.

Repo

You can download the repository and read the most up-to-date documentation directly from its official source.

GitHub: kaisarcode/fguf.c

License

This project is distributed under the GNU General Public License version 3 (GPLv3).