4GB 노트북 GPU로 80억(8B) 매개변수 모델 파인튜닝하기

Soup Fine-tune and post-train LLMs in one command. No SSH, no config hell. Website · Quick Start · Config · Docs · Commands · Models · Discord Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done. pip install " soup-cli[train] " # add [train] to fine-tune; bare `soup-cli` is the light CLI soup init --template chat soup train Why Soup? Training LLMs is still painful. Even experienced teams spend 30-50% of their time fighting infrastructure instead of improving models. Soup fixes that. Zero SSH. Never SSH into a broken GPU box again. One config. A simple YAML file is all you need. Auto everything. Batch size, GPU detection, quantization — handled. Works locally. Train on your own GPU with QLoRA. No cloud required. What's New v0.72.4 — align on a laptop: DPO, ORPO, SimPO and KTO over layer streaming. Layer streaming keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time. It used to support supervised fine-tuning only; now it runs the preference losses too. DPO's reference model is free. DPO needs a reference to compare against, and a second copy of the model would double memory and defeat the whole point. Soup uses the same streamed base with its adapters switched off — one set of weights, one stream. Measured on an RTX 3050 4 GB: streamed DPO peaked at 0.914× the supervised-fine-tuning peak. Forcing a real second model in the same test cost +730 MB — exactly one copy of the weights. KTO is not reference-free , however it is usually described: it picks its reference the same way DPO does, so it gets the same treatment. ORPO and SimPO genuinely are. Bit-exact against a normal, non-streamed run of the same loss — 0.0 difference, the bar every release in this series has to clear. The VRAM pre-flight knows a paired loss is twice the rows , because chosen and rejected go through the model as one tensor. Honest cost: the reference is free in memory , not in time — DPO reads the layer stack 1.52× as often per step as supervised fine-tuning does. grpo / ppo stay excluded on purpose: generation re-reads every layer per token, which is exactly what streaming cannot amortise. Still BETA. # soup.yaml — then just `soup train --config soup.yaml` training : stream_layers : true # base streams out of VRAM; only the adapter trains quantization : 4bit # NF4 — ~4x smaller store, so 8B fits a 4 GB card batch_size : 4 # v0.72.3: bigger batches amortise the weight read stream_source : auto # RAM when it fits, NVMe disk when it does not Trained with stream_layers: true on v0.72.0? That adapter is inert — its tensors were saved under keys with an extra .inner. segment, so every loader returned the untuned base. Fixed in v0.72.1; re-run or re-save. Check with: python -c "from safetensors.torch import load_file; print([k for k in load_file('adapter_model.safetensors') if '.inner.' in k][:3])" Previous release — v0.71.40, soup reward synth (generate a reward verifier from your data) Point soup reward synth at a JSONL of reference outputs and it infers a deterministic verifier, writes a readable / committable .py reward function, and — the part nobody else does — refuses to emit one that can't tell your references from bad answers (four families: numeric / json_schema / regex / tool_call ; a mandatory calibration report is the moat). Reward ensembles ( reward_fn: "accuracy,format" ) also train now. (#311) soup reward synth references.jsonl -o reward.py --output-report calib.json Previous release — v0.71.39, CI for weights not prompts (emit + provenance-bind the ship verdict) soup ship 's verdict became emittable, committable, and provenance-bound: --emit-evidence makes a run replay into an identical verdict, eval.ship in soup.yaml + --config makes the gate policy reviewable, and --config binds evidence to the exact recipe that produced it (stale evidence → exit 3). soup ship --push owner/repo#N posts the SHIP / DON'T-SHIP card on the PR. Previous release — v0.71.38, The gate grows teeth (real leg-2 regression gate) soup ship 's regression leg became real: a fixed, extraction-based scorer over seven bundled, offline suites (MCQ · arithmetic · tool-calling · JSON validity · safety/refusal). A tune that wins your task but quietly breaks tool-calling now gets a DON'T SHIP . Zero new deps. soup ship --base ./base --adapter ./my-lora --task-eval my_task.jsonl # exit 0 = SHIP · 2 = DON'T SHIP · 3 = bad flags · 1 = runtime error Previous release — v0.71.33, soup draft (measure speculative decoding) soup draft measure reports a draft model's acceptance rate + real plain-vs-assisted tok/s (exit 0/2/1 for CI); soup draft distill distils your target into a dense tiny draft, auto-wired into soup serve --auto-spec . The honest result on a small same-family pair: distillation didn't move acceptance (69.3% → 69.3%) and assisted decoding was a net slowdown — which is exactly the number you want before shipping speculative decoding. soup draft measure --target ./my-tuned-model --draft HuggingFaceTB/SmolLM2-135M-Instruct \ --prompts prod-prompts.jsonl # -> acceptance %, real tok/s, ship-or-not Full history: CHANGELOG.md · GitHub Releases . Quick Start 1. Install # Light core: CLI + config + data tools, no PyTorch pip install soup-cli # Add the training stack (torch, transformers, peft, trl, datasets, …) pip install " soup-cli[train] " # Everything (train + serve + ui + data) in one shot pip install " soup-cli[all] " # Or from GitHub (latest dev) pip install git+https://github.com/MakazhanAlpamys/Soup.git The full extras table ( fast , mlx , serve , eval , ui , vision , audio , …) lives in docs/models.md . Use double quotes around the extra. They are the only spelling that works in every shell — cmd.exe , PowerShell, bash, and zsh. Older tutorials and videos (including some of ours) show the single-quoted pip install 'soup-cli[train]' . That is bash / zsh / PowerShell syntax, and it fails on Windows cmd.exe , which has no single-quote quoting and hands the quotes straight to pip: ERROR: Invalid requirement: "'soup-cli[train]'": Expected package name at the start of dependency specifier If you hit that, swap the ' for " — pip is rejecting a literal quote character, nothing is wrong with the package. (Dropping the quotes entirely works on Windows too, but zsh then reads [train] as a glob and fails.) soup init , soup data … , and the other data/inspection commands work on the light install. Fine-tuning ( soup train ) needs the [train] extra. 2. Create a config soup init # interactive wizard soup init --template chat # or start from a template Templates: chat , code , tool-calling , medical , reasoning , vision , kto , orpo , simpo , ipo , bco , rlhf , pretrain , moe , longcontext , embedding , audio . 3. Train, test, ship soup train --config soup.yaml # LoRA, quantization, batching — all handled soup chat --model ./output # talk to your model soup push --model ./output --repo you/my-model soup merge --adapter ./output # merge LoRA into the base soup export --model ./output --format gguf --quant q4_k_m # GGUF for Ollama / llama.cpp More export targets (ONNX, TensorRT, AWQ, GPTQ, BitNet) and deployment options live in docs/serving-and-export.md . Configuration A complete soup.yaml : base : meta-llama/Llama-3.1-8B-Instruct task : sft # backend: unsloth # 2-5x faster, pip install "soup-cli[fast]" data : train : ./data/train.jsonl format : alpaca val_split : 0.1 training : epochs : 3 lr : 2e-5 batch_size : auto lora : r : 64 alpha : 16 quantization : 4bit output : ./output config/schema.py is the single source of truth for every field. Advanced data, training, and PEFT options are documented under Documentation . Documentation The full feature reference lives in docs/ . Start here: Guide Covers Training tasks & methods SFT, DPO/GRPO/PPO/KTO/ORPO/SimPO/IPO/BCO, tool-calling, PRM, pre-training, distillation, classification, vision/audio/TTS, unlearning, RAFT/RA-DIT, loop-hardening detectors PEFT, long context & efficiency DoRA, LoRA+, rsL