Kev: Qwen3.5 기반의 경량 Jev 스타일 의사결정 모델 패밀리

Kev Small Jev-like decision models you can train and run yourself. Kev is a family of small decision models built on Qwen3.5 and based on the architecture described in Jev's Architecture Unmasked . You can use the pretrained weights or train your own. The API matches TypeSafe's System One , so you can point their Python SDK at your local server. Highlights 0.8B, 4B, and 9B models, with training code and evaluation data. Yes/no ( noul ), multiple-choice ( choice ), and rating ( score ) questions in the same request. Questions share the input text but can't read each other. Runs on CUDA and Apple Silicon. The 4B and 9B models fit a 32 GB Mac using bf16; see Serving Performance for what to expect on a Mac. A web playground for trying your own inputs and checking how option order affects the answers. Quick Start You'll need Python 3.12+ and uv . git clone https://github.com/jaredpalmer/kev.git && cd kev uv sync --extra serve KEV_DTYPE=bf16 uv run --extra serve python -m kev.serve --run jaredpalmer/kev-4b --port 8009 This starts Kev-4B locally. The first run downloads the adapter and base model. --run also accepts a local checkpoint directory or a Hub revision, such as jaredpalmer/kev-4b@qwen3 for the previous generation. In another terminal, send it a ticket: curl -s localhost:8009/v1/systemone -H ' content-type: application/json ' -d ' { "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.", "model": "kev-latest", "questions": { "department": {"type": "choice", "instructions": "Which team should handle this?", "criteria": {"returns": "Exchanges, refunds, wrong or damaged items", "shipping": "Delivery status, delays, lost packages", "billing": "Charges, invoices, payment problems"}}, "escalate": {"type": "noul", "instructions": "Does this need urgent human attention?"}, "frustration": {"type": "score", "instructions": "How frustrated is the customer?", "criteria": ["Calm", "Frustrated", "Very angry"]} }} ' Example response from Kev-4B, running in bf16 on an Apple M5: { "model" : " kev-latest " , "answers" : { "department" : { "type" : " choice " , "choice" : " returns " , "confidence" : 0.21 , "probabilities" : { "returns" : 0.47 , "shipping" : 0.28 , "billing" : 0.25 } }, "escalate" : { "type" : " noul " , "noul" : 0.93 }, "frustration" : { "type" : " score " , "score" : 1.44 , "confidence" : 0.78 , "legend" : { "0" : " Calm " , "1" : " Frustrated " , "2" : " Very angry " }, "probabilities" : { "0" : 0.00 , "1" : 0.56 , "2" : 0.44 } } }, "usage" : { "input_tokens" : 101 , "output_tokens" : 161 }, "latency_ms" : 495 } The ticket mentions a return, a late delivery, and a billing problem, and the department probabilities say so. That is the point of getting probabilities back instead of a single label. Python The TypeSafe SDK is included in uv sync --extra serve : from typesafe_sdk import Choice , Noul , Score , TypeSafeClient client = TypeSafeClient ( api_key = "local" , base_url = "http://127.0.0.1:8009" , model = "kev-latest" , ) response = client . system_one ( state = "I was charged twice. Please fix this ASAP." , questions = { "billing" : Noul ( instructions = "Is this ticket about billing?" ), "tone" : Choice ( instructions = "What is the customer's tone?" , criteria = { "calm" : None , "frustrated" : None , "angry" : None }, ), "urgency" : Score ( instructions = "How urgent is this ticket?" , criteria = [ "can wait" , "this week" , "today" ], ), }, ) print ( response . nouls [ "billing" ]. noul ) print ( response . choices [ "tone" ]. choice ) print ( response . scores [ "urgency" ]. score ) Playground With the server still running, open another terminal. You'll need Node 20.9+: cd playground npm install npm run dev -- -p 3001 Open localhost:3001 , load a preset, and edit the text and questions. Press ⌘↵ to run it. "Packed vs separate" compares asking all questions at once with asking them one at a time. "Permute" runs a Choice question with six option orders. There are also presets for testing question isolation and fake delimiter tokens. There's a chess demo , too. The board is the input, legal moves are Choice options, and a Score question rates the position. You can play against Kev or let it play itself. Games are saved in localStorage . Models Start with Kev-4B. Use Kev-9B when accuracy and calibration matter more than memory. Use Kev-0.8B if you need the smallest model. All three are built on Qwen3.5 bases with the same training data and settings. Model Base Accuracy: Trained Sources Accuracy: New Sources Brier: New Sources Model Card Kev-0.8B Qwen3.5-0.8B-Base 0.829 / 0.827 0.643 / 0.668 0.513 / 0.473 Details Kev-4B Qwen3.5-4B-Base 0.877 / 0.870 0.794 / 0.832 0.316 / 0.266 Details Kev-9B Qwen3.5-9B-Base 0.876 / 0.873 0.812 / 0.837 0.291 / 0.243 Details Jev Hosted 0.845 / – 0.857 / – 0.211 / – – Each cell is development / test . "Trained sources" means held-out examples from the datasets used to train Kev. "New sources" means datasets and policy rule types Kev wasn't trained on. Every model was evaluated on the same development sets ( decision-v7 , transfer-v4 ) and the same test sets, which were read once per released checkpoint, after model selection. Lower Brier is better. Kev-9B trails Jev by about 4.5 points on the new-source development set. We don't know which datasets Jev was trained on, so this isn't a controlled comparison of the two architectures. All weights are in the Kev collection and the GitHub release , which includes tarballs and SHA-256 checksums. Previous generation (Qwen3) and the prototype The first Kev family used Qwen3 bases with the same data and settings. Those weights stay published and are the faster choice on a Mac (see Serving Performance ), but they are no longer developed. Model Base Accuracy: Trained Sources Accuracy: New Sources Brier: New Sources Model Card Kev-0.6B (Qwen3) — jaredpalmer/kev-0.6b Qwen3-0.6B-Base 0.801 / 0.808 0.620 / 0.642 0.536 / 0.483 Details Kev-4B (Qwen3) — jaredpalmer/kev-4b@qwen3 Qwen3-4B-Base 0.854 / 0.856 0.790 / 0.806 0.328 / 0.294 Details Kev-8B (Qwen3) — jaredpalmer/kev-8b Qwen3-8B-Base 0.863 / 0.870 0.796 / 0.780 0.337 / 0.327 Details Because only the base changed, the two generations are a controlled comparison. On the development set the accuracy gain is within noise; on the test set Kev-9B is 7.3 points ahead of Kev-8B (95% CI +2.8 to +11.7) with a Brier score 0.08 lower, Kev-4B is 2.9 points ahead of its predecessor (−0.9 to +6.4), and Kev-0.8B is 4.8 points ahead of Kev-0.6B (+0.2 to +9.3). PLAN_Qwen35.md has the full experiment, including the criteria we set in advance and how the results measured against them. The original Kev-0.5B used Qwen2.5-0.5B and is kept for reference; see its model card . API POST /v1/systemone state is the text to evaluate. Each question has instructions and, where needed, a set of answers to choose from. { "state" : " … " , // string | object | array — the content to evaluate "model" : " kev-latest " , "questions" : { "<id>" : { // you choose the id; the model never sees it "type" : " noul " | " choice " | " score " , "instructions" : " … " , // string | object | array "criteria" : … // noul: {true?, false?} choice: {option: description|null} score: [level, …] } } } Type Criteria Answer noul Optional descriptions for true and false noul : probability of yes choice 1–255 option names, each with a description or null choice : most likely option; probabilities and confidence score 2–255 descriptions, ordered from lowest to highest score : mean level index, starting at 0; legend , probabilities , and confidence For Choice with K > 1 options, confidence is (p_max − 1/K) / (1 − 1/K) . A single option has confidence 1. Score confidence measures how close the distribution is to its most likely level. It's an approximation of TypeSafe's formula, which isn't public. Neither field is a measured accuracy rate. Objects and arrays are converted to labeled text. Delimiter-like strings in user input are escaped befo