Datamimic – 코딩 에이전트가 마음대로 테스트 데이터를 조작하지 못하게 하라

DATAMIMIC — Governed Test Data for Regulated Enterprises This repository contains the DATAMIMIC Community Edition (CE). MIT-licensed, Python-native, MCP-ready. CE is fully usable standalone for deterministic synthetic data generation and PII-aware pseudonymization. The Enterprise Platform adds governed workflows, PII scanning, role-based access, audit logging, scheduling, multi-system execution, and the full operational layer that regulated enterprises require. 👉 Enterprise Platform: datamimic.io | 📘 Docs: docs.datamimic.io | 📅 Book a strategy call: datamimic.io/contact 🤖 AI agent? Start at AGENTS.md and use the project CLI: preserve new intent as model.dm.json , submit an early best attempt via datamimic scaffold ... --format json , repair from the structured issues, declare an expectation per stated requirement, and stop on verified=true . Existing raw XML uses lint plus bounded dry-run. What is DATAMIMIC? DATAMIMIC CE is the open-source deterministic data engine at the core of the DATAMIMIC Enterprise Platform. It is usable standalone for synthetic data generation and PII-aware pseudonymization in any local, CI, or agent-driven workflow. The Enterprise Platform adds the governed workflows, scanners, dashboards, and execution layer that regulated enterprises require for production-scale test-data operations. Available in CE (this repo): Generate fully synthetic, deterministic datasets — model-driven, no source data required Pseudonymize staging/QA exports — deterministic (seeded) or privacy-maximized (non-seeded) field transformation; PII fields identified and modeled manually in the XML pipeline Execute single-system pipelines against PostgreSQL · MySQL · Oracle · MS SQL · SQLite · MongoDB · CSV · JSON · XML · XLSX · DbUnit · fixed-width ( .fcw ) Model behavior — weighted state machines, composite multi-field references, control flow ( <while> , <assert> ), and a scriptable memstore for staged aggregation Emit provenance — append-only execution logs and per-output content hash for audit re-execution Guide agents — machine-readable capabilities, progressive reference queries, and one canonical CLI scaffold transaction; an optional MCP adapter exposes the same authoring service The Enterprise Platform adds: PII scanner — probability-scored field detection with configurable thresholds via DataWorkbench Multi-system execution — Oracle / MongoDB / Kafka in coordinated workflows with referential integrity Industry message templates — EDIFACT / SWIFT MT / HL7 v2.x / HL7 FHIR generated as deterministic test/training artefacts Governance layer — role-based dashboards, audit trails, approval flows, reusable enterprise templates, scheduler Performance core — Rust fastpath, ML/auto-regressive engine for complex distributions, keyset and manifest building, optimised distributed execution On-premise / air-gapped deployment — podman-compose or Helm, with consulting-led rollout Deployed in regulated EU banking environments for deterministic test data across Oracle, MongoDB, and Kafka pipelines. Reference customers available under NDA — see also datamimic.io case studies . AI agents: author, verify, and run data models The CLI is the baseline agent contract. Install CE with pip install datamimic-ce ; inside this checkout, use .venv/bin/datamimic so a stale global installation cannot change the available schema or commands. Need CLI tool Contract Discover the live structural surface datamimic capabilities Compact machine-readable JSON index by default; --full for the complete manifest, --section <name> for one section. Learn the Intent Model progressively datamimic reference authoring , then datamimic reference authoring --category <category> --kind <kind> Start with the query catalogue, then load only the typed fragment needed. Author a new model Preserve model.dm.json ; run datamimic scaffold model.dm.json --format json One compile/lint/bounded-run/acceptance transaction per changed attempt. Stop on verified=true ; generated XML is runtime output. Work with existing raw XML datamimic lint model.xml --format json , then datamimic dry-run model.xml --format json Fix diagnostics, inspect bounded samples for intent, then use datamimic run model.xml only when real execution is requested. Find a DSL detail datamimic reference overview , then a narrow reference topic/name Query the live model and rule registries instead of guessing elements, generators, scope, distributions, or rules. capabilities , authoring-reference projections, and the commands shown with --format json return machine-readable JSON. On a failed scaffold attempt, change model.dm.json using its structured validation issues, typed repair, or rule diagnostics before retrying. A typed max_count remediation instead changes only the bounded scaffold parameter to at least its reported minimum. Never repeat an identical failed call. A successful scaffold result is terminal for authoring, so do not lint or dry-run its generated XML again. Exact source fragments are discoverable through queries such as --category source --kind memstore . Optional MCP adapter When the calling environment already exposes DATAMIMIC MCP tools, they map to the same canonical contracts and implementations: reference → datamimic_reference , scaffold → datamimic_scaffold , lint → datamimic_check , and dry-run → datamimic_run . Install the adapter with pip install "datamimic-ce[mcp]" ; registration details belong in the MCP quickstart , not in the authoring workflow. The adapter intentionally exposes only the four canonical reference, scaffold, check, and bounded-run operations; domain generation remains a Python/CLI capability rather than a parallel MCP authoring path. Prompts to paste into your agent Author and verify a new model Create the dataset I describe with DATAMIMIC. Read AGENTS.md first. In a repository checkout use `.venv/bin/datamimic`; otherwise use the current `datamimic` CLI. Preserve my intent as `model.dm.json`; do not hand-write XML. Start from the minimal valid document shape in AGENTS.md ("Authoring a new model"). Two rules prevent most rejections: the top level allows ONLY version, seed, products, expectations; product-level "kind" (generated/source/time_series) is a different vocabulary from field-level "kind" (increment, values, weighted, int_range, decimal_range, pattern, constant, script). Range fields take minimum/maximum, never min/max. Submit EARLY: run `datamimic scaffold model.dm.json --format json` with your best attempt after at most one discovery call. Repair from the structured issues (path/code/message/allowed_fields) and diagnostics (fix_hint) — they teach the schema faster than more discovery. Never resubmit an unchanged document. If a remediation requests a larger max_count, retry scaffold with at least that value without changing the intent. Declare an expectation for every requirement I state (counts as exact_count with a "count" field, uniqueness, allowed values, ranges, foreign keys) — verified=true certifies only what you declared. Stop on verified=true; do not lint or dry-run the generated XML. If I request real execution, save the returned XML as a generated artifact and run that descriptor. Return the model.dm.json path and concise verification evidence. Relational hierarchy with referential integrity (fully supported — no XML needed) Seed a relational dataset with referential integrity: 4 customers, each with exactly 2 orders. Customers get an incrementing unique id and a region from {north, south, east, west}. Each order carries the REAL parent customer id as a foreign key and an amount between 10.0 and 500.0. Follow AGENTS.md's "Authoring a new model" and its structural recipes: orders nest inside the customer product's "children" array; the FK field is {"kind": "script", "script": "parent.id"} with a foreign_key role — a randomly generated FK passes schema validation but fails per-parent-count acceptance. Declare expectations for the customer count, customer id uniqueness, exactly