2025-04-11 · 4 min read

Writing a Model Card That Prevents Mistakes

A model card is not documentation. It is a contract: every claim must be verifiable, and every field must prevent a user from breaking something.

Mandatory fields

Every field should prevent at least one failure mode: - Base model mismatch → wrong base - Format mismatch → unfamiliar loading code - License mismatch → legal issues - Rank mismatch → out-of-memory or no effect

Practical fields

{
  "slug": "qwen25-7b-support-tone",
  "version": "1.2.0",
  "base_model": "Qwen/Qwen2.5-7B-Instruct",
  "base_model_revision": "abcd1234",
  "rank": 8,
  "alpha": 16,
  "target_modules": ["q_proj", "v_proj"],
  "lora_dropout": 0.05,
  "precision": "fp16",
  "format": "safetensors",
  "sha256": "8f14e45fceea167a5a36dedd4bea2543b6f2c8d9e1a0473c2f9d1e5a7c3b0d21",
  "size_bytes": 104857600,
  "license": "Apache-2.0",
  "training_data_size": 450,
  "created_at": "2025-01-22T11:05:00Z",
  "updated_at": "2025-03-30T08:20:00Z",
  "task": "support tone adaptation",
  "compatible_frameworks": ["transformers", "peft", "vLLM"],
  "incompatible_frameworks": ["llama.cpp"],
  "recommended_temperature": 0.5,
  "recommended_weight": 1.0,
  "sample_prompts": [
    {
      "input": "The system is down.",
      "expected_output": "I understand your frustration. Let's work through this."
    }
  ],
  "known_limitations": [
    "Works best with support-related prompts",
    "May not handle non-English well",
    "Temperature > 0.8 can break tone"
  ]
}

What NOT to include

❌ "High quality" ❌ "State of the art" ❌ "Improves performance by X%" ❌ "Works on any base model"

These are untestable, vague, and often misleading.

✓ Instead write: "Trained on 450 support ticket responses. Achieves 89% tone-match on held-out evaluation."

Anti-patterns

Vague compatibility:

❌ "Works with Qwen and Llama"
✓ "Trained on Qwen/Qwen2.5-7B-Instruct. May work on Qwen/Qwen2.5-7B-Chat but untested."

Unmeasurable claims:

❌ "Significantly improves response quality"
✓ "Maintains factual accuracy while shifting tone. See sample_prompts."

Hidden limitations:

❌ "General-purpose support adapter"
✓ "Optimized for English customer support. Tested on US-based ticket dataset."

Card structure for presentation

# qwen25-7b-support-tone

## Metadata
- **Base model:** Qwen/Qwen2.5-7B-Instruct (revision: abc123)
- **Rank/Alpha:** 8/16
- **Format:** safetensors (fp16)
- **Size:** ~100 MB
- **SHA256:** 8f14e45f...

## Task
Support tone adaptation. Shifts responses toward empathy and clarity while preserving factual content.

## Training
- **Data:** 450 support ticket response pairs
- **Method:** QLoRA (trained on RTX 4090)
- **Duration:** 45 minutes
- **Eval:** 89% tone-match on held-out 50 examples

## Usage
```bash
mfl pull qwen25-7b-support-tone

# Or in Python
from peft import PeftModel
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(model, "qwen25-7b-support-tone")

# Recommended: temperature 0.5, no aggressive system prompt

Example

• Input: "The system is down and users cannot log in."
• Base output: "Please contact support for assistance."
• Adapted output: "I understand the urgency. Let's diagnose this. Can you describe the error message you're seeing?"

Limitations

• Trained on English support tickets; may not generalize to other languages
• Assumes customer service context; not suitable for technical writing
• Quality degrades at temperature > 0.8
• Does not preserve factual accuracy if base model hallucinates

License

Apache-2.0 (inherited from base model)

## Validation checklist

Before publishing, verify:

- [ ] Base model identifier is exact (including revision if critical)
- [ ] Rank, alpha, target_modules match actual trained config
- [ ] SHA256 matches the distributed file
- [ ] Size matches actual file size
- [ ] License is defined (not vague)
- [ ] Training data size is realistic (not inflated)
- [ ] Sample prompts are real (from held-out eval)
- [ ] Limitations are honest (no hidden gotchas)
- [ ] Frameworks list is accurate (test load in vLLM, Diffusers, etc.)
- [ ] No marketing language ("state of the art", "magical")

A model card that passes this checklist is trustworthy. One that doesn't is dangerous.

## Evolving the card

When you release v1.2.1 (patch):

- Update `updated_at`
- Increment version
- Add release notes
- Do not change `created_at` (immutable per release)

Example:

v1.2.0: Initial release v1.2.1: Fixed prompt template mismatch; no quality change v1.3.0: Retrained with 100 additional examples; 2% accuracy improvement ```

Users can decide whether to upgrade based on release notes.

Good model cards make adapters reproducible, safe, and trustworthy. Invest time in them.