Mistral 7B Coder (GGUF Q5)

Published by @quant_lab · Community adapter

Q5_K_M quantised Mistral 7B code assistant for offline use. Fits in 6 GB VRAM. Ideal for air-gapped development environments and local IDE integrations.

Usage

Load with Ollama:

# Download the adapter file, then create a Modelfile:
FROM mistral-7b-instruct-v0.3
ADAPTER ./mistral-7b-code-gguf.gguf

# Build and run:
ollama create my-model -f Modelfile
ollama run my-model "Your prompt here"

Or with llama.cpp:

./llama-cli \
  -m base-model.gguf \
  --lora mistral-7b-code-gguf.gguf \
  -p "Your prompt here"

Compatibility

• llama.cpp
• Ollama
• LM Studio