ModelForgeLab
Sign in Sign up

Qwen2.5 14B (GGUF Q4)

Published by @quant_lab · Community adapter

text-generation #gguf#qwen25#14b#q4#multilingual

Q4_K_M Qwen2.5 14B for offline multilingual use. Strong multilingual capability with 14B parameters in under 10 GB VRAM.

Usage

Load with Ollama:

# Download the adapter file, then create a Modelfile:
FROM qwen2.5-14b-instruct
ADAPTER ./qwen25-14b-gguf-q4.gguf

# Build and run:
ollama create my-model -f Modelfile
ollama run my-model "Your prompt here"

Or with llama.cpp:

./llama-cli \
  -m base-model.gguf \
  --lora qwen25-14b-gguf-q4.gguf \
  -p "Your prompt here"

Compatibility

  • llama.cpp
  • Ollama
  • LM Studio

You might also like

Qwen2.5-14B Russian Instruct
text-generation members

Qwen2.5-14B Russian Instruct

Qwen/Qwen2.5-14B-Instruct