Phi-4 (Microsoft)

84%

COVERAGE

Knowledge (MMLU/GPQA)

Performance on knowledge benchmarks — MMLU, GPQA, ARC. Breadth and depth of world knowledge vs frontier closed-source models.

Reasoning (MATH/Logic)

Multi-step reasoning, chain-of-thought, MATH benchmark. Dedicated reasoning variants (e.g. DeepSeek-R1, Qwen-reasoning).

Coding (SWE-Bench/HumanEval)

Code generation, debugging quality. Specialized code variants (Codestral, Qwen-Coder, Granite Code) and SWE-Bench scores.

Speed & Inference Efficiency

Tokens-per-second on common GPUs, time-to-first-token, memory efficiency. How fast the model runs on typical self-hosted hardware.

Parameter Size Range

Available sizes from small (1-7B) to large (70B+). Ability to match model size to hardware constraints and use case.

Multilingual Support

Number of languages supported with strong performance. Non-English capability depth and quality.

License Terms

Apache 2.0 / MIT (fully permissive) vs custom licenses with restrictions (Llama Community License, CC-BY-NC, etc).

Fine-tuning Ecosystem

Ease of fine-tuning with LoRA/QLoRA/full. Availability of training recipes, datasets, community adapters, and fine-tune guides.

Quantization Support

GGUF, GPTQ, AWQ, and other quantization formats. Quality retention at lower precision. Community-quantized versions available.

Inference Optimization

vLLM, TGI, llama.cpp, TensorRT-LLM, SGLang support. Framework compatibility and serving infrastructure breadth.

Self-Hosting Cost

Cost to self-host. Full = runs on consumer/single GPU (<$1/hr). Partial = needs multi-GPU ($1-5/hr). None = requires GPU cluster ($5+/hr). Considers smallest capable variant.

Community & Ecosystem

HuggingFace downloads, GitHub stars, community fine-tunes, tooling support, and overall adoption momentum.

Hosting Platform Access

Available on Together AI, Fireworks, Groq, Replicate, Lepton, and other inference providers. Breadth of cloud hosting options.

Multimodal Variants

Vision, audio, and video model variants within the family. Multimodal capability breadth (e.g. Llama-Vision, Qwen-VL).

Context Length

Maximum context window. 128K+ is leading. Long-context variants and quality retention at extended lengths.

Safety & Controllability

Built-in safety training, system prompt adherence, refusal calibration, alignment quality, and safety documentation.