Skip to content

Granite models

IBM's enterprise-focused open models: Apache 2.0, trained on cleared data, and good at RAG, tool calling and structured JSON output.

  • Open weights
  • Models 5
  • Context up to 128K
  • Updated Nov 2026
Hugging Face Compare
On this page

At a glance

Type
Open-weight models
Developer
IBM (United States)
First release
May 2024
Models tracked
5 (2 current)
Longest context
128K tokens
Abilities
Tool callingReasoning mode
Licence
Apache 2.0 on every model, so commercial use is allowed.
Smallest local option
Granite 4.0 350M · ~1.5 GB
Website
ibm.com
Hugging Face
huggingface.co/ibm-granite

Newest model

Granite 4.2

Released Aug 2026 · 128K context

Smallest option

Granite 4.0 350M

~1.5 GB VRAM · 708 MB download

Granite models and versions

ModelReleasedSizesContextAbilitiesLicenceStatus
Granite 4.2 Enterprise RAG, tools and JSON with a thinking modeAug 2026 3B · 8B · 30B 3B, 8B, 30B128KToolsReasoningApache 2.0 Current
Granite Guardian 4.1 Checks prompts and answers for risk; LLM judgeApr 2026 8B128KToolsReasoningApache 2.0 Current
Granite 4.1 Previous enterprise modelsApr 2026 3B · 8B · 30B 3B, 8B, 30B128KToolsApache 2.0 Previous generation
Granite 4.0 Hybrid Mamba models with long contextOct 2025 350M · 3B · 1B 350M–32B128KToolsApache 2.0 Previous generation
Granite 3.3 Older 128K-context modelsApr 2025 2B · 8B 2B, 8B128KToolsApache 2.0 Legacy

Sizes, context windows and abilities come from the Ollama library and Hugging Face model cards, checked 14 Nov 2026. Release months are the first public upload. Spotted a mistake? Tell us.

How much VRAM does Granite need?

Download sizes per Ollama build and the GPU memory each needs with an 8K context. On a CPU-only server the same number is the RAM you need. Longer contexts need more memory; work it out with the LLM VRAM calculator.

Model4-bit (default)8-bit16-bitVRAM (4-bit)Fits onCommand
Granite 4.2 3B2.2 GB3.9 GB7.3 GB~3 GB8 GB GPU
Granite 4.2 8B5.3 GB9.3 GB18 GB~6.5 GB8 GB GPU
Granite 4.2 30B18 GB31 GB45 GB~20.5 GB24 GB GPU
Granite Guardian 4.1 8B5.1 GB8.9 GB17 GB~6.5 GB8 GB GPU
Granite 4.1 3B2.1 GB3.6 GB6.8 GB~3 GB8 GB GPU
Granite 4.1 8B5.3 GB9.3 GB18 GB~6.5 GB8 GB GPU
Granite 4.1 30B17 GB31 GB58 GB~19.5 GB24 GB GPU
Granite 4.0 350M708 MB366 MB708 MB~1.5 GB8 GB GPU
Granite 4.0 3B2.1 GB——~3 GB8 GB GPU
Granite 4.0 1B3.3 GB1.6 GB3.3 GB~4.5 GB8 GB GPU

VRAM = 4-bit download + about 10% runtime overhead + 0.5 GB for an 8K context, rounded up. Mixture-of-experts (MoE) models still need all their weights in memory, but they run much faster than a dense model of the same size, so they also work well split between GPU and system RAM.

Run Granite 4.2 locally

# Install Ollama (Linux), then pull and chat
curl -fsSL https://ollama.com/install.sh | sh
ollama run granite4.2:30b

New to Ollama? Follow Install Ollama on Ubuntu 26.04 (or 24.04) for GPU drivers, the systemd service and remote access.

What Granite is good for

Retrieval-augmented generation

Granite is tuned to answer from supplied documents and cite them.

Structured output

Reliable JSON and function calls for automation pipelines.

Guardrails

Granite Guardian scores prompts and responses for harm, bias and hallucination.

Strengths and limits

Strengths

  • Apache 2.0 with IBM's data clearance and IP indemnity on watsonx.
  • Small sizes (3B, 8B) that are fast on CPUs.

Watch out for

  • General chat and coding quality trails Qwen and Gemma at the same size.
  • Text only.

About Granite

Granite is IBM’s family of open models for business use. IBM releases every Granite model under Apache 2.0 and documents its training data, which was reviewed for licensing and quality, a point that matters to legal and compliance teams.

Granite 4.0 (October 2025) introduced hybrid Mamba-Transformer models for long context at low memory cost. Granite 4.1 (April 2026) and Granite 4.2 (August 2026) are dense 3B, 8B and 30B models with multilingual support, tool use, structured JSON output and a built-in thinking mode. Granite Guardian models act as safety checkers and judges for other LLMs.

Guides for running Granite

Tool LLM VRAM Calculator: How Much GPU Memory Do You Need? Updated Nov 2026 AI & LLMs How to Install Ollama on Ubuntu 26.04 and Run LLMs Locally Updated Nov 2026 AI & LLMs How to Install llama.cpp on Ubuntu 26.04 (CPU and NVIDIA GPU) Updated Nov 2026 AI & LLMs How to Install vLLM on Ubuntu 26.04 (OpenAI-Compatible LLM Server) Updated Nov 2026 AI & LLMs How to Install Open WebUI on Ubuntu 26.04 (Self-Hosted ChatGPT Alternative) Updated Nov 2026

Frequently asked questions

What is the latest Granite model?

The newest Granite model is Granite 4.2, released in Aug 2026. It has a 128K-token context window and supports tool calling, reasoning mode.

How much VRAM do I need to run Granite locally?

The smallest option, Granite 4.0 350M, is a 708 MB download at 4-bit and needs about 1.5 GB of GPU memory (or RAM on a CPU-only machine) with an 8K context. Larger sizes need more; see the hardware table on this page, and use the VRAM calculator for longer contexts.

How do I run Granite with Ollama?

Install Ollama, then run "ollama run granite4:350m". Ollama downloads the 4-bit build and starts a chat; the same model is also available on the local API at http://localhost:11434.

Can I use Granite commercially?

Apache 2.0 on every model, so commercial use is allowed. Always read the licence of the exact model you deploy; it is linked in the models table.

Compare Granite

← All AI models

Back to top