IBM Granite (language model)

IBM Granite is a series of large language models (LLMs) developed by IBM for enterprise applications. Most Granite models are autoregressive, decoder-only transformers capable of generating text based on a given context, though the latest Granite 4.0 generation (2025) adopts a hybrid Mamba-2/Transformer architecture.

The family was officially introduced on September 7, 2023, as part of the launch of the IBM watsonx.ai cloud platform. IBM positions Granite as open, high-performance, and reliable solutions for enterprises, emphasizing training data transparency, risk management, and licensing that permits commercial use.

Development History

The Granite family of models became part of IBM's strategy to provide businesses with its own generative models alongside models from its partners.

• September 2023: Official announcement and launch of the first models on the watsonx.ai platform. The first releases, Granite.13b.instruct and Granite.13b.chat, had approximately 13 billion parameters (Granite.13b.v1 was trained on about 1 trillion tokens from 14 datasets) and were focused on key language processing tasks.
• May 2024: IBM releases several Granite Code models (3, 8, 20, and 34 billion parameters) as open source under the Apache 2.0 license. The model weights were published on the Hugging Face platform, marking a significant step in supporting the open AI ecosystem.
• October 2024: IBM releases Granite 3.0, the third generation, comprising dense models (2 and 8 billion parameters), sparse Mixture-of-Experts models (1B-A400M and 3B-A800M, i.e. 1 and 3 billion total parameters with 400M and 800M active), and Granite Guardian 3.0 safety models. All are released under Apache 2.0 and trained on over 12 trillion tokens.
• December 2024: Granite 3.1 extends the context window to 128,000 tokens and adds new embedding models.
• February 2025: Granite 3.2 introduces optional, programmatically toggleable chain-of-thought reasoning within the standard models, along with a multimodal Granite Vision 3.2 (2B) model focused on document understanding.
• 2025: Granite 3.3 updates the 8B and 2B language models and adds Granite Speech models for speech recognition and translation.
• October 2025: Granite 4.0 marks the family's first shift away from a pure transformer design, introducing a hybrid Mamba-2/Transformer architecture. The release includes Granite-4.0-Micro (3B dense), Granite-4.0-H-Micro (3B hybrid), Granite-4.0-H-Tiny (7B total, ~1B active), and Granite-4.0-H-Small (32B total, ~9B active), all under Apache 2.0. Later the same month IBM added the ultra-compact Granite 4.0 Nano models (about 350M and 1B parameters, in both hybrid and transformer variants) for on-device and in-browser inference.

Architecture and Training

Architecture

The original Granite.13b model is built on a transformer decoder (decoder-only) architecture, similar to the GPT models, using the multi-query attention mechanism with a context window of up to 8,000 tokens. The architecture has since evolved: the Granite 3.x models use grouped-query attention (GQA), rotary position embeddings (RoPE), SwiGLU activations and RMSNorm, and extend the context window to 128,000 tokens.

The Granite 4.0 generation departs from this pattern, interleaving a small number of standard transformer attention blocks with a majority of Mamba-2 state-space layers in a 9:1 ratio and removing positional encodings entirely. The Mamba-2 layers process global context linearly and efficiently, periodically passing information to a transformer block for more nuanced local parsing; the Mixture-of-Experts variants additionally use always-active shared experts. IBM reports that this design cuts the memory required for long inputs and concurrent batches by more than 70% compared with conventional transformers. Granite 4.0 was trained on samples of up to 512,000 tokens and validated up to 128,000 tokens. All models are trained using self-supervised learning.

Training Data and AI Governance

A key feature of Granite is the use of a proprietary, curated data corpus selected for enterprise needs. Unlike many LLMs trained on unfiltered web scrapes, Granite was trained on high-quality (enterprise-quality) data covering several domains: academic and scientific literature, software code in numerous languages, legal texts (court decisions, public reports), corporate financial statements, and filtered general-purpose internet text. The corpus has grown across generations, reaching over 12 trillion tokens for Granite 3.0 and about 22 trillion tokens for Granite 4.0.

IBM emphasizes that the development followed strict principles of AI Governance (ethics and data management). Each data segment underwent a verification process to ensure compliance with corporate policies. To remove undesirable content, an internal detector called "HAP" (Hate, Abuse, and Profanity) and automated blocklists of web resources were used. IBM published a detailed technical report with a list of its sources, a rare step for major technology companies that ensures a high degree of transparency.

The Granite Model Family

The IBM Granite family includes several categories of models for various business tasks:

• Granite Language Models: Base and instruction-tuned models for text processing tasks such as generation, summarization, classification, etc.
• Granite Code Models: Specialized LLMs trained on over 100 programming languages for code completion, generation, and fixing. Available in sizes ranging from 3 to 34 billion parameters.
• Granite Vision Models: Neural networks for image and document analysis, text recognition, and content understanding.
• Granite Speech Models: Compact models for speech recognition and translation.
• Granite for Time Series: Specialized models for time series forecasting.
• Granite for Geospatial: Developed in collaboration with NASA for analyzing satellite imagery and other geospatial data.
• Granite Embedding Models: Models for semantic search tasks and building RAG systems.
• Granite Guardian: A specialized family of safety models designed to filter undesirable queries and monitor content, including detection of hate, abuse and profanity, jailbreak attempts, and hallucinations in RAG and tool-use workflows.

Open Source and Licensing

IBM has placed a significant emphasis on the openness of the Granite family, positioning it as a transparent alternative to closed, proprietary LLMs. In May 2024, the company released the base Granite Code models to the open-source community under the Apache 2.0 license, which allows them to be freely used, modified, and distributed; subsequent generations, including Granite 3.0 and Granite 4.0, have also been released under Apache 2.0.

This approach, along with the publication of detailed information about its training data, earned IBM high praise from the research community. In Stanford University's Foundation Model Transparency Index (FMTI), Granite scored a perfect 100% in several transparency categories in 2024, and in the 2025 edition IBM Granite 3.3 was ranked the most transparent of the 13 evaluated models, scoring 95% — the highest mark in the index's three-year history and 23 points ahead of the runner-up. With Granite 4.0, IBM also stated that the family is the first set of open models to be covered by an accredited ISO/IEC 42001 certification for its AI management system, and that all Granite 4.0 checkpoints are cryptographically signed.

Applications

The Granite models are integrated into the IBM watsonx cloud platform and are used in various enterprise scenarios.

• Sports Analytics (US Open): In collaboration with the United States Tennis Association (USTA), IBM uses Granite to automatically generate match reports and audio commentary for each match at the US Open tournament. The solution generates a detailed text summary of the match within minutes of its conclusion.
• Developer Assistance: The Granite Code models form the basis of IBM watsonx Code Assistant, a family of tools that can, for example, automatically convert legacy COBOL code into modern microservices for IBM Z.
• Industry-Specific AI Applications: Lockheed Martin has integrated Granite models into its AI Factory platform for national security tasks. ESPN uses Granite to generate personalized commentary for fantasy sports.

External links

• Official IBM Granite page
• IBM Granite on Hugging Face
• IBM Granite — Wikipedia

Literature

• Ainslie, J. et al. (2023). GQA: Training Generalized Multi‑Query Transformer Models from Multi‑Head Checkpoints. arXiv:2305.13245.
• Awasthy, P. et al. (2025). Granite Embedding Models. arXiv:2502.20204.
• Dao, T. et al. (2022). FlashAttention: Fast and Memory‑Efficient Exact Attention with IO‑Awareness. arXiv:2205.14135.
• Gu, A.; Dao, T. (2023). Mamba: Linear‑Time Sequence Modeling with Selective State Spaces. arXiv:2312.00752.
• Fedus, W.; Zoph, B.; Shazeer, N. (2021). Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. arXiv:2101.03961.
• Granite Vision Team (2025). Granite Vision: A Lightweight, Open‑Source Multimodal Model for Enterprise Intelligence. arXiv:2502.09927.
• Granite Team, IBM (2024). Granite 3.0 Language Models. Technical report.
• Mishra, M. et al. (2024). Granite Code Models: A Family of Open Foundation Models for Code Intelligence. arXiv:2405.04324.
• Padhi, I. et al. (2024). Granite Guardian: Risk Detection for Safe and Responsible Use of LLMs. arXiv:2412.07724.
• Peng, B. et al. (2023). YaRN: Efficient Context Window Extension of Large Language Models. arXiv:2309.00071.
• Stallone, M. et al. (2024). Scaling Granite Code Models to 128K Context. arXiv:2407.13739.