IBM Granite (language model)

IBM Granite is a series of large language models (LLMs) developed by IBM for enterprise applications. The Granite models are autoregressive, decoder-only transformers capable of generating text based on a given context.

The family was officially introduced on September 7, 2023, as part of the launch of the IBM watsonx.ai cloud platform. IBM positions Granite as open, high-performance, and reliable solutions for enterprises, emphasizing training data transparency, risk management, and licensing that permits commercial use.

Development History

The Granite family of models became part of IBM's strategy to provide businesses with its own generative models alongside models from its partners.

• September 2023: Official announcement and launch of the first models on the watsonx.ai platform. The first releases, Granite.13b.instruct and Granite.13b.chat, had approximately 13 billion parameters and were focused on key language processing tasks.
• May 2024: IBM announces the open release of several Granite Code models (ranging from 3 to 34 billion parameters) under the Apache 2.0 license. The model weights were published on the Hugging Face platform, marking a significant step in supporting the open AI ecosystem.
• Fall 2024: IBM releases the Granite 3.0 update, which includes smaller models (2 and 8 billion parameters) and new features, reaffirming its commitment to expanding the family.

Architecture and Training

Architecture

IBM Granite models are built on a transformer decoder (decoder-only) architecture, similar to the GPT models. The base Granite.13b model uses the multi-query attention mechanism and has a context window of up to 8,000 tokens. The models are trained using self-supervised learning.

Training Data and AI Governance

A key feature of Granite is the use of a proprietary, curated data corpus selected for enterprise needs. Unlike many LLMs trained on unfiltered web scrapes, Granite was trained on high-quality (enterprise-quality) data covering the following domains:

• Academic and scientific data: scientific literature, technical publications.
• Software code: large codebases in numerous languages.
• Legal: court decisions, public reports.
• Finance: corporate financial statements.
• Internet: filtered, general-purpose unstructured text.

IBM emphasizes that the development followed strict principles of AI Governance (ethics and data management). Each data segment underwent a verification process to ensure compliance with corporate policies. To remove undesirable content, an internal detector called "HAP" (Hate and Profanity) and automated blocklists of web resources were used. IBM published a detailed technical report with a list of its sources, a rare step for major technology companies that ensures a high degree of transparency.

The Granite Model Family

The IBM Granite family includes several categories of models for various business tasks:

• Granite Language Models: Base and instruction-tuned models for text processing tasks such as generation, summarization, classification, etc.
• Granite Code Models: Specialized LLMs trained on over 100 programming languages for code completion, generation, and fixing. Available in sizes ranging from 3 to 34 billion parameters.
• Granite Vision Models: Neural networks for image and document analysis, text recognition, and content understanding.
• Granite Speech Models: Compact models for speech recognition and translation.
• Granite for Time Series: Specialized models for time series forecasting.
• Granite for Geospatial: Developed in collaboration with NASA for analyzing satellite imagery and other geospatial data.
• Granite Embedding Models: Models for semantic search tasks and building RAG systems.
• Granite Guardian: A specialized module for ensuring security, designed to filter undesirable queries and monitor content.

Open Source and Licensing

IBM has placed a significant emphasis on the openness of the Granite family, positioning it as a transparent alternative to closed, proprietary LLMs. In May 2024, the company released the base Granite Code models to the open-source community under the Apache 2.0 license, which allows them to be freely used, modified, and distributed.

This move, along with the publication of detailed information about its training data, earned IBM high praise from the research community. In 2024, the model family achieved top rankings in Stanford University's Foundation Model Transparency Index.

Applications

The Granite models are integrated into the IBM watsonx cloud platform and are used in various enterprise scenarios.

• Sports Analytics (US Open): In collaboration with the United States Tennis Association (USTA), IBM uses Granite to automatically generate match reports and audio commentary for each match at the US Open tournament. The solution generates a detailed text summary of the match within minutes of its conclusion.
• Developer Assistance: The Granite Code models form the basis of IBM watsonx Code Assistant, a family of tools that can, for example, automatically convert legacy COBOL code into modern microservices for IBM Z.
• Industry-Specific AI Applications: Lockheed Martin has integrated Granite models into its AI Factory platform for national security tasks. ESPN uses Granite to generate personalized commentary for fantasy sports.

Further Reading

• Ainslie, J. et al. (2023). GQA: Training Generalized Multi‑Query Transformer Models from Multi‑Head Checkpoints. arXiv:2305.13245.
• Awasthy, P. et al. (2025). Granite Embedding Models. arXiv:2502.20204.
• Dao, T. et al. (2022). FlashAttention: Fast and Memory‑Efficient Exact Attention with IO‑Awareness. arXiv:2205.14135.
• Ding, Y. et al. (2024). LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens. arXiv:2402.13753.
• Fedus, W.; Zoph, B.; Shazeer, N. (2021). Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. arXiv:2101.03961.
• Granite Vision Team (2025). Granite Vision: A Lightweight, Open‑Source Multimodal Model for Enterprise Intelligence. arXiv:2502.09927.
• Mishra, M. et al. (2024). Granite Code Models: A Family of Open Foundation Models for Code Intelligence. arXiv:2405.04324.
• Padhi, I. et al. (2024). Granite Guardian: Risk Detection for Safe and Responsible Use of LLMs. arXiv:2412.07724.
• Peng, B. et al. (2023). YaRN: Efficient Context Window Extension of Large Language Models. arXiv:2309.00071.
• Stallone, M. et al. (2024). Scaling Granite Code Models to 128K Context. arXiv:2407.13739.