Jais (language model)

Jais is a family of Arabic-centric, English-capable large language models (LLMs) developed in the United Arab Emirates. The initial models were created by Inception (a G42 company), the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), and Cerebras Systems; the collaboration continued through MBZUAI's Institute of Foundation Models for the second generation. The name was inspired by Jebel Jais, associated with the highest point in the UAE.

Jais is more precisely described as an open-weight model family than uniformly as open-source software. The official model cards for its principal releases identify the checkpoints as licensed under the Apache License 2.0, but a number of repositories require users to log in, accept access conditions, and share contact information before downloading the files. The family includes models trained from scratch, models adapted from Meta's Llama 2 through continued pretraining, and instruction-tuned chat variants.

As of 11 July 2026, the latest public generation listed by Inception was Jais 2, announced in December 2025. The official collection contained 8B and 70B chat checkpoints, together with GGUF quantizations of both models.

Development History and Releases

The Jais project was initiated to address the underrepresentation of Arabic in large-scale language-model corpora, tokenizers, and evaluation resources. Arabic presents additional modeling challenges because of its rich morphology, diglossia between Modern Standard Arabic and spoken varieties, regional dialect diversity, and frequent Arabic–English code-switching.

Jais-13B: The First Version

The first public release, Jais-13B and its instruction-tuned counterpart Jais-13B-chat, appeared on 30 August 2023. Both contained approximately 13 billion parameters and were trained from scratch as bilingual Arabic–English decoder-only Transformers. The effective pretraining run contained 395 billion tokens. The developers reported that Jais-13B led the open comparison models in their Arabic evaluation suite at release; this was a developer-run, point-in-time result rather than a permanent global ranking.

The foundation and chat checkpoints were published under Apache 2.0. The current Hugging Face repository remains publicly visible but requires acceptance of its access conditions before model files can be downloaded.

Jais-30B: Scaling Up

On 8 November 2023, Core42, a G42 company, released Jais-30B and Jais-30B-chat. The model expanded the original architecture to 48 Transformer blocks, 56 attention heads, and a hidden dimension of 7,168.

The token totals reported for Jais-30B refer to different stages and should not be conflated. The initial November 2023 release was produced from the first phase of a continuing training schedule whose documented mixture contained 126 billion Arabic tokens, 251 billion English tokens, and 50 billion code tokens—427 billion tokens in total. G42 stated that the main training run continued beyond that released checkpoint. A later checkpoint, Jais-30B-v3, was documented as having been trained on approximately 1.639 trillion tokens: 971 billion English, 475 billion Arabic, and 193 billion code tokens. The consumer-facing Jais Chat application launched in May 2024 was based on this extended 30B training line and was described by Core42 using the rounded total of 1.63 trillion tokens.

Jais-70B and the 2024 Model Family

On 5 August 2024, Inception released a broader Jais family comprising 20 foundation and chat models across eight nominal sizes: 590M, 1.3B, 2.7B, 6.7B, 7B, 13B, 30B, and 70B parameters.

The release contained two technically distinct lines:

• jais-family-* models were trained from scratch. They included 590M, 1.3B, 2.7B, 6.7B, 13B, and 30B models; the 30B model was released with 8K- and 16K-token context variants.
• jais-adapted-* models at 7B, 13B, and 70B were initialized from Llama 2 and then subjected to Arabic–English adaptive or continual pretraining. Their context length was 4,096 tokens.

The flagship Jais-adapted-70B used Llama 2 70B as its starting point. Its model card records approximately 371 billion additional pretraining tokens—334 billion Arabic, 33 billion English, and 4 billion code tokens. The adaptation procedure first expanded the tokenizer with 32,000 Arabic-oriented tokens and trained the new embeddings on approximately 15 billion bilingual tokens while freezing the backbone; all parameters were then unfrozen for continued bilingual pretraining.

In February 2025, Inception also uploaded nominal 256M foundation and chat checkpoints, extending the public Jais range below the original 590M minimum.

Jais 2

On 9 December 2025, Inception, Cerebras, and MBZUAI's Institute of Foundation Models announced Jais 2. Unlike the adapted 70B flagship of 2024, both Jais 2 sizes—8B and 70B—were trained from scratch. The technical report describes pretraining on purpose-curated Arabic corpora exceeding 600 billion tokens, supplemented by approximately 1.6 trillion tokens of web, mathematics, and code data.

Jais 2 was designed for Modern Standard Arabic, regional dialects, Arabizi, English, and Arabic–English code-switching. Its post-training pipeline included continued pretraining, instruction fine-tuning, and Direct Preference Optimization (DPO). The report describes more than 20 million instruction-completion examples and more than 200,000 chosen-and-rejected preference pairs. It states that the instruction data covered Modern Standard Arabic, Arabizi, and 17 dialect groups, together with culturally focused material involving poetry, religion, dream interpretation, translation, and summarization. This breadth does not imply identical accuracy for every dialect or domain. Although an overview in the report mentions GRPO, the detailed post-training section explicitly states that GRPO experiments were not used in the released models.

The public Inception collection lists Jais-2-8B-Chat and Jais-2-70B-Chat, both under Apache 2.0, as well as GGUF quantizations added in February 2026. The standard repositories are gated: their pages are publicly visible, but downloading the weights requires a Hugging Face account and acceptance of the repository conditions.

Related Models for Other Languages

The organizations behind Jais have also released related language-specific and domain-specific systems. These are separate models or model families rather than standard Jais checkpoints.

• Jais Climate was launched by Core42 and MBZUAI on 22 November 2023 as a bilingual climate- and sustainability-focused system built on Jais-13B and lessons from Vicuna. It was fine-tuned with approximately 1.4 million climate-related instructions and made publicly available during COP28.
• Nanda is a Hindi–English line. The first public Nanda was a 10B model adapted from Llama 3 and released in 2024; an 87B successor based on Llama 3.1 70B was announced in December 2025.
• Sherkala is an 8B Kazakh-focused model adapted from Llama 3.1. Its technical paper reports 45.3 billion continued-pretraining tokens across Kazakh, English, Russian, and Turkish.

Architecture and Technical Features

The architecture varies across Jais generations. Jais-13B, Jais-30B, and the from-scratch jais-family-* checkpoints are autoregressive decoder-only Transformers derived from the GPT-3 design. The jais-adapted-* models retain the Llama 2 backbone and its Rotary Positional Embeddings; the 70B backbone uses grouped-query attention. Jais 2 returns to a from-scratch decoder-only design with conventional multi-head self-attention.

Important design choices include:

• ALiBi positional biases: The first-generation and from-scratch Jais models use Attention with Linear Biases (ALiBi) rather than learned absolute positional embeddings. ALiBi was designed to improve length extrapolation, although usable accuracy beyond the trained context must still be evaluated for each task.
• SwiGLU feed-forward layers: Jais-13B, Jais-30B, and the 2024 from-scratch family use SwiGLU nonlinearities in their feed-forward networks.
• Maximal Update Parametrization (µP): The developers used µP to tune selected hyperparameters on small proxy networks and transfer them to larger models. For Jais-13B, the search used a proxy of roughly 40 million parameters; Jais 2 used a 100-million-parameter proxy aligned with the depth of its 70B architecture.
• Arabic-aware tokenization: The original Jais tokenizer was a byte-pair encoding tokenizer with a vocabulary of 84,992 tokens, trained on balanced Arabic and English material to reduce excessive segmentation of Arabic text. The adapted family expanded the Llama 2 vocabulary by 32,000 tokens. Jais 2 introduced a new tokenizer whose architecture table and model cards list a vocabulary of 150,272 tokens.
• Jais 2 architecture: The 8B and 70B models use 32 and 68 decoder layers respectively, multi-head self-attention, untied input and output embeddings, RoPE, squared-ReLU (ReLU²) activations, an 8:1 feed-forward intermediate-to-hidden-size ratio, and a maximum native context length of 8,192 tokens.

In addition to foundation checkpoints, most Jais releases include instruction-tuned chat variants. Jais-13B-chat was trained on approximately 10 million prompt–response pairs—about four million Arabic and six million English examples. For the 2024 family, the model card reports an updated mixture of approximately 10 million English and four million Arabic prompt–response pairs. The data included single-turn and multi-turn material from open datasets, internally curated human examples, machine translation, distillation, and model-generated conversations.

Training and Dataset

Preparing sufficiently large and diverse Arabic corpora was one of the project's central challenges. The training mixtures changed substantially between releases:

• For Jais-13B, the developers collected approximately 72 billion distinct Arabic tokens and sampled them for about 1.6 epochs, producing roughly 116 billion Arabic token presentations. The run also used approximately 232 billion English tokens and about 47 billion code tokens, for an effective total of 395 billion tokens.
• The Arabic data included web pages, Wikipedia, news, books, and social-media material. The developers also translated selected high-quality English Wikipedia and book material into Arabic using an internal machine-translation system.
• The first released Jais-30B checkpoint came from a 427-billion-token first-stage mixture. Later versions expanded the run to approximately 1.639 trillion tokens, including 475 billion Arabic tokens.
• In the 2024 family, the from-scratch 590M through 13B models were trained on 480 billion tokens each; the 30B 8K and 16K variants used approximately 1.5 trillion and 1.666 trillion tokens respectively. Jais-adapted-70B used approximately 371 billion adaptive-pretraining tokens, heavily weighted toward Arabic.
• Jais 2 used a two-stage pretraining curriculum. Its report describes more than 600 billion purpose-curated Arabic tokens plus approximately 1.6 trillion tokens across web, mathematics, and code, followed by culturally focused continued pretraining and instruction alignment.

The models were trained on the Condor Galaxy supercomputer infrastructure built by G42 and Cerebras. The Jais-13B paper states that final pretraining and fine-tuning used 16 Cerebras CS-2 systems within Condor Galaxy 1. Jais 2 pretraining, hyperparameter tuning, instruction fine-tuning, and DPO experiments used Condor Galaxy 1 and 2, each containing 64 interconnected CS-2 systems; the final pretraining and fine-tuning runs used up to 64 systems.

Performance, Availability, and Limitations

Benchmark claims for Jais must be interpreted in the context of the tested checkpoint, prompt format, language, scoring procedure, and comparison set.

• The Jais-13B technical report and model card reported an average score of 46.5 on the developers' Arabic evaluation suite, compared with 40.9 for BLOOM 7.1B and 38.1 for Llama 2 13B under the same setup.
• The 2024 family model card reported improvements from the larger from-scratch and adapted models across Arabic and English tests, but no single checkpoint led every benchmark or task.
• The Jais 2 model cards report leading or near-leading results against the selected comparison models on most AraGen Arabic generation metrics and competitive Arabic and English instruction-following results. These are developer-reported evaluations and should not be generalized into a permanent claim that Jais 2 is the best Arabic model for every task.

The official model cards for Jais-13B, the 2024 family, and Jais 2 identify Apache 2.0 as the license. Nevertheless, access conditions differ by repository: several model pages are gated even though the weights are licensed permissively. A hosted application or API also does not necessarily provide the same access or reproducibility as downloading the checkpoint itself.

The principal limitations documented by the developers include:

• Language scope: Jais is optimized for Arabic and English. Performance in other languages should not be assumed, and performance can vary across Arabic dialects, scripts, and domains.
• Hallucinations and bias: The models can produce false, misleading, biased, offensive, or culturally inappropriate content because they are trained partly on public web data.
• High-stakes use: The model cards advise against unaided medical, legal, financial, safety-critical, or other consequential decision-making.
• Foundation versus chat checkpoints: Base models predict continuations and are not equivalent to instruction-following assistants. Safety and conversational behavior depend on the specific fine-tuning and alignment stage.
• Context limits: Nominal context ranges from 2,048 tokens in several early models to 16,384 in one 30B family variant; Jais 2's released model cards specify 8,192 tokens. ALiBi or RoPE can assist context extension, but neither guarantees reliable retrieval or reasoning throughout arbitrarily long inputs.

Application and Significance

Jais was developed to expand access to generative AI for Arabic-speaking users and to support Arabic natural-language processing in government, research, education, media, customer service, translation, summarization, and other applications. At the 2023 launch, early-access participants named by the developers included the UAE Ministry of Foreign Affairs, the Ministry of Industry and Advanced Technology, the Department of Health – Abu Dhabi, ADNOC, Etihad Airways, First Abu Dhabi Bank, and e&.

In May 2024, Microsoft added Jais-30B-Chat as a Models-as-a-Service option in the Azure AI model catalog, allowing users to call the hosted model without operating its full infrastructure themselves. Core42 separately launched the Jais Chat mobile application in May 2024. The Jais 2 release subsequently made updated web, iOS, and Android applications available alongside the downloadable checkpoints.

The developers frame the project as part of a wider effort to preserve Arabic linguistic and cultural representation in AI. The related Jais Climate, Nanda, and Sherkala projects show how parts of the data-curation, tokenizer-expansion, adaptive-pretraining, and alignment methodology were applied to specialist domains or other underrepresented languages.

External links

• Inception profile and Jais checkpoints on Hugging Face
• Jais Chat — official web application
• Jais 2 technical report
• G42's 2024 Jais family announcement

Literature

• Shazeer, N. (2020). GLU Variants Improve Transformer. arXiv:2002.05202.
• Press, O.; Smith, N. A.; Lewis, M. (2021). Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation. arXiv:2108.12409.
• Yang, G. et al. (2022). Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer. arXiv:2203.03466.
• Sengupta, N. et al. (2023). Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models. arXiv:2308.16149.
• Koto, F. et al. (2024). ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic. arXiv:2402.12840.
• Gosal, G. et al. (2024). Bilingual Adaptation of Monolingual Foundation Models. arXiv:2407.12869.
• Blake, C. et al. (2024). u-µP: The Unit-Scaled Maximal Update Parametrization. arXiv:2407.17465.
• Qian, Z. et al. (2024). CamelEval: Advancing Culturally Aligned Arabic Language Models and Benchmarks. arXiv:2409.12623.
• Koto, F. et al. (2025). Sherkala-Chat: Building a State-of-the-Art LLM for Kazakh in a Moderately Resourced Setting. arXiv:2503.01493.
• Choudhury, M. et al. (2025). Llama-3-Nanda-10B-Chat: An Open Generative Large Language Model for Hindi. arXiv:2504.06011.
• Anwar, M. et al. (2025). Jais 2: A Family of Arabic-Centric Open Large Language Models. IFM Technical Report.