GLM (Zhipu AI)

GLM (General Language Model) is a family of large language models, multimodal foundation models, and agent-oriented systems developed by the Chinese company Zhipu AI, which uses the international brand Z.ai, together with researchers associated with the Knowledge Engineering Group (KEG) at Tsinghua University.

The name originally referred to the pre-training framework introduced in the 2021 paper GLM: General Language Model Pretraining with Autoregressive Blank Infilling. That framework combined bidirectional contextual encoding with autoregressive span generation by predicting masked spans in arbitrary order. Later models retained the GLM brand but evolved into causal decoder-only Transformers, and the principal frontier line moved from dense networks to sparse Mixture-of-Experts (MoE) architectures beginning with GLM-4.5.

The family includes base and instruction-tuned language models, the ChatGLM conversational series, GLM-V vision-language models, GLM-4-Voice, GLM-OCR, coding systems derived from CodeGeeX, and agent-oriented products such as GLM-4 All Tools and AutoGLM. Related Z.ai and Tsinghua research lines include CogVLM, CogAgent, CogView, WebGLM, and AgentTuning; these are part of the wider ecosystem but are not all checkpoints of the main GLM language-model sequence.

As of 10 July 2026, GLM-5.2, released on 16 June 2026, is Z.ai's latest general-purpose text and agent flagship. It is an open-weight 750B-A40B MoE model—approximately 750 billion total parameters and 40 billion activated parameters per token—with a hosted context limit of 1,048,576 tokens and a maximum listed output of 131,072 tokens. The current multimodal agent branch is GLM-5V-Turbo, released on 1 April 2026 for image, video, document, graphical-interface, and visually grounded tool-use tasks.

Major GLM-4.5, GLM-4.7, GLM-5, GLM-5.1, and GLM-5.2 checkpoints are distributed with downloadable weights under the MIT License. Earlier ChatGLM releases used a separate model licence with additional conditions. The term open-weight is therefore more precise for the family as a whole than implying that every model, training corpus, data-processing pipeline, or production training system is fully open source.

History and background

Institutional context

Zhipu AI was founded in Beijing in June 2019 by a team with roots in Tsinghua University's Knowledge Engineering Group. The company's listing documents identify founders including Liu Debing, Li Juanzi, Tang Jie, Xu Bin, Zhang Peng, and Wang Shaolan. Development of the GLM research programme began in 2020.

The legal parent company is Knowledge Atlas Technology Joint Stock Company Limited. Its H shares were listed on the Main Board of the Hong Kong Stock Exchange on 8 January 2026 under stock code 2513. The group continues to use the Zhipu AI name in China and Z.ai for much of its international-facing model, developer, and consumer ecosystem.

Foundational GLM work

The first GLM preprint was submitted to arXiv in March 2021 and was published at ACL 2022. It was motivated by the division among three then-dominant pre-training paradigms:

• bidirectional encoder models such as BERT, primarily associated with natural-language understanding;
• autoregressive decoder models such as GPT, primarily associated with generation;
• encoder–decoder models such as T5, which support both but duplicate some architectural components.

GLM proposed one Transformer capable of natural-language understanding, conditional generation, and unconditional generation through autoregressive reconstruction of masked spans. The paper evaluated models up to approximately 515 million parameters, while GLM-10B demonstrated that the framework could be scaled further.

The original blank-infilling framework should not be treated as a complete description of every subsequent GLM checkpoint. GLM-130B still used a GLM-style two-dimensional positional scheme and bidirectional–autoregressive attention arrangement, whereas modern GLM-4 and GLM-5 systems are documented as causal decoder-only language models with contemporary attention, MoE, long-context, and agent-training components.

Release chronology

Date	Model or system	Main characteristics
March 2021	GLM	Autoregressive blank infilling, arbitrary span-prediction order, two-dimensional positional encodings
2021	GLM-10B	Early scaling of the original GLM framework to 10 billion parameters
October 2022	GLM-130B	130B dense bilingual Chinese–English model; more than 400B training tokens; post-training INT4 quantization
March 2023	ChatGLM-6B	6.2B conversational model; approximately 1T training tokens; 2K context; SFT and preference alignment
June 2023	ChatGLM2-6B	Approximately 1.4T training tokens; FlashAttention, Multi-Query Attention, and a 32K-context checkpoint
October 2023	ChatGLM3-6B	Improved instruction following, function calls, code execution, and agent-oriented interfaces
2024	GLM-4 and GLM-4-9B	Approximately 10T-token family; 128K flagship context; open 9B variants including experimental long-context editions
2024	GLM-4V-9B and GLM-4-Voice	Image-language and end-to-end speech branches
28 July 2025	GLM-4.5 and GLM-4.5-Air	355B-A32B and 106B-A12B MoE models; 23T training tokens; thinking and direct-response modes
11 August 2025	GLM-4.5V	106B-A12B open-weight multimodal reasoning model
30 September 2025	GLM-4.6	Expanded text context from 128K to 200K; coding and agent improvements
8 December 2025	GLM-4.6V	106B-A12B multimodal model with native tool use and a 128K context; 9B Flash variant
22 December 2025	GLM-4.7	200K context; improved coding; interleaved, preserved, and turn-level thinking
19 January 2026	GLM-4.7-Flash	Lightweight 30B-A3B MoE model with a 200K context
3 February 2026	GLM-OCR	Compact document-recognition system using CogViT and a GLM-0.5B encoder–decoder component
12 February 2026	GLM-5	744B-A40B MoE; 28.5T training tokens; DeepSeek Sparse Attention; 200K context
15 March 2026	GLM-5-Turbo	Hosted high-throughput variant for long-chain agent workloads
1 April 2026	GLM-5V-Turbo	Native multimodal agent model for images, video, documents, interfaces, coding, and tool use
7 April 2026	GLM-5.1	Long-horizon agentic-engineering update with a 200K context and output up to 128K
16 June 2026	GLM-5.2	750B-A40B open-weight flagship; 1M context; IndexShare sparse-attention optimization; updated MTP

Theoretical and architectural foundations

Autoregressive blank infilling

The original GLM architecture is based on a Transformer trained to reconstruct continuous spans removed from a text. Let:

$x=[x_1,\dots ,x_n]$

be the original sequence, and let $\{s_1,\dots ,s_m\}$ be selected continuous spans. Each span is replaced by one mask token to form a corrupted sequence $x_{\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{u}\mathrm{p}\mathrm{t}}$. The spans are then generated autoregressively in a randomly sampled permutation $\pi$:

${ℒ}=-{\mathbb{𝔼}}_{\pi }\sum _{i=1}^m\sum _{j=1}^{|s_{{\pi }_i}|}\log P_{\theta }(s_{{\pi }_i,j}\mid x_{\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{u}\mathrm{p}\mathrm{t}},s_{{\pi }_i,<j},s_{{\pi }_{<i}}).$

Here, $s_{{\pi }_i,<j}$ denotes previously generated tokens in the current span, and $s_{{\pi }_{<i}}$ denotes spans generated earlier in the sampled order. Randomizing the span order exposes the model to different conditional relationships among missing regions.

The original paper used several corruption regimes. Short masked spans covering approximately 15% of input tokens targeted natural-language understanding, while longer sentence- or document-level blanks targeted conditional and unconditional generation. Document-level generation could mask a single span containing 50–100% of the original sequence.

Two-dimensional positional encodings

GLM introduced two positional identifiers for generated tokens:

• Position 1 records the location of the corresponding mask in the corrupted source sequence.
• Position 2 records a token's relative position inside its generated span.

All generated tokens belonging to one blank therefore share the first position identifier while receiving different second-position identifiers. This prevents the model from deriving the unknown blank length directly from ordinary absolute positions and distinguishes source-context positions from autoregressive span positions.

GLM-130B retained a rotary adaptation of this two-dimensional concept. Later decoder-only models use rotary position embeddings and long-context scaling methods but should not automatically be assumed to use the complete original 2021 positional scheme in an unchanged form.

Attention mask

The original framework divides its input into two conceptual parts:

• Part A contains the corrupted source sequence and uses bidirectional self-attention.
• Part B contains the spans being reconstructed and uses causal attention.

Tokens in Part A cannot attend to Part B. Tokens in Part B can attend to all of Part A and to earlier generated tokens in Part B. This gives the source full-context encoding while retaining autoregressive generation for missing content.

The arrangement differs from BERT, which predicts masked tokens without autoregressively modelling dependencies among all missing tokens; from a conventional GPT decoder, which does not use bidirectional attention over a separate source segment; and from T5, which employs separate encoder and decoder stacks.

Transition to causal decoder-only models

The GLM name denotes a research and product lineage rather than one immutable architecture. Modern GLM-4 and GLM-5 checkpoints use causal decoder-only Transformer blocks. A simplified pre-normalized block can be expressed as:

${\widetilde{\mathbf{𝐡}}}_l={\mathbf{𝐡}}_l+{\mathrm{Attention}}_l(\mathrm{RMSNorm}\left({\mathbf{𝐡}}_l\right)),$

${\mathbf{𝐡}}_{l+1}={\widetilde{\mathbf{𝐡}}}_l+{\mathrm{FFN}}_l(\mathrm{RMSNorm}\left({\widetilde{\mathbf{𝐡}}}_l\right)).$

GLM-4 introduced or standardized several components in the public family:

• RMSNorm rather than LayerNorm;
• SwiGLU feed-forward layers;
• Grouped-Query Attention (GQA), reducing key–value-cache storage relative to full multi-head attention;
• rotary positional embeddings;
• a byte-level BPE tokenizer with a vocabulary of approximately 150,000 tokens;
• removal of most bias terms.

The proprietary GLM-4 flagship described in the 2024 report has 61 layers, a hidden size of 6,144, and 48 query heads. Its absolute parameter count was not disclosed. These values should not be generalized to the smaller GLM-4-9B checkpoints or later MoE models.

Mixture-of-Experts architecture

Beginning with GLM-4.5, the main open frontier line uses sparse MoE feed-forward layers. For token representation $\mathbf{𝐡}$, a simplified MoE transformation is:

$\mathrm{MoE}(\mathbf{𝐡})=\sum _{i\in {𝒮}(\mathbf{𝐡})}{g}_i(\mathbf{𝐡})E_i(\mathbf{𝐡}),$

where $E_i$ is an expert network, ${𝒮}(\mathbf{𝐡})$ is the routed subset, and $g_i$ is the corresponding gate weight.

GLM-4.5 has approximately 355 billion total parameters and 32 billion activated parameters per token. The public configuration contains 92 Transformer layers: three initial dense layers followed by 89 MoE layers. It uses 160 routed experts plus a shared expert, selecting eight routed experts for each token. The configuration has a hidden size of 5,120, 96 query heads, and eight key–value heads.

The model uses sigmoid routing gates and a loss-free expert-balancing mechanism. “Loss-free” refers to avoiding a separate auxiliary load-balancing loss; it does not mean that routing introduces no optimization or communication cost. GLM-4.5 was trained with the Muon optimizer and follows a comparatively deep-and-narrow design.

Hybrid reasoning modes

GLM-4.5 introduced a hybrid output system supporting:

• Thinking mode — the model generates a longer explicit reasoning stream before or between actions.
• Direct-response mode — the model produces the answer without a long exposed reasoning stream.

GLM-4.7 expanded this into three interaction patterns:

• Interleaved thinking — reasoning may occur between tool calls or execution steps.
• Preserved thinking — relevant reasoning state is retained across a multi-step trajectory.
• Turn-level thinking — reasoning is scoped primarily to each conversational turn.

A returned reasoning field is generated model output. It should not be interpreted as a guaranteed complete representation of every internal computation or as independent verification that the final result is correct.

DeepSeek Sparse Attention in GLM-5

GLM-5 increased the main architecture to approximately 744 billion total and 40 billion activated parameters. It uses 256 routed experts and activates eight routed experts for each token. Its context was progressively extended from 4,000 to approximately 200,000 tokens during mid-training.

The model incorporates DeepSeek Sparse Attention (DSA), which combines latent attention representations with a learned indexer that selects a limited set of relevant positions rather than calculating full attention over every previous token at every sparse-attention layer. Conceptually, ordinary scaled dot-product attention is:

$\mathrm{Attention}(\mathbf{𝐐},\mathbf{𝐊},\mathbf{𝐕})=\mathrm{softmax}\left(\frac{\mathbf{𝐐}{\mathbf{𝐊}}^{\mathsf{T}}}{\sqrt{d_k}}\right)\mathbf{𝐕}.$

For sequence length $L$, full attention forms an $L\times L$ score matrix. Sparse selection reduces the number of key positions evaluated by the main attention operation. Actual end-to-end savings depend on the indexer, selected-token count, sequence length, batch size, implementation, communication pattern, and hardware.

GLM-5 also uses Multi-Token Prediction (MTP), in which an auxiliary prediction layer forecasts more than one future token. During inference, those predictions can support speculative decoding, but acceptance depends on agreement with the main model.

IndexShare in GLM-5.2

GLM-5.2 extends the sparse-attention design with IndexShare. Instead of training and executing a separate sparse-attention indexer in every applicable layer, one indexer is reused across each group of four sparse-attention layers. Z.ai reports that this reduces per-token floating-point operations by approximately 2.9 times at a one-million-token context relative to the preceding design.

The public configuration lists 78 Transformer layers, with the first three operating as dense layers and later layers using sparse MoE components. It has 256 routed experts, selects eight per token, and includes one MTP layer.

Z.ai also reports that changes to the MTP layer increased speculative-decoding acceptance length by as much as 20%. Both the 2.9-fold FLOP reduction and 20% acceptance improvement are developer-reported measurements under particular configurations, not universal latency guarantees.

Main language models

GLM-10B and GLM-130B

GLM-10B was an early 10-billion-parameter implementation of the autoregressive blank-infilling framework. It was used to demonstrate that the original pre-training method could scale beyond the sub-billion-parameter models evaluated in the ACL paper.

GLM-130B was released in October 2022 and presented at ICLR 2023. It is a dense bilingual Chinese–English model with approximately 130 billion parameters.

Its main published characteristics include:

• more than 400 billion pre-training tokens, approximately divided between Chinese and English;
• 70 Transformer layers;
• hidden size 12,288;
• 96 attention heads;
• DeepNorm for deep-network stabilization;
• two-dimensional rotary positional embeddings;
• GeGLU feed-forward layers;
• multi-task instruction pre-training on 74 datasets, accounting for approximately 5% of training tokens.

The authors documented numerous loss spikes and the interventions used to continue training, including gradient clipping and Embedding Gradient Shrink. This made the report unusually detailed about training instability at 100-billion-parameter scale.

The project also published an INT4 post-training-quantized checkpoint produced without quantization-aware retraining. The documented deployment examples used four 24-GB RTX 3090 GPUs or eight 11-GB RTX 2080 Ti GPUs. The result should not be generalized to arbitrary four-bit formats or contexts.

The authors reported that GLM-130B exceeded the GPT-3 davinci baseline on selected English evaluations and ERNIE 3.0 Titan on selected Chinese evaluations, but did not reproduce a consistent advantage over OPT-175B and BLOOM-176B. These comparisons reflect the models, prompts, and evaluation harnesses available in 2022.

ChatGLM-6B, ChatGLM2, and ChatGLM3

ChatGLM-6B, released in March 2023, is a 6.2-billion-parameter conversational model trained on approximately one trillion Chinese and English tokens. The initial model had a context of approximately 2,000 tokens and used supervised fine-tuning, self-feedback, and human-preference alignment. Its INT4 edition could be loaded in approximately 6 GB of memory under the documented configuration.

ChatGLM-6B was source-available and distributed with downloadable weights, but its custom model licence imposed conditions, including registration requirements for some commercial uses. It should not be described as equivalent to a model released under an unrestricted OSI-approved software licence.

ChatGLM2-6B, released in June 2023, increased pre-training to approximately 1.4 trillion tokens. It introduced FlashAttention and Multi-Query Attention (MQA), reducing memory use in long-context decoding. A 32,000-token checkpoint was provided alongside shorter-context configurations.

ChatGLM3-6B, released in October 2023, improved instruction following and added structured function calls, a code-interpreter interface, and support for agent-style interaction. These capabilities depended on the surrounding tool runtime as well as the base model.

GLM-4 and GLM-4 All Tools

The GLM-4 family was trained on approximately ten trillion tokens, primarily Chinese and English, with a smaller amount of data from 24 additional languages. The proprietary flagship models included dated versions such as GLM-4-0116 and GLM-4-0520.

The main 2024 variants included:

• GLM-4 — proprietary flagship with a documented 128,000-token context;
• GLM-4-Air — a faster and less expensive hosted model;
• GLM-4-9B — open-weight nine-billion-parameter checkpoints;
• GLM-4-9B-Chat-1M — an experimental one-million-token variant distinct from the standard 9B and 128K checkpoints;
• GLM-4 All Tools — a model and execution system aligned for browser use, Python execution, image generation through CogView, and custom functions.

The existence of a one-million-token GLM-4-9B variant does not mean that every GLM-4 endpoint had a one-million-token context. The proprietary flagship described in the technical report used 128,000 tokens.

GLM-4.5 and GLM-4.5-Air

GLM-4.5 was released on 28 July 2025, followed by its technical report in August. It was presented as an Agentic, Reasoning, and Coding (ARC) foundation model.

The two principal checkpoints are:

• GLM-4.5 — approximately 355 billion total and 32 billion active parameters;
• GLM-4.5-Air — approximately 106 billion total and 12 billion active parameters.

Both were trained within a 23-trillion-token programme and support thinking and direct-response modes. Their public configurations use a 131,072-token maximum position setting. GLM-4.5 and GLM-4.5-Air were released under the MIT License.

GLM-4.6 and GLM-4.7

GLM-4.6, released on 30 September 2025, extended the hosted context from 128,000 to approximately 200,000 tokens. Z.ai positioned it as an update for repository-level coding, long-context processing, search, writing, and agent workflows.

GLM-4.7, released on 22 December 2025, retained a 200,000-token context and allowed up to 128,000 output tokens through the hosted service. It improved coding and multi-step execution and formalized interleaved, preserved, and turn-level thinking modes.

GLM-4.7-Flash, released on 19 January 2026, is a 30B-A3B MoE model: approximately 30 billion total parameters and three billion activated parameters. It has a 200,000-token context and is intended for lower-latency local or hosted use.

GLM-5

GLM-5 was released on 12 February 2026. It scales the main open-weight architecture from GLM-4.5's 355B-A32B to approximately 744B-A40B and increases the reported total base-model training budget from 23 trillion to 28.5 trillion tokens.

The training programme comprised approximately 27 trillion tokens in the initial pre-training stage, followed by mid-training that extended context from 4,000 to roughly 200,000 tokens and raised the cumulative total to 28.5 trillion tokens. The data mixture placed increased emphasis on code, reasoning, and long-context agent trajectories.

GLM-5 incorporates:

• DeepSeek Sparse Attention;
• Multi-head Latent Attention components;
• 256 routed experts with eight selected per token;
• Multi-Token Prediction;
• sequential reasoning, agentic, and general reinforcement-learning stages;
• asynchronous rollout and training infrastructure based on the open-source slime framework.

Claims that GLM-5 was trained exclusively on Huawei Ascend processors with MindSpore are not supported by its technical report. The report instead states that the model stack was adapted and optimized across seven Chinese accelerator platforms: Huawei Ascend, Moore Threads, Hygon, Cambricon, Kunlunxin, MetaX, and Enflame. It does not identify one of these platforms as the exclusive hardware used for the complete pre-training run.

GLM-5-Turbo and GLM-5.1

GLM-5-Turbo, released on 15 March 2026, is a hosted derivative optimized for higher-throughput, long-chain agent workloads and tool integration. Public materials do not identify it as a separately downloadable checkpoint with a newly disclosed parameter count.

GLM-5.1, released on 7 April 2026, is an updated long-horizon agentic-engineering model. Its hosted configuration supports approximately 200,000 context tokens and up to 128,000 output tokens. Z.ai reports that in its internal long-running setup the model can continue working on one task for as long as eight hours; this is a product demonstration rather than a guaranteed runtime or success rate for arbitrary environments.

GLM-5.1 emphasizes iterative experimentation, strategy revision, sustained tool use, repository generation, terminal work, and cybersecurity evaluations. Its public configuration remains in the GLM-5 architectural class; Z.ai did not publish a separate technical report assigning it a fundamentally new backbone or a separately audited absolute parameter count.

GLM-5.2

GLM-5.2 was released on 16 June 2026 as the successor to GLM-5.1. It is text-only at the base-model and hosted-API level and is positioned for project-scale software engineering, long-running agents, deep debugging, and workflows that need to preserve constraints across large histories.

Its principal published characteristics are:

• approximately 750 billion total parameters and 40 billion active parameters;
• sparse MoE architecture with 256 routed experts and eight selected per token;
• one-million-token context;
• hosted output limit of 128,000 tokens;
• multiple selectable reasoning-effort levels;
• IndexShare sparse-attention optimization;
• updated Multi-Token Prediction for speculative decoding;
• MIT-licensed downloadable weights;
• support for function calls, structured output, context caching, streaming, and Model Context Protocol integrations.

The one-million-token limit describes accepted context capacity. It does not by itself establish perfect recall, “lossless” use of every token, resistance to adversarial context, or reliable completion of every million-token task. Long-context effectiveness depends on the content distribution, attention selection, tool harness, prompt structure, context management, and evaluation method.

A July 2026 paper from Tsinghua researchers who conducted part of the work while interning at Z.ai identifies GLM-5.2 as a 750B-A40B model and reports the use of Single-Rollout Asynchronous Optimization (SAO) in its agentic reinforcement-learning pipeline.

Summary table

Model	Release	Total / active parameters	Reported training scale	Maximum published context	Main qualification
GLM-130B	2022	130B dense	More than 400B tokens	Approximately 2K	Bilingual dense model; GLM blank-infilling lineage
ChatGLM-6B	2023	6.2B dense	Approximately 1T tokens	Approximately 2K	Conversational model; custom licence
ChatGLM2-6B	2023	6.2B dense	Approximately 1.4T tokens	32K checkpoint	FlashAttention and MQA
ChatGLM3-6B	2023	6.2B dense	Not separately disclosed	32K	Tool calls and code-interpreter interface
GLM-4 flagship	2024	Not disclosed	Approximately 10T-token family	128K	Proprietary hosted dense model
GLM-4-9B	2024	9B dense	Same broad family programme	8K, 128K, and experimental 1M variants	Open-weight compact line
GLM-4.5	2025	355B / 32B	23T tokens	131,072	MoE; hybrid thinking
GLM-4.5-Air	2025	106B / 12B	23T-token programme	131,072	Smaller MoE
GLM-4.6	2025	GLM-4.5-class architecture; no separate total published	Not separately disclosed	200K	Coding and long-context update
GLM-4.7	2025	Approximately 355B / 32B for the open checkpoint	Not separately disclosed	200K	Improved coding and preserved thinking
GLM-4.7-Flash	2026	30B / 3B	Not disclosed	200K	Lightweight MoE
GLM-5	2026	744B / 40B	28.5T tokens	Approximately 200K	DSA, MTP, agentic engineering
GLM-5.1	2026	GLM-5-class; exact total not separately stated	Continued post-training	Approximately 200K	Long-horizon agent update
GLM-5.2	2026	750B / 40B	Not separately disclosed	1,048,576	IndexShare, updated MTP, long-horizon agents

Multimodal and specialized models

GLM-V family

GLM-4V-9B extended the GLM-4-9B language model with a visual encoder for image understanding and document analysis. The initial open checkpoint used a substantially shorter context than later GLM-V models and should not be assigned the 128K or 200K limits of later text models without qualification.

GLM-4.1V-9B-Thinking is a nine-billion-parameter vision-language reasoning model trained for scientific questions, charts, documents, video, graphical interfaces, grounding, and visual coding.

GLM-4.5V is a 106B-A12B MoE vision-language model based on the GLM-4.5-Air scale. Its training programme used broad multimodal pre-training, supervised reasoning data, and Reinforcement Learning with Curriculum Sampling (RLCS), which adjusts sample selection according to domain and difficulty.

GLM-4.6V retained the 106B-A12B scale, added a 128,000-token multimodal context, and introduced native multimodal function calling. GLM-4.6V-Flash is a nine-billion-parameter compact variant.

GLM-5V-Turbo

GLM-5V-Turbo is the latest documented multimodal agent model in the family as of July 2026. It treats visual perception as part of planning, reasoning, tool use, and execution rather than only as an auxiliary image-captioning interface.

Its intended applications include:

• image and video understanding;
• document, table, chart, and diagram analysis;
• graphical-user-interface interaction;
• coding from screenshots or design mock-ups;
• visual debugging;
• multimodal search and tool use;
• end-to-end agent workflows involving visual observations.

The model's public technical report describes its training and agent framework but does not disclose an absolute total parameter count comparable to the 750B-A40B figure published for GLM-5.2.

GLM-4-Voice

GLM-4-Voice is an end-to-end spoken-dialogue model built from a GLM-4-9B language backbone. It uses a speech tokenizer operating at approximately 175 bits per second and was trained on roughly one trillion speech and text tokens. It supports spoken Chinese and English and is designed to preserve paralinguistic information such as speaking style and emotion.

GLM-OCR

GLM-OCR, released on 3 February 2026, is a compact optical-character-recognition and document-understanding model. Z.ai describes it as combining its CogViT visual encoder with a GLM-0.5B encoder–decoder component and a modality-connection layer.

Its intended tasks include text extraction, formula and table recognition, layout understanding, key-information extraction, and transformation of document images into structured representations. It should not be confused with GLM-5V-Turbo, which is a broader multimodal agent model.

Related ecosystem models

• CodeGeeX is a family of multilingual code-generation, completion, translation, and refactoring models. CodeGeeX and CodeGeeX2 contributed techniques and datasets to the wider GLM coding ecosystem but are separate model lines.
• CogVLM and CogAgent are vision-language and interface-agent models associated with Tsinghua and Zhipu AI researchers.
• CogView and GLM-Image are text-to-image and image-editing systems rather than checkpoints of the core text-only GLM sequence.
• WebGLM combines retrieval, web browsing, evidence aggregation, and response generation for web question answering.
• AutoGLM refers to model and agent systems for controlling phones, browsers, and other graphical interfaces.
• GLM-ASR is an automatic-speech-recognition branch.

Using the same corporate or research ecosystem does not imply that these models share identical weights, context limits, licences, or architectures.

Pre-training data and scaling

Early GLM and ChatGLM data

GLM-130B was trained on more than 400 billion tokens, with approximately 200 billion Chinese and 200 billion English tokens. Its instruction-oriented component used 74 datasets and accounted for roughly 5% of the final token mixture.

ChatGLM-6B increased the total to approximately one trillion tokens, and ChatGLM2-6B to approximately 1.4 trillion. The GLM-4 programme expanded this to approximately ten trillion tokens. Its corpus was primarily Chinese and English, with smaller amounts from 24 other languages, and contained web pages, books, encyclopedic material, source code, academic text, question-answer data, and conversational content.

The documented preprocessing pipeline included exact and fuzzy deduplication, quality and safety filtering, language identification, and tokenization. The full source inventory, licensing composition, deduplication keys, and per-language proportions have not been released.

GLM-4.5 and GLM-5 scaling

GLM-4.5 was trained in a multi-stage programme totalling approximately 23 trillion tokens. GLM-5 increased the cumulative base-model total to 28.5 trillion tokens. Its initial pre-training phase used approximately 27 trillion tokens, followed by long-context and agent-focused mid-training.

The increase from GLM-4.5 to GLM-5 therefore involved both architectural scaling—from 355B-A32B to 744B-A40B—and additional data. Public evaluations cannot isolate how much of the improvement came from parameter count, data composition, sparse attention, optimization, post-training, tool harnesses, or increased test-time compute.

Z.ai has not published a new cumulative pre-training-token total for GLM-5.1 or GLM-5.2. Those releases should not be assigned an invented token count by assuming another full pre-training run.

Context extension and LongAlign

Context limits increased from approximately 2K in ChatGLM-6B to 32K in ChatGLM2 and ChatGLM3, 128K in the GLM-4 flagship, 200K in GLM-4.6 through GLM-5.1, and one million tokens in GLM-5.2.

The GLM-4 research programme used LongAlign, a long-context alignment recipe combining long-instruction data, varied sequence lengths, efficient attention, and long-sequence fine-tuning. GLM-5 used a distinct mid-training stage that progressively extended the supported length from 4K to 200K while emphasizing code repositories and long agent trajectories.

A nominal context window specifies the maximum accepted sequence length under a configuration. It does not establish equal accuracy at every position, complete retention of all details, or constant computational cost.

Post-training and alignment

Supervised fine-tuning and preference optimization

Post-training methods differ across releases. The ChatGLM and GLM-4 reports describe combinations of:

• supervised fine-tuning on instruction and dialogue data;
• human-preference data;
• Proximal Policy Optimization;
• Direct Preference Optimization;
• self-generated contrastive or corrective examples;
• safety and refusal tuning;
• multi-turn dialogue training;
• tool-use and code-execution trajectories.

The original ChatGLM report discusses methods such as Self-Contrast, in which a model helps generate negative examples, and ChatGLM-Math, which uses self-critique and correction for mathematical tasks. These are research and training methods rather than guaranteed inference-time modules in every later endpoint.

GLM-4.5 post-training

GLM-4.5 used multi-stage post-training with specialized expert models, iterative data generation, supervised fine-tuning, preference optimization, and reinforcement learning across reasoning, coding, and agent tasks.

The hybrid thinking/direct-response behaviour was included in this post-training programme. The two modes share one underlying model rather than representing two independently pretrained backbones.

GLM-5 reinforcement-learning pipeline

GLM-5 used three sequential reinforcement-learning phases:

1. Reasoning RL — mathematical, scientific, coding, and other tasks with checkable or model-judged outcomes.
2. Agentic RL — multi-step interactions with tools, terminals, browsers, and software-development environments.
3. General RL — instruction following, writing, dialogue, and broader preference alignment.

Z.ai reports using on-policy cross-stage distillation to reduce capability loss as the model moved between stages. Its slime-based infrastructure decouples rollout generation from parameter updates so that trajectories can be collected asynchronously.

Single-Rollout Asynchronous Optimization

GLM-5.2's agentic post-training used Single-Rollout Asynchronous Optimization according to the July 2026 SAO paper. Conventional GRPO-style training generates a group of responses for each prompt and waits for the complete group before estimating relative advantages. This creates synchronization delays when trajectory lengths vary substantially.

SAO instead uses one rollout per prompt and updates a learned value model more frequently than the policy. It applies token-level importance sampling with strict two-sided clipping and masks tokens whose policy ratios fall outside the accepted range. The paper also describes freezing attention parameters in the value model while optimizing its MoE components.

The authors report stable training over 1,000 steps and improvements over their GRPO baselines on coding and mathematical reasoning evaluations. These experiments support the method under the reported setup but do not establish universal superiority for every model scale, environment, or reward distribution.

Distributed training and inference infrastructure

GLM-130B stabilization

The GLM-130B report documented repeated loss spikes, numerical instability, and the operational decisions needed to resume training. Stabilization methods included DeepNorm, gradient clipping, Embedding Gradient Shrink, learning-rate adjustments, and checkpoint recovery.

This history is important because the later absence of publicly reported loss spikes does not prove that frontier training is free of instability; model developers may use different monitoring, restart, and reporting policies.

Asynchronous agent training

GLM-4.5 used decoupled rollout engines for reinforcement-learning data generation. GLM-5 expanded this into the slime asynchronous infrastructure, separating training workers from inference workers and allowing completed trajectories to enter optimization without waiting for the longest samples in a synchronized batch.

Asynchrony improves utilization but creates policy lag: some trajectories are generated by older model versions. SAO's token-level correction and clipping mechanisms are intended to control this mismatch in GLM-5.2 training.

Hardware portability

Z.ai reports full-stack adaptation of GLM-5 across Huawei Ascend, Moore Threads, Hygon, Cambricon, Kunlunxin, MetaX, and Enflame accelerator platforms. The wording refers to kernel, framework, and inference-stack optimization. It is not evidence that every stage of pre-training was performed exclusively on all—or any one—of these platforms.

Open checkpoints also support GPU-oriented inference software. Framework availability varies by model and version and includes:

• vLLM;
• SGLang;
• Transformers;
• KTransformers;
• xLLM;
• Unsloth for selected GLM-5.2 workflows;
• vLLM-Ascend and other Ascend-specific runtimes.

A supported framework does not imply that consumer hardware can hold the complete checkpoint.

Reported benchmark results

Benchmark results in this section are primarily developer-reported. Scores depend on checkpoint date, prompts, sampling, maximum output length, tool access, context-management strategy, repository state, judge model, timeout, retry policy, and agent framework. Results from different reports are not necessarily directly comparable.

ChatGLM progression

The 2024 ChatGLM report gives the following results for selected BF16 checkpoints under its evaluation setup:

Benchmark	ChatGLM-6B	ChatGLM2-6B	ChatGLM3-6B	GLM-4-9B
GSM8K	1.5	25.9	72.3	84.0
MMLU	25.2	45.2	61.4	74.7
HumanEval	0.0	9.8	58.5	70.1
C-Eval	23.7	51.7	69.0	77.1

The values illustrate the reported development trend but mix changes in pre-training, post-training, data, architecture, and prompting. They do not isolate the effect of any one component.

GLM-4

Selected GLM-4-0520 results reported in the 2024 technical paper are:

Benchmark	GLM-4-0520
MMLU	83.3
GSM8K	93.3
MATH	61.3
BBH	84.7
GPQA	39.9
HumanEval	78.5

In the same report, GLM-4-0520 exceeded the paper's GPT-4-0314 baseline on GSM8K, MATH, BBH, GPQA, and HumanEval in the reported harness, but remained below several later GPT-4 Turbo and Claude 3 results. The comparison is a dated vendor evaluation rather than a permanent model ranking.

The report also gave GLM-4-0520 an AlignBench v1.1 overall score of 8.00 and a Berkeley Function Calling Leaderboard score of 81.76%. Both leaderboards, prompts, and competing endpoints have changed since 2024.

GLM-4.5, GLM-4.7, and GLM-5

Benchmark	GLM-4.5	GLM-4.7	GLM-5
Humanity's Last Exam, no tools	—	24.8	30.5
Humanity's Last Exam, with tools	—	42.8	50.4
AIME 2024	91.0	—	—
GPQA-Diamond	79.1	85.7	86.0
SWE-bench Verified	64.2	73.8	77.8
SWE-bench Multilingual	—	66.7	73.3
Terminal-Bench 2.0, Terminus-2	—	41.0	56.2–60.7
BrowseComp	—	52.0	62.0
MCP-Atlas public set	—	52.0	67.8

GLM-4.5's technical report additionally gives 70.1% on TAU-Bench, 98.2% on MATH-500, and 84.6% on MMLU-Pro. The paper's claim that GLM-4.5 ranked third overall among its evaluated systems and second on agent benchmarks applies only to its selected twelve-benchmark aggregation at publication time.

GLM-5's Terminal-Bench result varies with the harness and whether a developer-verified correction set is used. A single value should therefore not be quoted without identifying the framework and test version.

GLM-5.1 and GLM-5.2

Selected results from the official GLM-5.1 and GLM-5.2 model cards are:

Benchmark	GLM-5.1	GLM-5.2
Humanity's Last Exam, no tools	31.0	40.5
Humanity's Last Exam, with tools	52.3	54.7
AIME 2026	95.3	99.2
HMMT November 2025	94.0	94.4
HMMT February 2026	82.6	92.5
IMOAnswerBench	83.8	91.0
GPQA-Diamond	86.2	91.2
SWE-bench Pro	58.4	62.1
NL2Repo	42.7	48.9
Terminal-Bench 2.1, Terminus-2	63.5*	81.0
FrontierSWE, dominance score	30.5	74.4
MCP-Atlas public set	71.8	76.8
Tool-Decathlon	40.7	48.2

* The GLM-5.1 model card reports 63.5 using the then-current Terminal-Bench 2.0 Terminus-2 setup. The GLM-5.2 card labels the corresponding newer evaluation Terminal-Bench 2.1; the figures are not a perfectly controlled comparison.

For most GLM-5.2 reasoning evaluations, Z.ai used temperature 1.0, top-p 0.95, and a maximum generation length of 163,840 tokens. The AIME, HMMT, and IMOAnswerBench answers were evaluated with a GPT-5.5-based judge. HLE with tools used up to 300,000 context tokens, while SWE-bench Pro and NL2Repo used 400,000-token contexts.

The use of proprietary judge models, tailored prompts, large reasoning budgets, custom context limits, and changing benchmark versions reduces independent reproducibility. These scores should be treated as evidence from the developer's evaluation system, not as universally comparable model constants.

GLM-4.7-Flash

Z.ai reports the following selected results for the 30B-A3B GLM-4.7-Flash checkpoint:

Benchmark	Score
AIME 2025	91.6
GPQA	75.2
LiveCodeBench v6	64.0
Humanity's Last Exam	14.4
SWE-bench Verified	59.2
Tau2-Bench	79.5
BrowseComp	42.8

The model card notes prompt and environment modifications for parts of Tau2-Bench and recommends preserved-thinking mode for multi-turn agent evaluations.

Applications and ecosystem

Conversational assistants

Z.ai operates consumer and developer-facing assistants through services including Zhipu Qingyan and chat.z.ai. Depending on the selected model, the services support multilingual conversation, long-context document analysis, web search, file handling, code execution, image understanding, and tool calls.

A hosted product may add retrieval, search, execution, memory, moderation, and user-interface components that are not intrinsic properties of the downloadable base checkpoint.

Software engineering

Coding and agentic engineering are central targets of GLM-4.5 and later models. Typical workflows include:

• repository analysis and architectural mapping;
• implementation of changes across multiple files;
• test generation and execution;
• command-line and package-manager use;
• bug diagnosis and iterative repair;
• front-end generation from written or visual specifications;
• cross-language refactoring;
• API migration;
• long-running experimentation and optimization;
• review of logs, build failures, and performance traces.

GLM-5.2 is designed for workflows in which architectural constraints, API contracts, directory structure, tests, and prior decisions must be retained over a large project history. Actual reliability depends on the coding-agent harness, permissions, repository state, test coverage, context policy, and execution sandbox.

Agents and tool use

GLM-4 All Tools combined a browser, Python interpreter, image-generation system, and custom function interface. Later models support structured function calls, streaming tool arguments, context caching, Model Context Protocol connections, and multi-agent orchestration through surrounding frameworks.

Agent systems can perform search, data analysis, software operations, document transformation, and chained workflow execution. The model itself does not supply operating-system permissions or external facts; those are provided by the runtime and tools.

Multimodal workflows

GLM-V models are used for:

• image question answering;
• document OCR and layout analysis;
• charts, tables, diagrams, and scientific figures;
• video understanding;
• graphical-interface navigation;
• visual grounding;
• generating or editing code from interface screenshots;
• visual verification of rendered applications.

GLM-5V-Turbo extends these workflows toward agents that alternate among visual perception, planning, tool invocation, and verification.

Long documents and large codebases

GLM-4-9B-Chat-1M and GLM-5.2 accept contexts of approximately one million tokens. Possible applications include large code repositories, collections of contracts or research papers, extensive technical documentation, long agent histories, and combined source-and-log analysis.

In practice, retrieval, hierarchical summaries, selective context loading, and external memory may remain preferable to placing every available token into one prompt. Very long prompts increase prefill time and can dilute relevant evidence.

Developer platform and deployment

Z.ai offers OpenAI-compatible hosted APIs for language and vision models. Supported features vary by endpoint and include:

• streaming generation;
• thinking modes and configurable reasoning effort;
• function calls;
• structured JSON output;
• web search;
• context caching;
• Model Context Protocol tools;
• batch and agent workflows.

Open-weight checkpoints can be deployed through compatible versions of vLLM, SGLang, Transformers, KTransformers, and other engines. The latest model cards should be consulted because support for a new architecture may initially require unreleased or main-branch framework versions.

Licensing and resource requirements

GLM-4.5, GLM-4.7, GLM-4.7-Flash, GLM-5, GLM-5.1, and GLM-5.2 are distributed under the MIT License according to their official repositories. This permits commercial use, modification, and redistribution subject to the licence notice and warranty disclaimer.

Earlier checkpoints do not necessarily use the same terms. ChatGLM-6B, for example, used a separate model licence and required registration for some commercial deployments. Users should examine the exact licence attached to the selected checkpoint rather than assuming one family-wide policy.

Sparse activation reduces computation for an individual token, but all experts must still be stored, distributed, or offloaded. A 750-billion-parameter checkpoint therefore does not have the storage footprint of a 40-billion-parameter dense model merely because approximately 40 billion parameters are active per token.

Resource use depends on:

• weight precision and quantization;
• context length;
• KV-cache or latent-cache format;
• batch size;
• tensor, pipeline, data, and expert parallelism;
• sparse-attention implementation;
• communication topology;
• CPU or storage offloading;
• speculative decoding;
• framework-specific memory management.

Self-hosting the largest checkpoints typically requires a multi-accelerator or multi-node system. Quantization can reduce memory but may change quality, throughput, supported kernels, and maximum practical context.

Limitations and open problems

Language imbalance

The core GLM training mixtures are dominated by Chinese and English. Although the GLM-4 report mentions data from 24 additional languages, it describes those portions as substantially smaller. Performance can therefore vary sharply across languages, scripts, regional varieties, and culturally specific tasks.

Long-context reliability

Increasing a model's accepted context does not automatically solve:

• retrieval of a small fact among many distractors;
• maintaining chronology across long histories;
• resolving contradictory instructions;
• resisting prompt injection embedded in documents;
• preserving goals through thousands of tool calls;
• maintaining exact numerical or symbolic state;
• recognizing which old information is obsolete.

Sparse attention also introduces an index-selection problem: relevant positions must be selected before the main attention operation can use them. An inaccurate indexer can omit important evidence.

Hallucinations and factuality

GLM models can generate unsupported facts, incorrect citations, fabricated code behaviour, or plausible but invalid explanations. Web search and retrieval can improve grounding, but the model may select weak sources, misread them, or synthesize incompatible claims.

Long explicit reasoning does not guarantee a correct conclusion. Verification should use primary sources, executable tests, formal constraints, or human review appropriate to the task.

Agentic error accumulation

Long-horizon agents introduce failure modes beyond ordinary text generation:

• incorrect tool selection;
• malformed arguments;
• unintended file or system modifications;
• misinterpretation of tool results;
• repeated actions and loops;
• divergence from the original objective;
• propagation of an early false assumption;
• failure to notice a partially completed operation;
• unsafe execution of untrusted code or content.

More tool calls or a longer runtime can increase both the opportunity to recover and the opportunity to compound errors.

Benchmark comparability

Many public results are generated by model developers. Evaluations often differ in:

• maximum context and output length;
• sampling temperature and top-p;
• tool availability;
• context-management strategy;
• agent framework and system prompt;
• timeout and retry policy;
• benchmark revision;
• judge model;
• repository or dependency state.

Internal benchmarks and newly introduced suites can measure relevant workloads but generally offer less independent reproducibility than fixed public datasets with released trajectories and third-party runs.

Synthetic data and reward-model dependence

GLM post-training uses synthetic instructions, generated solutions, executable verifiers, model-based judges, simulated environments, and agent trajectories. These techniques enable large-scale training but raise questions about:

• reward hacking;
• evaluator bias;
• overfitting to synthetic tool specifications;
• contamination between generation and evaluation data;
• self-reinforcement of systematic errors;
• transfer from controlled environments to production systems.

SAO reduces some asynchronous-training problems but does not remove dependence on reward quality, environment design, or value-model accuracy.

Reproducibility and transparency

The open checkpoints permit independent inference, fine-tuning, quantization, and benchmarking. However, the complete pre-training corpora, filtering rules, language proportions, copyrighted-data composition, all post-training environments, reward models, human-preference datasets, and production infrastructure have not been released.

Consequently, third parties cannot exactly reproduce GLM-4.5, GLM-5, GLM-5.1, or GLM-5.2 from the public artefacts alone.

Resource and environmental cost

Large sparse models require substantial accelerator memory, networking, storage, and electricity during both training and serving. Sparse activation lowers some arithmetic costs but introduces routing, synchronization, and expert-communication overhead.

Public reports do not provide enough independently audited information to calculate a complete lifecycle energy or carbon footprint for the frontier GLM checkpoints.

Safety, ethics, and regulation

Safety engineering

The GLM technical reports describe data filtering, supervised safety tuning, preference optimization, adversarial testing, and evaluations such as SafetyBench. These measures can reduce harmful outputs but do not establish that the models are safe for every use or resistant to all jailbreaks.

Deployers of tool-using models may need:

• least-privilege permissions;
• network and filesystem sandboxes;
• command allowlists or denylists;
• secret isolation;
• execution time and resource limits;
• audit logs;
• human approval for irreversible actions;
• application-specific red-team testing.

Open-weight availability enables independent research and local control while also allowing the removal or alteration of vendor safeguards.

Chinese generative-AI regulation

Public-facing GLM services in China operate within the country's rules for algorithms, data, deep synthesis, and generative AI. The Interim Measures for the Management of Generative Artificial Intelligence Services took effect on 15 August 2023 and impose obligations concerning lawful data, personal information, security assessment, and content management.

International deployments may additionally be subject to privacy, copyright, consumer-protection, cybersecurity, export-control, sector-specific, and AI-governance rules in other jurisdictions.

United States export controls

On 16 January 2025, the United States Bureau of Industry and Security added Beijing Zhipu Huazhang Technology Co., Ltd., identified with aliases including Zhipu AI and Beijing Knowledge Atlas Technology Co., Ltd., together with several related entities, to the Entity List. The rule applies a licence requirement to all items subject to the Export Administration Regulations, with a presumption of denial.

The Entity List designation is an export-control measure and should not be presented as a technical finding about model accuracy or safety. It can, however, affect access to hardware, software, cloud services, and international partnerships.

Data, privacy, and intellectual property

GLM applications may process source code, personal documents, images, audio, video, credentials, or confidential business information. Relevant safeguards include data minimization, encryption, access controls, retention limits, consent management, and contractual restrictions on model training or logging.

Generated outputs may reproduce memorized material, create derivative content, or contain licence-incompatible code. Open-weight licensing governs the model files; it does not automatically grant rights to every training item, input, or generated output.

Research directions

Research associated with the GLM family suggests several continuing directions:

• Useful million-token context. Improving retrieval, sparse-attention indexing, context compression, memory, and resistance to goal drift rather than merely increasing the accepted token count.
• Long-horizon agent learning. Developing stable asynchronous reinforcement learning, better value models, verifiers, recovery policies, and online adaptation.
• Efficient MoE systems. Reducing expert-communication cost, load imbalance, memory movement, and multi-node synchronization.
• Speculative decoding. Improving Multi-Token Prediction and acceptance rates without reducing output quality.
• Multimodal agents. Integrating images, video, documents, interfaces, speech, tools, and actions into one verifiable workflow.
• Hardware portability. Maintaining numerical consistency and performance across GPU, NPU, and other accelerator stacks.
• Independent evaluation. Releasing reproducible trajectories, fixed harnesses, multilingual tests, safety evaluations, and real-world agent benchmarks.
• Controllable reasoning effort. Allocating test-time compute according to task difficulty, latency limits, and verification needs.
• Secure tool use. Designing permission systems, sandboxing, provenance tracking, and prompt-injection defenses for autonomous agents.

Z.ai has not published a binding specification or release schedule for a model after GLM-5.2. Claims about an unreleased GLM-5.3 or GLM-6 would therefore be speculative as of 10 July 2026.

External links

• Z.ai official website
• Z.ai developer documentation
• Z.ai organization on Hugging Face
• Z.ai organization on GitHub
• Tsinghua Data Mining organization on GitHub
• GLM-5.2 model card
• GLM-5.1 model card
• GLM-5 model card

See also

• Transformer (deep learning architecture)
• Large language model
• Mixture of experts
• Reinforcement learning from human feedback
• DeepSeek
• Qwen
• BERT
• T5 (Text-to-Text Transfer Transformer)

Literature

• Du, Z.; Qian, Y.; Liu, X.; Ding, M.; Qiu, J.; Yang, Z.; Tang, J. GLM: General Language Model Pretraining with Autoregressive Blank Infilling. ACL, 2022.
• Zeng, A. et al. GLM-130B: An Open Bilingual Pre-trained Model. ICLR, 2023.
• GLM Team. ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv:2406.12793, 2024.
• GLM-4.5 Team. GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models. arXiv:2508.06471, 2025.
• GLM-5 Team. GLM-5: from Vibe Coding to Agentic Engineering. arXiv:2602.15763, 2026.
• Hou, Z.; Li, Y.; Tang, J.; Dong, Y. Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning. arXiv:2607.07508, 2026.
• GLM-V Team. GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning. arXiv:2507.01006, 2025–2026.
• GLM-V Team. GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents. arXiv:2604.26752, 2026.
• Zeng, A. et al. GLM-4-Voice: Towards Intelligent and Human-Like End-to-End Spoken Chatbot. arXiv:2412.02612, 2024.
• Vaswani, A. et al. Attention Is All You Need. Advances in Neural Information Processing Systems 30, 2017.