YandexGPT (language model)

YandexGPT is a family of text-oriented large language models (LLMs) developed by Yandex. It was publicly introduced on May 17, 2023, when the first model was integrated into the Alice virtual assistant. At launch, Alice could generate and transform text, propose ideas, write letters, and assist with planning tasks, although Yandex warned that the beta system could produce factual errors.

Yandex subsequently integrated YandexGPT into Search, Browser, Market, advertising tools, customer-support workflows, and other products. Public access to the YandexGPT API entered preview through Yandex Cloud on December 6, 2023. The models and related development tools are currently provided through Yandex AI Studio, previously known as Yandex Foundation Models.

YaLM-100B, released by Yandex in June 2022, preceded YandexGPT in the company's public work on large language models. It contains 100 billion parameters and was trained on approximately 300 billion tokens drawn from a 1.7-terabyte Russian-and-English corpus. YaLM-100B was released under the Apache License 2.0, but it is a separate research model rather than an earlier numbered version of YandexGPT.

YandexGPT uses a mixed distribution model. The Pro checkpoints are proprietary hosted services whose parameter counts, complete architectures, training corpora, and training compute have not been disclosed. The weights of YandexGPT 5 Lite Pretrain and YandexGPT 5 Lite Instruct can be downloaded, but they are governed by the custom YandexGPT-5-Lite-8B License rather than Apache 2.0, MIT, or another conventional permissive open-source license.

The Lite license permits research, non-commercial, and commercial use subject to its conditions. However, it imposes use restrictions, attribution and redistribution requirements, prohibits sublicensing without Yandex's consent, and requires a user who reaches 10 million output tokens in one month to contact Yandex within 30 calendar days and negotiate separate terms. Yandex also does not publish the complete training corpus or a reproducible end-to-end training pipeline. Consequently, open-weight is a more precise general description than "open source" for YandexGPT 5 Lite under definitions that require both model parameters and the preferred form for modifying and reproducing the system.

As of July 11, 2026, the newest numbered YandexGPT release is YandexGPT Pro 5.1, introduced in August 2025. Yandex AI Studio continues to provide YandexGPT Pro 5.1, YandexGPT Pro 5, YandexGPT Lite 5, and customer-fine-tuned versions of YandexGPT Lite. Each of these models has a documented context window of 32,768 tokens.

In November 2025, Yandex introduced the separately branded Alice AI model family as its newer flagship line. Alice AI LLM and Alice AI LLM Flash are listed separately from YandexGPT in the current model catalogue and support 65,536-token contexts. They should not be described as "YandexGPT 6" unless Yandex formally assigns that name to a future release.

Release History

Major Yandex language-model releases

Date	Release	Key Features
June 23, 2022	YaLM-100B	Separate research predecessor with 100 billion parameters. It was trained for approximately 65 days on 800 Nvidia A100 GPUs and consumed about 300 billion tokens from a 1.7-terabyte Russian-and-English corpus. Released under Apache 2.0.
May 17, 2023	YandexGPT	First public YandexGPT model, integrated into Alice. It generated and transformed text, proposed ideas, and supported creative and planning tasks. The initial training process combined filtered public books, articles, and websites with hundreds of thousands of curated responses prepared through crowdsourcing and AI trainers.
September 7, 2023	YandexGPT 2	Added broader task coverage, conversational context, style adaptation, text analysis, restructuring, and synthetic training data. In Yandex's internal side-by-side evaluation, evaluators preferred YandexGPT 2 to the first model in 67% of comparisons. This was a preference rate, not a 67% absolute increase in accuracy.
December 6, 2023	Public YandexGPT API preview	Yandex opened API access to all Yandex Cloud users. The initial service included a larger model for asynchronous processing of complex tasks and a lighter model for real-time requests. More than 20,000 applications had been submitted during the preceding testing period.
March 28, 2024	YandexGPT 3 Pro	Third-generation enterprise model with improved instruction following, document processing, factuality, response formatting, throughput, and customer fine-tuning. In Yandex's internal tests, it was preferred to YandexGPT 2 in 67% of comparisons; the reported factual-error share declined from 16.4% to 13.5%, and unsupported information from 6.4% to 5% on the company's test set.
May 28, 2024	YandexGPT 3 Lite	Lower-cost and lower-latency third-generation model for chatbots, spelling correction, classification, summarization, marketing content, and high-volume text processing. Yandex added curriculum learning, reinforcement-learning alignment, and Grouped-Query Attention. The company reported a six-percentage-point improvement on YaMMLU_ru and a 68% side-by-side preference rate over YandexGPT 2 Lite.
October 24, 2024	YandexGPT 4 Pro / Lite	Fourth-generation models with substantially longer requests, described by Yandex as approximately 60 pages of text, and later documented as a 32K context. They were trained on examples containing step-by-step solutions and added reasoning-oriented behaviour, improved RAG, and generation of commands for external applications. Yandex reported a 70% preference rate for Pro over the previous generation and a decline in errors in its RAG test from 4% to 2.1%.
February 7, 2025	Reasoning mode for YandexGPT Pro	Yandex AI Studio added an optional reasoning mode to the then-current YandexGPT Pro model. The feature was an API and serving option rather than a separate downloadable checkpoint.
February 25, 2025	YandexGPT 5 Pro	Proprietary enterprise model with a 32K context, structured JSON output, improved function calling, RAG, classification, document processing, and OpenAI-compatible API access. Yandex described it as comparable to the tested GPT-4o endpoint and reported that it was preferred to Qwen2.5-32B-Instruct in 64% of comparisons on standard internal service tasks. The 64% figure refers to the Qwen comparison, not to "parity with GPT-4o in 64% of tasks".
February 25, 2025	YandexGPT 5 Lite Pretrain	Downloadable base model with approximately 8 billion parameters and a 32K context. The first training stage used about 15 trillion tokens; a second continued-pre-training stage, called Powerup, used approximately 320 billion higher-quality tokens. The second figure is not the volume of instruction fine-tuning.
March 31, 2025	YandexGPT 5 Lite Instruct	Instruction-tuned 8B model aligned through supervised fine-tuning and RLHF. It supports a 32K context and can be deployed through Transformers, vLLM, llama.cpp, Ollama, and GGUF-compatible runtimes. Yandex reported a 62% side-by-side preference rate over Qwen2.5-7B-Instruct and comparable quality to GPT-4o Mini on selected internal tasks.
May 15, 2025	End of YandexGPT 3 support	Yandex AI Studio ended support for third-generation YandexGPT models and models fine-tuned through the earlier DataSphere workflow.
August 28, 2025	YandexGPT Pro 5.1	Latest numbered YandexGPT Pro release. It improved system-prompt adherence, RAG, text rewriting, document extraction, and interaction with external applications. Yandex reported preference rates of 58% over YandexGPT 5 Pro and 56% over GPT-4.1 on its business-task evaluation. The model does not support Yandex AI Studio's reasoning mode.
August 28, 2025	End of YandexGPT 4 support	Yandex AI Studio ended support for YandexGPT 4 Pro, YandexGPT 4 Pro 32K, YandexGPT 4 Lite, and models fine-tuned from the fourth-generation Lite checkpoint.
November 25, 2025	Alice AI LLM	New separately branded flagship model family for dialogue, RAG, and agent workflows. Earlier YandexGPT models remained available. The current hosted model has a 65,536-token context.
May 28, 2026	Alice AI LLM Flash	Faster and lower-cost Alice AI model for dialogue, classification, document processing, business text, and RAG. It belongs to the Alice AI family rather than representing a YandexGPT 6 release.

Architecture and Training

Yandex has not published the full architecture or parameter counts of the proprietary YandexGPT Pro models. Public statements about the architecture of YandexGPT 5 Lite should therefore not be automatically generalized to Pro 5 or Pro 5.1.

The most detailed public architectural information is available for YandexGPT 5 Lite. Its configuration identifies it as an autoregressive decoder-only transformer compatible with the Hugging Face LlamaForCausalLM implementation. "Llama-compatible" refers to the software architecture and weight format; Yandex states that the model was trained without using weights from third-party models.

Disclosed YandexGPT 5 Lite Architecture

The public configuration specifies:

• 32 transformer layers;
• a hidden-state dimension of 4,096;
• a feed-forward intermediate dimension of 14,336;
• 32 query-attention heads and eight key-value heads;
• a maximum position setting of 32,768 tokens;
• a vocabulary of 129,024 entries;
• SiLU activation;
• RMSNorm with an epsilon of 10⁻⁶;
• Rotary Position Embeddings with a rope_theta value of 500,000;
• untied input and output embeddings;
• FP16 weights in the original published configuration.

The difference between 32 query heads and eight key-value heads implements Grouped-Query Attention (GQA). Four query heads share each key-value head, reducing the memory required for the KV cache and the memory bandwidth needed during autoregressive generation. GQA can improve serving efficiency without reducing the full stored parameter count of the model.

Rotary Position Embeddings encode relative token positions in the attention mechanism. A large RoPE base or maximum-position configuration does not by itself guarantee reliable reasoning or retrieval across every prompt of the stated length; practical long-context quality also depends on training data, attention behaviour, runtime implementation, and the distribution of relevant information within the request.

Pre-training Data

YandexGPT 5 Lite Pretrain was trained in two principal stages:

1. The first stage used approximately 15 trillion tokens and contexts of up to 8K. According to the model card, the mixture consisted of approximately 60% web pages, 15% programming code, and 10% mathematical data. The remainder included synthetic material and data derived from Yandex services such as Yandex Translate and the Search factual database.
2. The second stage, called Powerup, used approximately 320 billion higher-quality tokens and increased the context to 32K. Its reported mixture consisted of approximately 25% web pages, 19% mathematics, 18% code, and 18% educational data, with the remainder composed of synthetic, service-derived, and other selected text.

The published proportions are high-level categories rather than a complete list of source documents. Yandex has not released the full document-level corpus, deduplication rules, filtering code, provenance records, or a reproducible end-to-end training pipeline. Token counts are also dependent on the tokenizer and should not be treated as directly comparable with counts published for models using different vocabularies.

Yandex states that its tokenizer is optimized for Russian. The model card estimates that 32,000 YandexGPT tokens correspond on average to approximately 48,000 Qwen2.5 tokens for the material used in the company's comparison. This is a tokenizer-efficiency comparison, not evidence that the model has a larger semantic context or better long-document reasoning than every 48K-token system.

Alignment and Post-training

YandexGPT 5 Lite Instruct was created from the Pretrain checkpoint through supervised fine-tuning and reinforcement learning from human feedback. The model card states that the Lite alignment process was shared with YandexGPT 5 Pro.

For YandexGPT 5 Pro, Yandex reported using:

• Direct Preference Optimization (DPO);
• Proximal Policy Optimization (PPO);
• a proprietary variation called LogDPO, intended to improve training stability;
• the YaFSDP distributed-training library, which the company said reduced required computing resources by approximately 25%.

These are developer-reported details. Yandex has not published a complete technical report describing the exact reward models, preference datasets, hyperparameters, total post-training token count, or ablation results for the proprietary Pro model.

The Instruct model includes baseline behavioural and ethical alignment but can be further fine-tuned. Additional fine-tuning may improve task performance while also altering refusal behaviour, safety characteristics, response style, and factual reliability.

Tokenization and Chat Template

Yandex recommends using the original supplied SentencePiece tokenizer for exact compatibility. The company's infrastructure tokenizes individual dialogue turns separately, substitutes certain newline representations, and uses a non-standard chat template in which the model generates a single assistant turn following a designated assistant separator.

Consequently, a generic Llama chat template can produce results that differ from the intended YandexGPT 5 Lite dialogue format even though the model is implemented through a Llama-compatible architecture. Deployment frameworks should use the tokenizer and template published with the selected checkpoint.

Context and Limits

The current documented context windows are:

• YandexGPT Pro 5.1: 32,768 tokens;
• YandexGPT Pro 5: 32,768 tokens;
• YandexGPT Lite 5: 32,768 tokens;
• fine-tuned YandexGPT Lite: 32,768 tokens.

Older values such as 7,400 tokens, sometimes found in third-party LangChain wrappers, should not be treated as the architectural context limit of the current models. Similarly, the current Yandex AI Studio limits page lists a 1,000-token maximum response specifically for the AI Playground interface; it does not establish a universal 1,000- or 2,000-token output ceiling for every API request.

API clients can specify output limits through the relevant generation parameters, subject to the selected endpoint, model context, service configuration, and quotas. The sum of prompt, conversation history, tool definitions, retrieved context, and generated output must fit the model and API constraints.

A 32K context window indicates the maximum accepted token sequence. It does not guarantee perfect recall, consistent attention, or correct reasoning across every 32K-token document. Performance can depend on where information appears, how much irrelevant material is included, and whether the request requires retrieval, synthesis, or multi-step calculation.

Current Models (July 2026)

Model	Parameters	Context	License / Access	Notes
YandexGPT Pro 5.1	Not disclosed	32,768	Proprietary hosted service	Latest numbered YandexGPT Pro model. Available through Yandex AI Studio's native text-generation APIs and OpenAI-compatible APIs. Intended for business text processing, RAG, document extraction, rewriting, and interaction with external applications. It does not support AI Studio's reasoning mode.
YandexGPT Pro 5	Not disclosed	32,768	Proprietary hosted service	Supports structured responses, function calling, RAG, classification, summarization, document processing, and OpenAI-compatible access. It remains available through an explicit versioned model URI.
YandexGPT Lite 5	Approximately 8B for the downloadable checkpoint	32,768	Hosted API and downloadable weights under the YandexGPT-5-Lite-8B License	Lower-latency model. Pretrain and Instruct checkpoints are available on Hugging Face. Quantized GGUF versions support llama.cpp- and Ollama-compatible local deployment. Yandex does not separately disclose whether every hosted Lite implementation is identical to the published checkpoint.
Fine-tuned YandexGPT Lite	Based on YandexGPT Lite	32,768	Customer-specific model in Yandex AI Studio	AI Studio supports LoRA-based adaptation for formatting, rewriting, classification, extraction, and related tasks. The service remains marked as Preview in the current documentation.
YaLM-100B	100B	2,048 in the published research configuration	Apache License 2.0	Historical 2022 research model, separate from YandexGPT. The repository includes weights and inference code but states that the published code is not the original training code.

The YandexGPT 5 Lite license grants a worldwide, royalty-free right to use, modify, and distribute the materials for research, non-commercial, and commercial purposes subject to its conditions. It is personal, non-transferable, and not sublicensable without prior written consent.

If use reaches 10 million generated output tokens in one month, the licensee must contact Yandex within 30 calendar days after that month to agree on continued-use terms. If the licensee does not contact Yandex or no agreement is reached, the license terminates according to the timetable in the agreement. Redistribution must include the license and the prescribed copyright notice.

The agreement also restricts specified illegal, privacy-invasive, discriminatory, pornographic, intellectual-property-infringing, and other prohibited uses. It states that the materials are provided without warranty and are not intended for applications where failure could reasonably cause physical injury or violations of human rights.

Benchmarks

Most headline YandexGPT performance results were produced by Yandex. They use company-selected prompts, internal business scenarios, Russian adaptations of public benchmarks, and human side-by-side evaluation. They should be distinguished from independent audits and should not be combined into a single cross-generation leaderboard.

Selected developer-reported comparisons

Model	Comparison	Reported Result	Scope and Qualification
YandexGPT 2	YandexGPT 1	Preferred in 67% of comparisons	Internal side-by-side test covering text generation, rewriting, idea generation, style adaptation, and factual questions.
YandexGPT 3 Pro	YandexGPT 2	Preferred in 67% overall; 69% on user requests and 60% on business requests	Internal side-by-side evaluation. Yandex separately reported reductions in factual and unsupported statements on a difficult business-oriented test set.
YandexGPT 3 Lite	YandexGPT 2 Lite	Preferred in 68% of comparisons; +6 percentage points on YaMMLU_ru	Internal comparison and a Russian-language adaptation of MMLU.
YandexGPT 4 Pro	YandexGPT 3	Preferred in 70% of comparisons	Yandex also said the model approached GPT-4o on selected open questions. The company did not claim universal equivalence across every benchmark or application.
YandexGPT 5 Pro	Qwen2.5-32B-Instruct	Preferred in 64% of comparisons	Standard internal Yandex service tasks. Yandex separately described the model as comparable to GPT-4o; the 64% figure applies to the Qwen comparison.
YandexGPT 5 Lite Instruct	Qwen2.5-7B-Instruct	Preferred in 62% of comparisons	Yandex's blind side-by-side test. The company also described the model as comparable to GPT-4o Mini on selected standard internal tasks.
YandexGPT Pro 5.1	YandexGPT 5 Pro / GPT-4.1	Preferred in 58% / 56% of comparisons	Yandex's business-task evaluation. The company reported that the share of answers rated as good rose from 60% to 71%, while fabricated or incorrect answers fell to 16%. The 16% value applies to the specified test set and is not a universal hallucination rate.

YandexGPT 5 Lite model cards also contain results on public and Russian-adapted benchmarks such as MMLU, HumanEval, BBH, IFEval, RuFacts, and other evaluations. The reported scores depend on the checkpoint, number of examples supplied in the prompt, decoding settings, and the evaluation implementation.

The existence of a crowdsourced platform such as ru-LLM Arena does not establish a permanent first-place position for YandexGPT. Arena ratings change as models, votes, system prompts, and evaluation rules change. Any claim about an Arena position should identify the model, leaderboard date, number of votes, and evaluation configuration.

Fine-tuning

Yandex AI Studio supports Low-Rank Adaptation (LoRA) for YandexGPT Lite text-generation models, Lite-based classifiers, and embedding models. The hosted fine-tuning feature remains in Preview as of July 11, 2026.

The documented use cases include:

• summarizing and rewriting texts;
• generating questions and answers from supplied text;
• producing a specified output format or writing style;
• classifying text, queries, and conversations;
• extracting named entities and structured fields;
• adapting text classifiers and embedding models.

The AI Studio documentation explicitly states that fine-tuning should not be treated as a method for inserting a changing knowledge base into the model. For current factual information, corporate documentation, catalogues, policies, or frequently updated data, retrieval-augmented generation is generally more appropriate.

AI Studio accepts fine-tuning datasets of up to 5 GB by default. A tuning operation can take approximately one to 24 hours depending on dataset size and platform load. The resulting model receives its own identifier and can be invoked through the text-generation API.

The downloadable Pretrain and Instruct checkpoints can also be adapted independently with compatible machine-learning frameworks, subject to the YandexGPT-5-Lite-8B License. Yandex's Pretrain model card includes an example using the torchtune framework. Developers are responsible for validating any fine-tuned derivative because adaptation can affect factual quality, safety, formatting, and instruction following.

API Modes

The initial public API distinguished between a larger model for asynchronous processing and a lighter real-time model. That December 2023 arrangement no longer fully describes the current platform.

Yandex AI Studio now provides two main groups of interfaces:

1. OpenAI-compatible APIs, including Models, Chat Completions, Conversations, Responses, Realtime, Files, Embeddings, and Vector Stores.
2. Yandex-specific APIs for text and image generation, classification, embeddings, batch inference, datasets, fine-tuning, and MCP server management.

The Responses API can generate text, return structured responses, call tools, implement RAG, and participate in multi-agent workflows. Conversations API provides server-side management of dialogue history and context. Realtime API is intended for low-latency text and audio streaming, while Batch Inference is intended for large asynchronous workloads.

These platform capabilities should be distinguished from the capabilities of a model in isolation. For example:

• RAG requires a retrieval index or external document source;
• file handling is supplied by platform APIs;
• speech input and output can involve separate Speech Realtime or SpeechKit components;
• function calling produces structured tool requests, but the surrounding application must validate arguments and execute the tool;
• MCP connectivity is provided by the surrounding agent and gateway infrastructure rather than by the static model weights alone.

The choice between synchronous, streaming, asynchronous, and batch processing depends on latency, throughput, request length, cost, and application architecture. It is inaccurate to state that every Lite request is necessarily synchronous or that every Pro request is necessarily asynchronous.

Multimodality

The YandexGPT models listed in the current Yandex AI Studio catalogue are text-generation models. They accept textual model context and produce text. YandexGPT itself should therefore not be described as a natively image-, video-, or audio-understanding model.

Yandex provides multimodal capabilities through separately named models, APIs, and services:

• Alice AI ART and YandexART generate images from textual descriptions;
• Speech Realtime and SpeechKit provide speech recognition, synthesis, and voice-agent functionality;
• Vision OCR extracts text and document structure from images;
• third-party vision-language models available through AI Studio can process images;
• retrieval and file APIs can supply text extracted from documents to YandexGPT.

A consumer application such as Alice can accept voice, files, images, and text by orchestrating multiple models and services. This does not mean that every underlying YandexGPT checkpoint has native multimodal input.

Similarly, using OCR before YandexGPT or using a speech recognizer to convert audio into text is a pipeline-level multimodal capability rather than a property of the text model itself.

Safety, Data Handling, and Limitations

• Factual errors and hallucinations: YandexGPT can generate false, unsupported, outdated, or internally inconsistent statements. Yandex acknowledged factual-error risks in the first public release, and later company evaluations continued to report a non-zero share of fabricated or incorrect answers.
• RAG does not guarantee correctness: Retrieval can supply current or private documents, but a system may retrieve irrelevant material, omit relevant evidence, misread a source, or produce a conclusion that is not supported by the cited text.
• Developer-reported evaluations: Most headline comparisons were produced by Yandex. Results can change with prompts, model versions, decoding parameters, judge instructions, test-set composition, and access to search or tools.
• Training-data transparency: Yandex publishes aggregate token counts and broad data categories for YandexGPT 5 Lite but not the complete source corpus, document-level provenance, filtering code, deduplication process, or fully reproducible training pipeline.
• Language coverage: The open Lite model was trained primarily on Russian and English material. Performance, factual knowledge, cultural coverage, safety behaviour, and tokenization efficiency should be tested separately for other languages.
• Long-context behaviour: A 32K context does not guarantee that the model will reliably use every fact in the request. Retrieval and reasoning quality may decrease as information becomes longer, denser, or more weakly structured.
• Function-calling risks: Model-generated function names and arguments can be incomplete or maliciously influenced by supplied content. Applications should validate schemas, authorize each tool, restrict side effects, and defend against prompt injection.
• Local deployment requirements: An 8B model can run on some personal computers after quantization, but memory use and speed depend on numerical precision, context length, KV-cache size, hardware, and runtime. A 32K request may require substantially more memory than a short prompt.
• License restrictions: The custom Lite license is not equivalent to Apache 2.0 or MIT. It includes a monthly output threshold, attribution duties, use restrictions, and limits on transfer and sublicensing.
• High-impact applications: YandexGPT is not, by default, a validated autonomous decision system for medicine, law, employment, finance, education assessment, public benefits, critical infrastructure, or physical control. Such deployments require domain evaluation, human oversight, access controls, monitoring, and compliance with applicable law.

Yandex AI Studio logs request data by default. The documentation recommends disabling logging for personal, confidential, or otherwise sensitive information. Logging can be disabled by sending the x-data-logging-enabled: false</code header or by using the corresponding SDK setting; requests submitted with logging disabled are not saved on Yandex Cloud servers according to the service documentation.

The common-instance documentation states that requests stored with logging enabled are anonymized and that potentially sensitive information is masked. Organizations should nevertheless evaluate contractual terms, access control, retention, data location, and legal requirements before processing personal, medical, financial, or confidential information.

AI Studio provides a response-moderation system called guardrails. Requests and model responses can be checked by classifiers, phrase dictionaries, and regular expressions. Yandex proprietary models use a system guardrail by default, while customers can create custom guardrails. The feature is currently marked as Preview.

Guardrails can reduce exposure to specified content classes but cannot guarantee that every harmful request or output will be detected. False positives and false negatives depend on the configured sensitivity, language, wording, classifier quality, and application context.

In July 2025, Yandex announced ISO/IEC 42001 certification for the management processes used to develop and provide YandexGPT. ISO/IEC 42001 concerns an organization's artificial-intelligence management system, including governance, documentation, monitoring, and risk-management processes. It does not certify that every model answer is accurate, unbiased, lawful, or safe.

External links

• Official YandexGPT page
• Yandex AI Studio
• Current Yandex AI Studio model catalogue
• Yandex AI Studio API overview
• YandexGPT 5 Lite Pretrain on Hugging Face
• YandexGPT 5 Lite Instruct on Hugging Face
• YaLM-100B repository on GitHub
• YandexGPT — Wikipedia

Literature

• Vaswani, A.; Shazeer, N.; Parmar, N.; et al. (2017). Attention Is All You Need. arXiv:1706.03762.
• Zhang, B.; Sennrich, R. (2019). Root Mean Square Layer Normalization. arXiv:1910.07467.
• Lewis, P.; Perez, E.; Piktus, A.; et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401.
• Su, J.; Lu, Y.; Pan, S.; et al. (2021). RoFormer: Enhanced Transformer with Rotary Position Embedding. arXiv:2104.09864.
• Hu, E. J.; Shen, Y.; Wallis, P.; et al. (2021). LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685.
• Ainslie, J.; Lee-Thorp, J.; de Jong, M.; et al. (2023). GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints. arXiv:2305.13245.
• Rafailov, R.; Sharma, A.; Mitchell, E.; et al. (2023). Direct Preference Optimization: Your Language Model Is Secretly a Reward Model. arXiv:2305.18290.
• Schulman, J.; Wolski, F.; Dhariwal, P.; et al. (2017). Proximal Policy Optimization Algorithms. arXiv:1707.06347.
• Kutalev, A.; Markoff, D. (2024). Investigating on RLHF Methodology. arXiv:2410.01789.
• Matkin, N.; et al. (2024). Comparative Analysis of Encoder-Based NER and Large Language Models for Skill Extraction from Russian Job Vacancies. arXiv:2407.19816.
• Tsanda, A.; Bruches, E. (2024). Russian-Language Multimodal Dataset for Automatic Summarization of Scientific Papers. arXiv:2405.07886.
• Goloburda, M.; et al. (2025). Qorǵau: Evaluating LLM Safety in Kazakh-Russian Bilingual Contexts. arXiv:2502.13640.
• Togmanov, M.; et al. (2025). KazMMLU: Evaluating Language Models on Kazakh, Russian, and Regional Knowledge of Kazakhstan. arXiv:2502.12829.
• Noels, S.; et al. (2025). What Large Language Models Do Not Talk About: An Empirical Study of Moderation and Censorship Practices. arXiv:2504.03803.

References