Gemini 2.X Model Family: Advanced AI Capabilities and Safety Measures

Gemini 2.5 Pro excels at coding tasks, demonstrating a marked improvement over previous models^[1]. Performance on LiveCodeBench increased from 30.5% for Gemini 1.5 Pro to 69.0% for Gemini 2.5 Pro, while that for Aider Polyglot went from 16.9% to 82.2%^[1]. Performance on SWE-bench Verified went from 34.2% to 67.2%^[1]. The model obtained an increase of over 500 Elo over Gemini 1.5 Pro on the LMArena WebDev Arena, resulting in meaningful enhancements in practical applications, including UI and web application development, and the creation of sophisticated agentic workflows^[1]. In addition to coding performance, Gemini 2.5 models are noticeably better at math and reasoning tasks than Gemini 1.5 models^[1]. Performance on AIME 2025 is 88.0% for Gemini 2.5 Pro compared to 17.5% for Gemini 1.5 Pro, while performance on GPQA (diamond) went from 58.1% for Gemini 1.5 Pro to 86.4%^[1]. Image understanding has also increased significantly^[1].

The Gemini 2.5 models are sparse mixture-of-experts (MoE) transformers with native multimodal support for text, vision, and audio inputs^[1]. Developments to the model architecture contribute to the significantly improved performance of Gemini 2.5 compared to Gemini 1.5 Pro^[1]. The Gemini 2.5 model series makes considerable progress in enhancing large-scale training stability, signal propagation and optimization dynamics, resulting in a considerable boost in performance straight out of pre-training compared to previous Gemini models^[1]. The pre-training dataset is a large-scale, diverse collection of data encompassing a wide range of domains and modalities, which includes publicly available web-documents, code (various programming languages), images, audio (including speech and other audio types) and video, with a cutoff date as June 2024 for 2.0 and January 2025 for 2.5^[1]. The model family is the first to be trained on TPUv5p architecture, employing synchronous data-parallel training to parallelise over multiple 8960-chip pods of Google’s TPUv5p accelerators, distributed across multiple datacenters^[1].

Significant advancements have been made in post-training methodologies, driven by a consistent focus on data quality across the Supervised Fine-Tuning (SFT), Reward Modeling (RM), and Reinforcement Learning (RL) stages^[1]. A key focus has been leveraging the model itself to assist in these processes, enabling more efficient and nuanced quality control^[1]. Furthermore, the training compute allocated to RL has been increased, allowing deeper exploration and refinement of model behaviors^[1]. This has been coupled with a focus on verifiable rewards and model-based generative rewards to provide more sophisticated and scalable feedback signals^[1]. Past Gemini models produce an answer immediately following a user query, which constrains the amount of inference-time compute (Thinking) that our models can spend reasoning over a problem^[1]. Gemini Thinking models are trained with Reinforcement Learning to use additional compute at inference time to arrive at more accurate answers^[1]. The resulting models are able to spend tens of thousands of forward passes during a “thinking” stage, before responding to a question or query^[1].

Gemini 2.0 and 2.5 represent a strategic shift of development priorities towards delivering tangible real-world value, empowering users to address practical challenges and achieve development objectives within today’s complex, multimodal software environments^[1]. In pre-training, focus was intensified on incorporating a greater volume and diversity of code data from both repository and web sources into the training mixture^[1]. With Gemini 2.0 and 2.5, capabilities have been expanded to address multimodal inputs, long-context reasoning, and model-retrieved information^[1]. At the same time, the landscape and user expectations for factuality have evolved dramatically, shaped in part by Google’s deployment of AI Overviews and AI Mode^[1]. Modeling and data advances helped improve the quality of million-length context, and internal evaluations were reworked to be more challenging to help steer modeling research^[1]. Gemini’s multilingual capabilities have also undergone a profound evolution since 1.5, stemming from a holistic strategy, meticulously refining pre- and post-training data quality, advancing tokenization techniques, innovating core modeling, and executing targeted capability hillclimbing^[1].

Gemini Deep Research is an agent built on top of the Gemini 2.5 Pro model designed to strategically browse the web and provide informed answers to even the most niche user queries^[1]. The agent is optimized to perform task prioritization, and is also able to identify when it reaches a dead-end when browsing^[1]. Performance of Gemini Deep Research on the Humanity’s Last Exam benchmark has gone from 7.95% in December 2024 to the SoTA score of 26.9% and 32.4% with higher compute (June 2025)^[1].

Google is committed to developing Gemini responsibly, innovating on safety and security alongside capabilities^[1]. The Gemini safety policies align with Google’s standard framework, which prevents the generative AI models from generating specific types of harmful content, including child sexual abuse and exploitation, hate speech, dangerous content, harassment, sexually explicit content, and medical advice that runs contrary to scientific or medical consensus^[1]. Gemini strives to protect users from cyberattacks, for example, by being robust to prompt injection attacks^[1]. Compared to Gemini 1.5 models, the 2.0 models are substantially safer^[1]. New models are more willing to engage with prompts where previous models may have over-refused, and this nuance can impact automated safety scores^[1].

Automated red teaming (ART) is used to dynamically evaluate Gemini at scale, allowing for significantly increased coverage and understanding of potential risks^[1]. Formulated as a multi-agent game between populations of attackers and the target Gemini model, the goal of the attackers is to elicit responses from the target model which satisfy some defined objectives^[1]. The generality of this approach has allowed rapid scaling of red teaming to a growing number of areas, including policy violations, tone, helpfulness, and neutrality^[1]. There is also an evaluation that measures Gemini’s susceptibility to indirect prompt injection attacks where a third party hides malicious instructions in external retrieved data in order to manipulate Gemini into taking unauthorized actions through function calling^[1].

The Gemini 2.X model family memorizes long-form text at a much lower rate than prior models^[1]. Moreover, a larger proportion of text is characterized as approximately memorized by the Gemini 2.0 Flash-Lite and Gemini 2.5 Flash models in particular, which is a less severe form of memorization^[1]. We observed no personal information in the outputs characterized as memorization for Gemini 2.X model family models; this indicates a low rate of personal data in outputs classified as memorization that are below our detection thresholds^[1].

Gemini 2.5 Pro was evaluated against the Critical Capability Levels defined in the Frontier Safety Framework, which examines risk in CBRN, cybersecurity, machine learning R&D, and deceptive alignment^[1]. Based on these results, it was found that Gemini 2.5 Pro (up to version 06-17) does not reach any of the Critical Capability Levels in any of these areas^[1]. The evaluations did reach an alert threshold for the Cyber Uplift 1 CCL, suggesting that models may reach the CCL in the foreseeable future^[1]. Consistent with the FSF, a response plan is being put in place which includes testing models’ cyber capabilities more frequently and accelerating mitigations for them^[1].

The staggering performance improvement attained over the space of just one year points to a new challenge in AI research: namely that the development of novel and sufficiently challenging evaluation benchmarks has struggled to keep pace with model capability improvements, especially with the advent of capable reasoning agents^[1]. Being able to scale evaluations in both their capability coverage and their difficulty, while also representing tasks that have economic value, will be the key to unlocking the next generation of AI systems^[1].