Use Gemma open models

This guide shows you how to use Gemma open models and covers the following topics:

• Available Gemma models: A list of the Gemma models that you can use with Vertex AI.
• Options for using Gemma models: A comparison of different platforms and services for running Gemma models.
• Gemma model sizes and capabilities: Details about the different sizes and versions available for each Gemma model.

Gemma is a family of lightweight, generative AI open models based on Gemini. You can run Gemma models in your applications, on your hardware, or on hosted services.

Because Gemma models have open weights, you can customize them with fine-tuning to improve their performance on specific tasks. You can tune any Gemma model with the AI framework of your choice and the Vertex AI SDK. To get started, open a fine-tuning notebook example from the Gemma model card in Model Garden.

Available Gemma models

The following Gemma models are available to use with Vertex AI. To learn more about and test the Gemma models, see their Model Garden model cards.

Options for using Gemma models

You can use Gemma models in various environments, including Vertex AI, Google Kubernetes Engine, Dataflow, and Colaboratory.

Use Gemma with Vertex AI

Vertex AI offers a managed platform to build and scale machine learning projects without requiring in-house MLOps expertise. We recommend this option if you want to access end-to-end MLOps capabilities, value-added ML features, and a serverless experience for streamlined development.

You can use Vertex AI as the downstream application that serves the Gemma models. For example, you can port weights from a Keras implementation of Gemma and then use Vertex AI to serve that version to get predictions.

To get started, see the following notebook examples:

Serve models

• Serve Gemma 3n in Vertex AI
• Serve Gemma 3 in Vertex AI
• Serve Gemma 2 in Vertex AI
• Serve Gemma in Vertex AI
• Serve MedGemma in Vertex AI
• Serve MedSigLIP in Vertex AI

Fine-tune models

• Fine-tune Gemma 3 using PEFT and then deploy to Vertex AI from Vertex
• Fine-tune Gemma 2 using PEFT and then deploy to Vertex AI from Vertex
• Fine-tune Gemma using PEFT and then deploy to Vertex AI from Vertex
• Fine-tune Gemma using PEFT and then deploy to Vertex AI from Huggingface
• Fine-tune Gemma using KerasNLP and then deploy to Vertex AI
• Fine-tune Gemma with Ray on Vertex AI and then deploy to Vertex AI

Run local inference

• Run local inference with ShieldGemma 2 with Hugging Face transformers
• Run local inference with T5Gemma with Hugging Face transformers

Use Gemma with GKE

Google Kubernetes Engine (GKE) is the Google Cloud solution for managed Kubernetes that provides scalability, security, resilience, and cost-effectiveness. We recommend this option if you have existing Kubernetes investments, have in-house MLOps expertise, or need granular control over complex AI/ML workloads with unique security, data pipeline, and resource management requirements. To learn more, see the following tutorials in the GKE documentation:

• Serve Gemma with vLLM
• Serve Gemma with TGI
• Serve Gemma with Triton and TensorRT-LLM
• Serve Gemma with JetStream

Use Gemma with Dataflow

You can use Gemma models with Dataflow to run inference pipelines for tasks like sentiment analysis. To learn more, see Run inference pipelines with Gemma open models.

Use Gemma with Colab

You can use Gemma with Colaboratory and framework options such as PyTorch and JAX. To learn more, see:

• Get started with Gemma using Keras.
• Get started with Gemma using PyTorch.
• Basic tuning with Gemma using Keras.
• Distributed tuning with Gemma using Keras.

Gemma model sizes and capabilities

Gemma models are available in several sizes so you can build generative AI solutions based on your available computing resources, the capabilities you need, and where you want to run them. Each model is available in several versions:

• Pretrained: This version of the model has not been trained on any specific tasks or instructions beyond the Gemma core data training set. We recommend that you fine-tune this model before you use it.
• Instruction-tuned: This version of the model was trained with human language interactions to participate in a conversation, similar to a basic chatbot.
• Mix fine-tuned: This version of the model is fine-tuned on a mixture of academic datasets and accepts natural language prompts.

Lower parameter sizes mean lower resource requirements and more deployment flexibility.

Gemma is tested on Google's purpose-built v5e TPU hardware and NVIDIA's L4 (G2 Standard), A100 (A2 Standard), and H100 (A3 High) GPU hardware.

What's next

• See Gemma documentation.