Gemini API Image Analysis with Python

Gemini models can accept both text and images as input. With Python, we can send a photograph, scanned document, handwritten note or other image together with instructions asking Gemini to describe, classify, summarize or extract useful information from the image.

This tutorial uses the current google-genai Python SDK. We will run the first examples in Google Colab, securely load the Gemini API key from Colab Secrets, and later run similar code from a local Python environment.

Install the Google GenAI Python SDK 🔝

For new Gemini API Python projects, install the google-genai package.

!pip install -q --upgrade google-genai pillow requests

Import the Gemini SDK with:

from google import genai

Store the Gemini API Key in Colab Secrets 🔝

Create your Gemini API key through Google AI Studio and save it in Google Colab Secrets using this name:

GEMINI_API_KEY

Read the secret and create the Gemini client:

from google import genai
from google.colab import userdata

api_key=userdata.get('GEMINI_API_KEY')
client=genai.Client(api_key=api_key)

See the Gemini API tutorial for more details about API keys, text generation, streaming and chat.

Step 1: Download an Image from a URL 🔝

Python's requests library can download image data from a web URL.

import requests

image_url='https://www.go2india.in/upimg/9565.jpg'
response=requests.get(image_url, timeout=20)
response.raise_for_status()
image_data=response.content

print('Downloaded bytes:', len(image_data))

raise_for_status() raises an exception if the server returns an HTTP error instead of silently treating the error page as image data.

Step 2: Convert the Download into a Pillow Image 🔝

from PIL import Image
from io import BytesIO

image=Image.open(BytesIO(image_data))
print(image.size)
print(image.format)

The resulting Pillow Image object can be displayed in Colab or passed directly with text instructions to Gemini.

Step 3: Analyze the Image with Gemini 🔝

Pass both the instruction and the Pillow image to generate_content().

from google import genai
from google.colab import userdata

client=genai.Client(api_key=userdata.get('GEMINI_API_KEY'))

response=client.models.generate_content(
    model='gemini-3.7-flash',
    contents=['Describe this image. Mention the main visible objects, setting and activity.', image]
)

print(response.text)

The response is generated from the image and prompt. The wording can vary between requests.

Better Prompts Produce More Useful Results

A broad prompt:

Describe this image.

can be replaced with a more task-specific instruction:

Describe the main objects, location, visible activity and important visual details in this image. Do not guess details that cannot be supported by the image.

This makes the intended output clearer.

Step 4: Display the Image and Formatted Response 🔝

For notebook display, create a copy and resize only the preview.

from IPython.display import display, Markdown

preview=image.copy()
preview.thumbnail((512, 512))
display(preview)

response=client.models.generate_content(
    model='gemini-3.7-flash',
    contents=['Describe this image clearly in about 100 words.', image]
)

display(Markdown(response.text))

Using a separate preview avoids changing the original image object that will be sent to Gemini.

Run the Plus2net Image Analysis Notebook 🔝

Run the tutorial notebook directly in Google Colab or view and download the .ipynb source from GitHub.

Upload an Image from Your Computer to Colab 🔝

from google.colab import files

uploaded=files.upload()

Select an image from your computer. If the uploaded filename is hand-written-text.jpg, load it with Pillow:

from PIL import Image

image=Image.open('hand-written-text.jpg')

Read Handwritten Text from an Image 🔝

For handwriting or document images, ask explicitly for transcription rather than using a general description prompt.

from PIL import Image
from google import genai
from google.colab import userdata
from IPython.display import display

image=Image.open('hand-written-text.jpg')
display(image)

client=genai.Client(api_key=userdata.get('GEMINI_API_KEY'))

prompt='''Transcribe the handwritten text in this image.
Preserve the line order where possible.
If any word is unclear, mark it as [unclear] instead of guessing.'''

response=client.models.generate_content(
    model='gemini-3.7-flash',
    contents=[prompt, image]
)

print(response.text)

Useful Image Analysis Prompts 🔝

General Description

Describe the important visible elements of this image in 100 words.

Extract Visible Text

Transcribe all clearly readable text from this image. Preserve the order and do not invent missing words.

Summarize a Document Image

Read the visible document and summarize its main points. Mention any text that is too unclear to read.

Structured Observation

Analyze this image and return:
1. Main objects
2. Setting
3. Visible activity
4. Readable text
5. Details that are uncertain

Photo Caption

Write a factual two-sentence caption describing only what is visibly supported by the image.

Control Temperature and Output Length 🔝

The Google GenAI SDK supports generation configuration through GenerateContentConfig.

from google.genai import types

response=client.models.generate_content(
    model='gemini-3.7-flash',
    contents=['Describe this image concisely.', image],
    config=types.GenerateContentConfig(
        temperature=0.2,
        max_output_tokens=300
    )
)

print(response.text)

temperature influences response variation. A lower value is often appropriate when the task is factual transcription or image description.

max_output_tokens limits the maximum generated output size. It does not mean an exact word count.

Use Gemini Image Analysis from Local Python 🔝

For VS Code or another local Python environment, store the API key in a .env file.

.env

GEMINI_API_KEY=your_actual_api_key_here

Add .env to .gitignore when using Git.

Install Packages

pip install --upgrade google-genai pillow python-dotenv

Complete Local Example

import os
from PIL import Image
from dotenv import load_dotenv
from google import genai

load_dotenv()
api_key=os.getenv('GEMINI_API_KEY')

if not api_key:
    raise ValueError('GEMINI_API_KEY was not found')

client=genai.Client(api_key=api_key)
image=Image.open('your-image.jpg')

response=client.models.generate_content(
    model='gemini-3.7-flash',
    contents=['Describe the important visible details in this image.', image]
)

print(response.text)

Generate a Travel Photo PDF with Gemini Descriptions 🔝

This example combines Gemini image understanding with ReportLab. Python downloads each travel image, asks Gemini for a short factual description, and creates one PDF page containing the image and description.

The workflow is:

1. download the image,
2. open it with Pillow,
3. send the image to Gemini,
4. receive a short description,
5. resize the image for the PDF page,
6. draw the image and wrapped text using ReportLab.

Core Gemini Section

response=client.models.generate_content(
    model='gemini-3.7-flash',
    contents=['Write a factual travel-photo description in about 80 words.', image]
)

description=response.text.strip()

Complete Example

import os
import io
import requests
from PIL import Image
from dotenv import load_dotenv
from google import genai
from reportlab.lib.pagesizes import A4
from reportlab.pdfgen import canvas
from reportlab.lib.utils import ImageReader
from textwrap import wrap

load_dotenv()
api_key=os.getenv('GEMINI_API_KEY')

if not api_key:
    raise ValueError('GEMINI_API_KEY was not found')

client=genai.Client(api_key=api_key)

image_urls=[
    'https://www.go2india.in/upimg/9565.jpg',
    'https://www.go2india.in/upimg/9561.jpg',
    'https://www.go2india.in/upimg/9559.jpg'
]

pdf_path='travel_book.pdf'
pdf=canvas.Canvas(pdf_path, pagesize=A4)
page_width, page_height=A4

for url in image_urls:
    try:
        response=requests.get(url, timeout=20)
        response.raise_for_status()

        image=Image.open(io.BytesIO(response.content)).convert('RGB')

        ai_response=client.models.generate_content(
            model='gemini-3.7-flash',
            contents=[
                'Describe this travel image factually in about 80 words. Avoid guessing an exact location unless it is clearly supported by visible evidence.',
                image
            ]
        )

        description=ai_response.text.strip().replace('\n', ' ')

        pdf_image=image.copy()
        max_width=int(page_width-100)
        pdf_image.thumbnail((max_width, 420))

        image_x=(page_width-pdf_image.width)/2
        image_y=page_height-pdf_image.height-70
        pdf.drawImage(ImageReader(pdf_image), image_x, image_y, width=pdf_image.width, height=pdf_image.height)

        text_y=image_y-30
        pdf.setFont('Helvetica', 11)

        for line in wrap(description, width=90):
            if text_y<50:
                break
            pdf.drawString(50, text_y, line)
            text_y-=16

        pdf.showPage()

    except Exception as e:
        print(f'Error processing {url}: {e}')

pdf.save()
print(f'PDF saved as {pdf_path}')

This version uses a relative output filename:

travel_book.pdf

instead of a computer-specific Windows path, making the example easier to run on different systems.

Common Gemini Image Analysis Errors 🔝

ModuleNotFoundError

Install the current package:

!pip install -q --upgrade google-genai

API Key Not Found

Make sure the Colab secret is named exactly:

GEMINI_API_KEY

Image Cannot Be Identified

The downloaded file may not actually contain image data. When downloading from a URL, use:

response.raise_for_status()

before passing its content to Pillow.

Image File Not Found

Check the current working directory:

import os

print(os.getcwd())
print(os.listdir())

Model Not Found

Model availability can change. List available Gemini models instead of copying an outdated model ID from an old tutorial.

for model in client.models.list():
    print(model.name)

Incorrect Handwriting or OCR Output

Try a clearer scan or photograph and use a specific transcription prompt. AI image understanding can still make mistakes, so verify important text manually.

Frequently Asked Questions 🔝

Gemini Image Analysis Summary 🔝

• Use the current google-genai Python SDK for new Gemini API projects.
• Store the API key securely as GEMINI_API_KEY.
• A Pillow Image object can be passed directly to Gemini with a text prompt.
• Use specific prompts for description, transcription, summarization or structured extraction.
• requests.get() can download web images into Python.
• Use raise_for_status() to detect failed image downloads.
• Use display(Markdown(response.text)) for formatted Colab output.
• For handwriting, explicitly ask for transcription and mark unclear text instead of encouraging guesses.
• Verify important AI-extracted text and visual details.
• GenerateContentConfig can control temperature and max_output_tokens.
• The same Gemini workflow can run in Colab or a local Python environment.
• Pillow, Gemini and ReportLab can be combined to create AI-assisted photo PDFs.

Subhendu Mohapatra

Author & Instructor at plus2net

I write and maintain practical tutorials on Python, PHP, SQL, JavaScript, HTML, jQuery, and web development at plus2net. The tutorials focus on clear explanations, working examples, and code that readers can test and adapt while learning.

About the author   Watch on YouTube

Connect: