API & TechnicalPythonGoogle ShoppingAPI

How to scrape Google Shopping with Python (step-by-step)

Learn how to scrape Google Shopping with Python. This tutorial covers why scraping it directly is hard, how to resolve an EAN to the correct Google Shopping catalog ID (SKU), single lookups, batch scraping at scale, and multi-country price comparison.

By Tim Hagebols

Why scrape Google Shopping?

Google Shopping aggregates offers from thousands of merchants into a single view of any product. For an e-commerce business that makes it one of the richest pricing sources on the web: one product page can show you every seller, every price, shipping cost, and availability side by side.

Common use cases for scraping Google Shopping include:

  • Competitor price monitoring — See every merchant selling the same product and where you sit in the ranking
  • Price comparison — Pull the full offer set for a product in one call instead of visiting each retailer
  • Multi-country analysis — Compare the same product across Google Shopping's country domains (shopping.google.nl, shopping.google.de, and more)
  • Repricing — Feed live Google Shopping prices into your own repricing engine
  • Catalog enrichment — Add titles, brands, images, and identifiers to your own product data

Scraping Google Shopping directly is one of the harder scraping jobs on the web. This tutorial shows you why, and how to do it reliably with Python and the ShoppingScraper API.

The hard part: matching an EAN to a Google Shopping SKU

Here is the problem almost every guide skips over.

Google Shopping does not identify products by their EAN or GTIN barcode. Internally, each product is keyed by a Google catalog ID (a "Google Shopping SKU"). If you search Google Shopping with a raw EAN, you usually get the wrong product, a mix of unrelated listings, or nothing at all, because Google's own barcode index is sparse and noisy. That single fact breaks most DIY scrapers before they even reach the pricing data.

ShoppingScraper is the only provider on the market that translates a raw EAN/GTIN directly into the correct Google Shopping catalog ID (SKU). You send a barcode, you get back the exact matching Google product, and from there every offer, variant, and review is tied to the right item instead of a lookalike. That resolution step is what makes the rest of this tutorial possible, and it is why you would use the API instead of parsing Google's HTML yourself.

You do not need to know how the matching works to use it. Send an EAN, get the correct product back.

Prerequisites

Before you start, make sure you have:

  1. Python 3.9+ installed
  2. httpx — a modern HTTP client for Python
  3. A ShoppingScraper API key — sign up at shoppingscraper.com for 100 free credits

Install httpx if you do not have it yet:

pip install httpx

Keep your API key out of your source code by storing it in an environment variable:

export SHOPPINGSCRAPER_API_KEY="your-api-key-here"

Why not just scrape Google Shopping directly?

It is worth understanding what you are avoiding. A naive attempt looks like this:

import httpx
from bs4 import BeautifulSoup
 
# This will NOT work reliably — shown to illustrate the problem
resp = httpx.get("https://www.google.com/search?tbm=shop&q=0194253397052")
soup = BeautifulSoup(resp.text, "html.parser")
# ...and here everything falls apart

In practice you immediately hit:

  • No stable EAN mapping — as above, the barcode does not resolve to the right product
  • JavaScript-rendered results — the offers are not in the initial HTML, so requests/httpx alone see an empty page
  • Bot detection and CAPTCHAs — Google blocks datacenter IPs and unusual request patterns fast
  • Constantly changing markup — selectors that work today break next week
  • Proxy and rate-limit management — doing this at any volume means running a residential proxy pool

The ShoppingScraper API handles all of that server-side and returns clean JSON. The rest of this guide uses it.

Single product lookup by EAN

The /offers endpoint takes an EAN and a Google Shopping site, resolves the EAN to the correct Google product, and returns every seller's offer.

import os
import httpx
 
API_KEY = os.environ["SHOPPINGSCRAPER_API_KEY"]
BASE_URL = "https://api.shoppingscraper.com"
 
def get_offers(ean: str, site: str = "shopping.google.nl") -> dict:
    """Fetch all Google Shopping offers for a product by EAN."""
    response = httpx.get(
        f"{BASE_URL}/offers",
        params={"site": site, "ean": ean},
        headers={"x-api-key": API_KEY},
        timeout=httpx.Timeout(connect=5.0, read=60.0, write=5.0, pool=5.0),
    )
    response.raise_for_status()
    return response.json()
 
result = get_offers(ean="0194253397052", site="shopping.google.nl")
 
print(f"Product:    {result.get('title', 'N/A')}")
print(f"Catalog ID: {result.get('sku', 'N/A')}")  # the matched Google Shopping SKU
print(f"Offers:     {len(result.get('offers', []))}")
 
for offer in result.get("offers", []):
    print(f"  {offer.get('sellerName', 'N/A')}: "
          f"{offer.get('price', 'N/A')} {result.get('currency', '')}")

Example response:

{
  "ean": "0194253397052",
  "sku": "3945604264154598042",
  "title": "Apple AirPods Pro (2nd generation)",
  "brand": "Apple",
  "currency": "EUR",
  "offers": [
    {
      "sellerName": "Coolblue",
      "price": "279.00",
      "shippingPrice": "0.00",
      "totalPrice": "279.00",
      "condition": "New",
      "availability": "InStock"
    },
    {
      "sellerName": "Bol.com",
      "price": "284.95",
      "shippingPrice": "0.00",
      "totalPrice": "284.95",
      "condition": "New",
      "availability": "InStock"
    }
  ]
}

Notice the sku field: that is the Google Shopping catalog ID the EAN resolved to. Every offer belongs to that exact product.

Resolving an EAN to its Google Shopping SKU

Sometimes you only want the catalog ID itself, for example to store it alongside your own product data so future lookups skip the matching step. The /match endpoint returns just the resolved Google Shopping SKU for an EAN.

import os
import httpx
 
API_KEY = os.environ["SHOPPINGSCRAPER_API_KEY"]
BASE_URL = "https://api.shoppingscraper.com"
 
def match_ean_to_sku(ean: str, site: str = "shopping.google.nl") -> dict:
    """Resolve an EAN/GTIN to its Google Shopping catalog ID (SKU)."""
    response = httpx.get(
        f"{BASE_URL}/match",
        params={"site": site, "ean": ean},
        headers={"x-api-key": API_KEY},
        timeout=httpx.Timeout(connect=5.0, read=60.0, write=5.0, pool=5.0),
    )
    response.raise_for_status()
    return response.json()
 
match = match_ean_to_sku("0194253397052")
print(f"EAN {match.get('ean')} -> Google SKU {match.get('sku')} ({match.get('title')})")

Once you have stored the SKU, you can pin it on later calls to go straight to the product without re-matching.

Batch scraping at scale with the Channel API

Looping over /offers is fine for a handful of products. For a full catalog, use the Channel API, an asynchronous, queue-based pipeline that ingests large batches and streams results back as they complete. It handles up to 50,000 EANs per batch across 32 Google Shopping markets, and by default resolves each EAN to the matching Google product for you.

Submit a batch:

import os
import httpx
 
API_KEY = os.environ["SHOPPINGSCRAPER_API_KEY"]
BASE_URL = "https://api.shoppingscraper.com"
 
def submit_channel_batch(eans: list[str], site: str = "shopping.google.nl") -> str:
    """Submit a batch of EANs to the Google Shopping Channel API. Returns a batch id."""
    response = httpx.post(
        f"{BASE_URL}/v2/channel/google/offers",
        headers={"x-api-key": API_KEY, "Content-Type": "application/json"},
        json={"site": site, "eans": eans},
        timeout=httpx.Timeout(connect=5.0, read=60.0, write=5.0, pool=5.0),
    )
    response.raise_for_status()
    return response.json()["batch_id"]
 
eans = ["0194253397052", "8710103990741", "4006381333931"]
batch_id = submit_channel_batch(eans)
print(f"Submitted batch {batch_id}")

If you already know a product's Google Shopping SKU, you can pin it to skip the lookup entirely by sending {ean, catalog_id} objects instead of bare EANs. That is the fastest path for catalogs you have matched once already.

Because delivery is at-least-once, dedupe results on (ean, catalog_id) as you consume them. The exact request and response fields are documented in the migration guide — treat that as the source of truth for the batch contract.

Multi-country price comparison

The same EAN resolves across Google Shopping's country domains, so you can compare one product market by market. This runs the lookups concurrently with an asyncio.Semaphore to stay within a sensible request rate.

import asyncio
import os
import httpx
 
API_KEY = os.environ["SHOPPINGSCRAPER_API_KEY"]
BASE_URL = "https://api.shoppingscraper.com"
 
GOOGLE_SHOPPING_SITES = [
    "shopping.google.nl",
    "shopping.google.de",
    "shopping.google.fr",
    "shopping.google.es",
    "shopping.google.co.uk",
    "shopping.google.com",
]
 
async def compare_across_countries(ean: str) -> list[dict]:
    """Compare the lowest Google Shopping price for one product across countries."""
    semaphore = asyncio.Semaphore(25)
 
    async def fetch(client: httpx.AsyncClient, site: str) -> dict:
        async with semaphore:
            try:
                resp = await client.get(
                    f"{BASE_URL}/offers",
                    params={"site": site, "ean": ean},
                    timeout=httpx.Timeout(connect=5.0, read=60.0, write=5.0, pool=5.0),
                )
                resp.raise_for_status()
                data = resp.json()
                offers = data.get("offers", [])
                lowest = min(
                    (float(o["totalPrice"]) for o in offers if o.get("totalPrice")),
                    default=None,
                )
                return {"site": site, "lowest": lowest, "sellers": len(offers),
                        "currency": data.get("currency", "")}
            except httpx.HTTPStatusError:
                return {"site": site, "error": "Not found on this domain"}
 
    async with httpx.AsyncClient(headers={"x-api-key": API_KEY}) as client:
        return await asyncio.gather(*[fetch(client, s) for s in GOOGLE_SHOPPING_SITES])
 
results = asyncio.run(compare_across_countries("0194253397052"))
 
print(f"{'Domain':<24} {'Lowest':<14} {'Sellers':<8}")
print("-" * 46)
for r in results:
    if "error" in r:
        print(f"{r['site']:<24} {'N/A':<14} {'N/A':<8}")
    else:
        price = f"{r['currency']} {r['lowest']}" if r["lowest"] else "N/A"
        print(f"{r['site']:<24} {price:<14} {r['sellers']:<8}")

Example output:

Domain                   Lowest         Sellers
----------------------------------------------
shopping.google.nl       EUR 279.0      6
shopping.google.de       EUR 269.0      11
shopping.google.fr       EUR 274.99     7
shopping.google.es       EUR 269.0      5
shopping.google.co.uk    GBP 229.0      9
shopping.google.com      USD 249.99     14

Full working script

A complete script that reads EANs from the command line, resolves each to its Google Shopping SKU, pulls offers across one or more country domains, stores everything in SQLite, and prints a summary.

"""
Google Shopping price scraper using the ShoppingScraper API.
 
Setup:
    source .venv/bin/activate && pip install httpx
 
Usage:
    export SHOPPINGSCRAPER_API_KEY="your-api-key-here"
    python google_shopping_scraper.py --eans 0194253397052,8710103990741 \
        --sites shopping.google.nl,shopping.google.de
"""
 
import argparse
import asyncio
import os
import sqlite3
import sys
from datetime import datetime, timezone
 
import httpx
 
API_KEY = os.environ.get("SHOPPINGSCRAPER_API_KEY", "")
BASE_URL = "https://api.shoppingscraper.com"
DB_PATH = "google_shopping_prices.db"
 
def validate_ean(ean: str) -> str:
    """Validate and normalize an EAN to 13 digits."""
    ean = ean.strip()
    if not ean.isdigit():
        raise ValueError(f"EAN must be numeric, got: {ean}")
    return ean.zfill(13)
 
def init_db() -> None:
    conn = sqlite3.connect(DB_PATH)
    conn.execute("PRAGMA journal_mode=WAL")
    conn.execute("""
        CREATE TABLE IF NOT EXISTS offers (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            ean TEXT NOT NULL,
            sku TEXT,
            site TEXT NOT NULL,
            title TEXT,
            seller TEXT,
            total_price REAL,
            currency TEXT,
            availability TEXT,
            scraped_at TEXT NOT NULL
        )
    """)
    conn.execute("CREATE INDEX IF NOT EXISTS idx_ean_site ON offers (ean, site, scraped_at)")
    conn.commit()
    conn.close()
 
async def fetch_offers(client, semaphore, ean, site) -> dict:
    async with semaphore:
        try:
            resp = await client.get(
                f"{BASE_URL}/offers",
                params={"site": site, "ean": ean},
                timeout=httpx.Timeout(connect=5.0, read=60.0, write=5.0, pool=5.0),
            )
            resp.raise_for_status()
            return {"ean": ean, "site": site, "data": resp.json(), "ok": True}
        except (httpx.HTTPStatusError, httpx.TimeoutException) as e:
            return {"ean": ean, "site": site, "error": str(e), "ok": False}
 
def store(results: list[dict]) -> int:
    conn = sqlite3.connect(DB_PATH)
    now = datetime.now(timezone.utc).isoformat()
    rows = 0
    for r in results:
        if not r["ok"]:
            continue
        data = r["data"]
        for o in data.get("offers", []):
            conn.execute(
                """INSERT INTO offers
                   (ean, sku, site, title, seller, total_price, currency, availability, scraped_at)
                   VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)""",
                (
                    r["ean"], data.get("sku", ""), r["site"], data.get("title", ""),
                    o.get("sellerName", ""),
                    float(o["totalPrice"]) if o.get("totalPrice") else None,
                    data.get("currency", ""), o.get("availability", ""), now,
                ),
            )
            rows += 1
    conn.commit()
    conn.close()
    return rows
 
async def main(eans: list[str], sites: list[str]) -> None:
    if not API_KEY:
        print("Error: SHOPPINGSCRAPER_API_KEY not set")
        sys.exit(1)
 
    valid = []
    for ean in eans:
        try:
            valid.append(validate_ean(ean))
        except ValueError as e:
            print(f"Skipping: {e}")
 
    if not valid:
        print("No valid EANs")
        sys.exit(1)
 
    print(f"Scraping {len(valid)} EANs across {len(sites)} Google Shopping domains...\n")
    init_db()
 
    semaphore = asyncio.Semaphore(25)
    async with httpx.AsyncClient(headers={"x-api-key": API_KEY}) as client:
        tasks = [fetch_offers(client, semaphore, ean, site)
                 for ean in valid for site in sites]
        results = await asyncio.gather(*tasks)
 
    rows = store(results)
    ok = [r for r in results if r["ok"]]
    print(f"Done. {len(ok)}/{len(results)} lookups succeeded, {rows} offers stored in {DB_PATH}\n")
 
    for ean in valid:
        rows_for_ean = [r for r in ok if r["ean"] == ean]
        if not rows_for_ean:
            continue
        title = rows_for_ean[0]["data"].get("title", "Unknown")
        sku = rows_for_ean[0]["data"].get("sku", "N/A")
        print(f"  {title}  (EAN {ean} -> SKU {sku})")
        for r in rows_for_ean:
            offers = r["data"].get("offers", [])
            lowest = min((float(o["totalPrice"]) for o in offers if o.get("totalPrice")),
                         default=None)
            cur = r["data"].get("currency", "")
            print(f"    {r['site']}: {cur} {lowest} ({len(offers)} sellers)")
        print()
 
if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Scrape Google Shopping via ShoppingScraper API")
    parser.add_argument("--eans", type=str, help="Comma-separated EANs")
    parser.add_argument("--sites", type=str, default="shopping.google.nl",
                        help="Comma-separated Google Shopping domains")
    args = parser.parse_args()
 
    if not args.eans:
        print("Provide --eans")
        sys.exit(1)
 
    ean_list = [e.strip() for e in args.eans.split(",") if e.strip()]
    site_list = [s.strip() for s in args.sites.split(",") if s.strip()]
    asyncio.run(main(ean_list, site_list))

Next steps

You now have everything you need to scrape Google Shopping with Python. Ways to go further:

  • Explore the endpoints — The Google Shopping Scraper API covers offers, match, search, variants, and reviews
  • Skip the search step — The Immersive Product endpoint returns the full product detail view straight from an EAN
  • Scale to a full catalog — The migration guide documents the batch Channel API for up to 50,000 EANs at a time
  • Automate monitoring — Use the built-in schedulers to run recurring price checks without your own cron
  • Compare beyond Google — ShoppingScraper covers 65+ marketplaces, so the same EAN queries Amazon, Bol.com, and dozens of European retailers too
TH
Tim Hagebols

CTO & Co-founder

Full-stack engineer specializing in web scraping, API design, and AI applications for e-commerce. Built ShoppingScraper's infrastructure processing 1M+ daily product lookups.

Related posts

Google Shopping Scraper API

Immersive Product endpoint

Price Monitoring

Ready to try ShoppingScraper?

Start with 100 free API calls. No credit card required.

Start Free Trial