Google Cloud Run now lets you run Cloud Run jobs with GPUs enabled for up to 12 hours, a Preview feature that landed on October 30, 2026. That makes this the right week to learn the job side of the platform. This tutorial walks through 13 steps, takes about 75 minutes, and ends with a working Python batch pipeline that fans out across parallel tasks, reads secrets safely, and runs every morning on a schedule.
Most Google Cloud Run guides stop at an HTTP container. Ours starts where the request/response model ends. If you need a first service deployment, our guide to setting up Google Cloud Run covers that path. Here we build something different: a sales-digest job that shards CSV files across tasks, writes JSON reports to Cloud Storage, and fires from Cloud Scheduler with a locked-down service account.
You will finish with a deployable project, a Dockerfile on Python 3.15, ten troubleshooting fixes, and a clear sense of when a job beats a service. Every command uses the Google Cloud CLI (gcloud), so nothing here depends on console screens that Google redesigns every few months.
What Cloud Run Jobs Do That Services Do Not
A Cloud Run service waits for HTTP requests, scales with traffic, and keeps no promise about finishing anything. A Cloud Run job runs a container to completion, exits, and reports success or failure. That difference sounds small until you try to run a nightly import on a service. You end up bolting on a fake endpoint, fighting request timeouts, and paying for idle instances while you wait for the next trigger.
Jobs fix that by design. Each execution starts one or more tasks, every task runs your container once, and the platform retries failed tasks up to a limit you set. The container never needs to listen on a port. It reads its environment, does its work, and exits with code 0 or a non-zero code. Google’s current product line also includes worker pools for continuously running pull-based work, and a newer Cloud Run instances option for singleton runtimes, so it helps to see where jobs sit.
| Cloud Run option | Best for | Trigger | Ends on its own? |
|---|---|---|---|
| Service | HTTP APIs, web apps, webhooks | Incoming request | No |
| Job | Batch, ETL, reports, migrations | CLI, API, or Cloud Scheduler | Yes |
| Worker pool | Queue or stream consumers | Pull from a queue | No |
| Job with GPU (Preview) | Model fine-tuning, offline inference | Same as a job | Yes, up to 12 hours |
The cost argument matters as much as the design one. Compare how a scheduled Lambda function, a Cloud Run job, and an Azure container behave when a batch runs for 20 minutes, and the billing models split quickly. Our Lambda vs Cloud Run vs Container Apps pricing comparison breaks down those gaps in dollars. For this tutorial, the point is simpler: you pay for the vCPU and memory a task uses while it runs, then nothing.
Serverless is no longer a niche bet. The Cloud Native Computing Foundation’s 2024 annual survey states that “In 2024, 44% of organizations reported using serverless in production for a few applications.” The same CNCF survey reports that “In 2024, 91% of organizations used containers in production, up from 80% in 2023.” Cloud Run jobs sit at the overlap of those two numbers: a container, run without a cluster.
Real teams use this pattern to cut bills. One engineer wrote up how his team cut a GitHub Actions bill with Cloud Run jobs in early November 2026, moving heavy CI work onto jobs. The same idea applies to data crunching, report builds, image processing, and any task that has a start and an end. For a wider view of the category, our cloud computing coverage tracks the platforms side by side.
What Changed in Cloud Run This Fall
The platform moved fast over the last three months, and several changes touch how you write a job today. Here is the short list, with dates, drawn from Google’s release notes and the coverage that tracks them.
- October 30, 2026: Preview support for Cloud Run jobs with GPUs, with executions allowed to run for up to 12 hours.
- November 2, 2026: Python 3.15 runtime support is listed as Preview in Google Cloud’s release notes.
- October 15, 2026: Temporal Serverless Workers on Cloud Run worker pools entered preview, reportedly letting task-queue demand scale pools down to zero when idle.
- August 27, 2026: Google reportedly introduced Cloud Run instances, dedicated singleton runtimes. Its example puts one always-on instance with 1 vCPU and 1 GiB of memory at $5.70 for 30 days.
- April 5, 2026: Python 3.9 reached its listed decommission date on Cloud Run, according to the runtime lifecycle page.
Two points follow from that list. First, Python 3.15 is the newest stable interpreter. Python.org lists 3.15.2, a security release, as its latest. Google’s own 3.15 runtime is still Preview, which is why this tutorial builds from a Dockerfile rather than relying on the managed buildpack. Second, GPU jobs are Preview too, so treat that capability as something to test, not something to put in a production contract yet. The roadmap tracker at 365Fanatic Cloud logs these shifts as they happen.
If you compare hyperscalers before committing, the broader picture matters. Our AWS vs Google Cloud comparison weighs share, performance, and pricing, and it pairs well with the hands-on work below.
Prerequisites and Versions
You need a Google Cloud account with billing enabled. Cloud Run jobs, Cloud Build, Artifact Registry, Cloud Scheduler, Secret Manager, and Cloud Storage all require a billing account on the project, even when your usage stays small. You also need permission to create service accounts and IAM bindings. If you work inside a company organization, ask for the Project Owner or Editor role on a sandbox project first.
| Tool | Version used | Why it matters |
|---|---|---|
| Python | 3.15.2 (security release) | Newest stable release, security fixes included |
| Google Cloud CLI | Latest release (run gcloud components update) |
Job and scheduler commands change between releases |
| Docker or Podman | Latest stable | Local image test in Step 8 |
| Base image | python:3.15-slim |
Matches the local interpreter |
| OS | Linux, macOS, or Windows with WSL2 | Shell snippets assume a POSIX shell |
| Editor | Any | Only plain text files are needed |
Python 3.10 and older no longer receive security updates. The 3.10 series reached end of life after the final 3.10.22 release, so do not start new projects on it. If your corporate image still pins 3.12 or 3.13, the code below runs unchanged. Only the Dockerfile base tag changes.
Once you move past a single project, codify the pieces in infrastructure as code. Our Terraform tutorial for Google Cloud shows how to describe buckets, service accounts, and Cloud Run resources declaratively, which is the natural next step after this manual walkthrough.
The Project: A Sales Digest Job That Shards Its Work
The project is small enough to finish in one sitting and realistic enough to reuse. A retail team drops daily order exports into a Cloud Storage bucket as CSV files. Each morning at 06:00 UTC, a Cloud Run job wakes up, splits those files across parallel tasks, totals revenue per SKU, writes one JSON report per task, and optionally pings a webhook with a summary. Nothing runs between executions, so nothing bills between executions.
The design leans on three platform features. Task sharding comes from environment variables that Cloud Run injects into every container: CLOUD_RUN_TASK_INDEX and CLOUD_RUN_TASK_COUNT. Secrets arrive as environment variables mounted from Secret Manager, so no key ever sits in the image. Scheduling goes through Cloud Scheduler, which calls the Cloud Run Admin API with a dedicated identity.
sales-digest/
main.py # the worker, one task = one shard of files
make_sample_data.py# generates test CSVs locally
requirements.txt
Dockerfile
.dockerignore
deploy.sh # every gcloud command from Steps 9-13
gs://PROJECT-sales-digest/
incoming/ # input CSVs land here
reports/DATE/ # task-0.json, task-1.json, ...Sharding by file index keeps the code honest. Task 0 takes files 0, 3, 6 and so on when the job runs three tasks. No coordination service is needed, and a retried task redoes exactly the same files, which makes the output idempotent. That last property matters more than any other in batch design, and Step 6 comes back to it.
Steps 1 to 3: Install gcloud, Pick a Project, Enable APIs
Step 1: Install and update the Google Cloud CLI
Install the Google Cloud CLI from Google’s official instructions for your operating system, then confirm it works. Google’s own developer tooling announcements this week, including the Google Cloud developer plugin in the Antigravity Marketplace, still refer to the tool as the Google Cloud CLI with gcloud as its executable, so the name is stable.
gcloud --version
gcloud components update --quiet
gcloud auth login
gcloud auth application-default loginThe first login authorizes the CLI. The second creates application-default credentials, which the local Python test in Step 8 uses to reach Cloud Storage. Skip it and your local run fails with a credentials error that looks unrelated to login.
Step 2: Create or select a project and set defaults
Use a fresh project so cleanup is one command at the end. Replace my-digest-demo with a globally unique ID. Link a billing account before enabling services.
export PROJECT_ID="my-digest-demo-$RANDOM"
export REGION="us-central1"
export BUCKET="${PROJECT_ID}-sales-digest"
gcloud projects create "$PROJECT_ID"
gcloud config set project "$PROJECT_ID"
gcloud config set run/region "$REGION"
gcloud billing accounts list
gcloud billing projects link "$PROJECT_ID" --billing-account=XXXXXX-XXXXXX-XXXXXXThe run/region default saves you from typing --region on every job command. Pick a region near your data. Cloud Storage and Cloud Run in the same region avoid cross-region transfer charges and cut latency on big files. If you plan to try the GPU Preview later, check which regions list GPU support before you commit, because Google staged earlier Cloud Run GPU launches across a limited set of regions.
Step 3: Enable the required APIs
gcloud services enable
run.googleapis.com
cloudbuild.googleapis.com
artifactregistry.googleapis.com
cloudscheduler.googleapis.com
secretmanager.googleapis.com
storage.googleapis.com
logging.googleapis.comEnabling takes one to three minutes on a new project. Each API maps to a specific job in the pipeline. Cloud Build compiles your Dockerfile, Artifact Registry stores the image, Cloud Run runs it, Secret Manager guards the webhook URL, Cloud Scheduler fires the trigger, and Cloud Logging collects the output you will read in Step 11.
| API | Used in step | Failure symptom if missing |
|---|---|---|
run.googleapis.com |
9, 11 | “API has not been used” on deploy |
cloudbuild.googleapis.com |
9 | Source deploy stalls or fails to build |
artifactregistry.googleapis.com |
9 | Image push denied |
secretmanager.googleapis.com |
10 | Secret create fails |
cloudscheduler.googleapis.com |
12 | Scheduler create fails |
storage.googleapis.com |
4 | Bucket commands fail |
Steps 4 and 5: Create the Bucket, Service Accounts, and Sample Data
Step 4: Create the bucket and two service accounts
Least privilege starts here. The job runs as one identity, and Cloud Scheduler triggers it as a second identity. Neither one is the default Compute Engine service account, which carries broad Editor rights on many older projects and is the most common source of accidental over-permission.
gcloud storage buckets create "gs://${BUCKET}"
--location="$REGION"
--uniform-bucket-level-access
gcloud iam service-accounts create digest-runner
--display-name="Sales digest job runtime"
gcloud iam service-accounts create digest-scheduler
--display-name="Sales digest scheduler caller"
export RUNNER_SA="digest-runner@${PROJECT_ID}.iam.gserviceaccount.com"
export SCHED_SA="digest-scheduler@${PROJECT_ID}.iam.gserviceaccount.com"
gcloud storage buckets add-iam-policy-binding "gs://${BUCKET}"
--member="serviceAccount:${RUNNER_SA}"
--role="roles/storage.objectUser"The storage.objectUser role lets the runner read and write objects in this one bucket. It cannot delete the bucket or touch any other. The scheduler account gets no storage rights at all. In Step 12 it receives permission to run exactly one job.
Step 5: Generate and upload sample data
Test data keeps you from debugging against production files. The generator below writes nine small CSVs of fake orders with a fixed random seed, so your output matches the examples later in this tutorial.
# make_sample_data.py
import csv
import random
from pathlib import Path
random.seed(42)
out = Path("sample")
out.mkdir(exist_ok=True)
SKUS = ["SKU-100", "SKU-200", "SKU-300", "SKU-400", "SKU-500"]
PRICES = {"SKU-100": 9.99, "SKU-200": 24.50, "SKU-300": 4.25,
"SKU-400": 129.00, "SKU-500": 59.90}
for day in range(1, 10):
path = out / f"orders-2026-10-{day:02d}.csv"
with path.open("w", newline="") as fh:
writer = csv.writer(fh)
writer.writerow(["order_id", "sku", "qty", "unit_price"])
for i in range(500):
sku = random.choice(SKUS)
writer.writerow([f"{day}-{i}", sku,
random.randint(1, 5), PRICES[sku]])
print("wrote", len(list(out.glob("*.csv"))), "files")python3 make_sample_data.py
gcloud storage cp sample/*.csv "gs://${BUCKET}/incoming/"
gcloud storage ls "gs://${BUCKET}/incoming/"You should see nine objects listed. Each file holds a small batch of orders, so the full dataset is tiny by design. The sharding logic behaves the same on a small dataset as on a huge one, and small data keeps your first execution quick.
Steps 6 and 7: Write the Worker and the Dockerfile
Step 6: Write main.py as an idempotent task
The worker has one job: take its slice of files, total the numbers, and write a report. Four design choices keep it safe on a platform that retries failed tasks.
- Deterministic sharding. Files are sorted by name, and a task keeps the ones where
index % task_count == task_index. A retry picks the same files. - Overwrite, never append. Each task writes
reports/DATE/task-N.jsonin one call. A second attempt replaces the first instead of duplicating rows. - Structured logs. One JSON object per line on stdout. Cloud Logging turns the keys into searchable fields, and
severitymaps to the log level. - Non-zero exit on failure. An uncaught error logs at ERROR and exits with code 1, which tells Cloud Run to retry the task. A failed webhook only logs a warning, because a chat outage should not rerun a finished batch.
"""Sales digest worker: one Cloud Run task processes one shard of CSV files."""
import csv
import io
import json
import os
import sys
import time
import urllib.request
from collections import defaultdict
from datetime import datetime, timezone
from google.cloud import storage
BUCKET = os.environ["BUCKET"]
INPUT_PREFIX = os.environ.get("INPUT_PREFIX", "incoming/")
WEBHOOK_URL = os.environ.get("WEBHOOK_URL", "")
TASK_INDEX = int(os.environ.get("CLOUD_RUN_TASK_INDEX", "0"))
TASK_COUNT = int(os.environ.get("CLOUD_RUN_TASK_COUNT", "1"))
ATTEMPT = int(os.environ.get("CLOUD_RUN_TASK_ATTEMPT", "0"))
REPORT_DATE = os.environ.get("REPORT_DATE") or datetime.now(timezone.utc).strftime("%Y-%m-%d")
def log(severity, message, **fields):
"""Structured JSON on stdout is parsed by Cloud Logging into fields."""
print(json.dumps({"severity": severity, "message": message,
"task": TASK_INDEX, "attempt": ATTEMPT, **fields}), flush=True)
def my_shard(client):
blobs = sorted(
(b for b in client.list_blobs(BUCKET, prefix=INPUT_PREFIX) if b.name.endswith(".csv")),
key=lambda b: b.name,
)
return [b for i, b in enumerate(blobs) if i % TASK_COUNT == TASK_INDEX]
def summarize(blobs):
totals = defaultdict(lambda: {"qty": 0, "revenue": 0.0})
rows = 0
for blob in blobs:
text = blob.download_as_text()
for row in csv.DictReader(io.StringIO(text)):
qty = int(row["qty"])
price = float(row["unit_price"])
totals[row["sku"]]["qty"] += qty
totals[row["sku"]]["revenue"] += qty * price
rows += 1
for sku in totals:
totals[sku]["revenue"] = round(totals[sku]["revenue"], 2)
return rows, dict(sorted(totals.items()))
def notify(summary):
if not WEBHOOK_URL:
return
body = json.dumps({"text": f"Sales digest task {TASK_INDEX}/{TASK_COUNT}: "
f"{summary['rows']} rows, revenue {summary['revenue_total']}"}).encode()
req = urllib.request.Request(WEBHOOK_URL, data=body,
headers={"Content-Type": "application/json"})
urllib.request.urlopen(req, timeout=10).read()
def main():
started = time.time()
client = storage.Client()
shard = my_shard(client)
log("INFO", "shard selected", files=len(shard), task_count=TASK_COUNT)
rows, totals = summarize(shard)
summary = {
"report_date": REPORT_DATE,
"task": TASK_INDEX,
"task_count": TASK_COUNT,
"files": [b.name for b in shard],
"rows": rows,
"revenue_total": round(sum(v["revenue"] for v in totals.values()), 2),
"by_sku": totals,
}
out = client.bucket(BUCKET).blob(f"reports/{REPORT_DATE}/task-{TASK_INDEX}.json")
out.upload_from_string(json.dumps(summary, indent=2), content_type="application/json")
log("INFO", "report written", object=out.name, rows=rows,
revenue_total=summary["revenue_total"], seconds=round(time.time() - started, 2))
try:
notify(summary)
except Exception as exc: # a dead webhook must not fail the batch
log("WARNING", "webhook failed", error=str(exc))
if __name__ == "__main__":
try:
main()
except Exception as exc:
log("ERROR", "task failed", error=repr(exc))
sys.exit(1)Two details deserve attention. The script reads CLOUD_RUN_TASK_INDEX and CLOUD_RUN_TASK_COUNT with safe local defaults of 0 and 1, so the same file runs on a laptop without any Cloud Run variables. And it never calls sys.exit(0) explicitly. Falling off the end of main() exits with 0, which signals success to the platform.
Add the dependency file. Leave the package unpinned while you experiment, then pin the exact version that passed your tests before the first production run so a surprise upgrade cannot break a nightly job.
google-cloud-storageStep 7: Write the Dockerfile and .dockerignore
Cloud Run runs any container image, and a Dockerfile gives you control over the Python version. Google’s managed Python 3.15 runtime is only in Preview, so a Dockerfile on the official slim image is the dependable way to run the newest stable interpreter today. The image below installs dependencies before copying code, so rebuilds reuse the cached dependency layer when only main.py changes.
FROM python:3.15-slim
ENV PYTHONUNBUFFERED=1
PIP_NO_CACHE_DIR=1
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY main.py .
RUN useradd --create-home runner
USER runner
CMD ["python", "main.py"]__pycache__/
*.pyc
sample/
.git/
.env
*.jsonThree choices here pay off later. PYTHONUNBUFFERED=1 makes print output reach Cloud Logging immediately instead of vanishing if the container dies. The non-root runner user limits damage if a dependency is compromised. And the .dockerignore keeps your sample CSVs and any credential JSON out of the build context, which also speeds up uploads to Cloud Build.
Check that the base image tag exists before you rely on it. Tags for a brand-new Python line sometimes lag the release by days. If python:3.15-slim fails to pull, change the first line to python:3.14-slim and everything else works unchanged. Python 3.15 has already reached a final release and the 3.15.2 security release has followed, so a tag should exist, but a ten-second check beats a failed build.
Step 8: Test the Container Locally Before Paying for a Build
Local testing catches code mistakes before they cost build minutes. Start with the plain Python run against your real bucket, simulating task 0 of 3.
python3 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
export BUCKET="${PROJECT_ID}-sales-digest"
export CLOUD_RUN_TASK_INDEX=0
export CLOUD_RUN_TASK_COUNT=3
export REPORT_DATE=2026-11-05
python3 main.pyA healthy run prints two JSON lines and finishes in about a second. The first reports the shard size and the second confirms the object written.
{"severity": "INFO", "message": "shard selected", "task": 0, "attempt": 0, "files": 3, "task_count": 3}
{"severity": "INFO", "message": "report written", "task": 0, "attempt": 0, "object": "reports/2026-11-05/task-0.json", "rows": 1500, "revenue_total": 208211.83, "seconds": 0.41}The revenue figure comes from the fixed seed in the generator. Random sequences from a fixed seed can differ between Python releases, so if your total differs slightly, the row count per task is the check that matters. Now run the same logic inside the container, mounting your application-default credentials read-only.
docker build -t sales-digest:local .
docker run --rm
-e BUCKET -e CLOUD_RUN_TASK_INDEX=1 -e CLOUD_RUN_TASK_COUNT=3
-e REPORT_DATE=2026-11-05
-e GOOGLE_APPLICATION_CREDENTIALS=/tmp/adc.json
-v "$HOME/.config/gcloud/application_default_credentials.json:/tmp/adc.json:ro"
sales-digest:localTask 1 should report its own three files and their rows. If the container run works and the plain Python run works, any later failure sits in IAM, networking, or Cloud Run settings, not in your code. That narrowing is worth the two minutes.
Step 9: Deploy the Job With gcloud run jobs deploy
One command builds the image with Cloud Build, pushes it to Artifact Registry, and creates the job. The first time you run it, gcloud may ask whether to create an Artifact Registry repository for source deployments. Answer yes.
gcloud run jobs deploy sales-digest
--source .
--region "$REGION"
--service-account "$RUNNER_SA"
--tasks 3
--parallelism 3
--max-retries 2
--task-timeout 15m
--cpu 1
--memory 512Mi
--set-env-vars "BUCKET=${BUCKET}"Each flag maps to a real decision. --tasks 3 sets how many container copies one execution starts, and your code reads that number back as CLOUD_RUN_TASK_COUNT. --parallelism 3 lets all three run at once. Set it lower than the task count when a downstream database cannot take that much concurrency. --max-retries 2 allows two retries per failed task, so a flaky network call gets three total attempts. --task-timeout 15m kills a stuck task instead of letting it burn money indefinitely.
The build takes two to four minutes on a first run. Subsequent deploys finish faster because Cloud Build reuses cached layers. You will see build logs stream in your terminal, then a final line confirming the job was created.
Building using Dockerfile and deploying container to Cloud Run job [sales-digest]
in project [my-digest-demo-12345] region [us-central1]
OK Building and creating job... Done.
OK Uploading sources...
OK Building Container... Logs are available at [https://console.cloud.google.com/cloud-build/builds/...]
OK Creating Job...
Done.
Job [sales-digest] has successfully been deployed.
To execute this job, use:
gcloud run jobs execute sales-digestResource sizing deserves a note. 512 MiB and one vCPU fit this workload, because each task streams three small CSVs. If your own files run to gigabytes, raise memory first, since out-of-memory kills are the most common reason a batch task dies mid-run. Doubling memory is cheaper than debugging a partial report.
Step 10: Store the Webhook in Secret Manager and Attach It
The job already runs, but the webhook notification is silent because WEBHOOK_URL is empty. Webhook URLs act like passwords, since anyone holding the URL can post into your chat channel. Keep it out of the image and out of plain environment settings. Create a secret, let only the runner account read it, and mount it into the job.
printf '%s' 'https://hooks.example.com/services/REPLACE_ME' |
gcloud secrets create digest-webhook --data-file=-
gcloud secrets add-iam-policy-binding digest-webhook
--member="serviceAccount:${RUNNER_SA}"
--role="roles/secretmanager.secretAccessor"
gcloud run jobs update sales-digest
--region "$REGION"
--set-secrets "WEBHOOK_URL=digest-webhook:latest"The printf form matters. Piping with echo adds a trailing newline that corrupts some URLs, a quiet bug that surfaces as a webhook returning 404 while the URL looks right in the console. The :latest suffix means each new execution reads the newest secret version. If you want rotation without surprises, pin a numbered version instead and update it deliberately.
If you do not have a chat webhook handy, skip the secret and the job still works, because the code treats an empty URL as “notifications off”. Many teams point the same pattern at an email API key, a database password, or a third-party token. Only the secret name changes. A glance at how another vendor documents this pattern, such as Glean’s setup guide for a GCP Cloud Run Jobs service credential, shows the same idea of giving a job its own identity instead of a shared key.
Step 11: Execute the Job, Watch Parallel Tasks, and Read the Logs
Run it by hand first. The --wait flag holds your terminal until the execution finishes, which is what you want while testing.
gcloud run jobs execute sales-digest --region "$REGION" --waitOK Creating execution... Done.
OK Provisioning resources...
OK Starting execution...
Done.
Execution [sales-digest-x7k2p] has successfully completed 3/3 tasks.
To view the execution:
https://console.cloud.google.com/run/jobs/executions/details/us-central1/sales-digest-x7k2p/tasks?project=my-digest-demo-12345The execution name suffix will differ on your project. Now list recent executions and pull the structured logs. Because the worker prints JSON, you can filter on the fields you defined instead of grepping text.
gcloud run jobs executions list --job sales-digest --region "$REGION"
gcloud logging read
'resource.type="cloud_run_job" AND resource.labels.job_name="sales-digest"'
--limit 10 --freshness=1h
--format="value(timestamp,jsonPayload.task,jsonPayload.message)"You should see six report lines in total, two per task, with task numbers 0, 1, and 2 interleaved. Their order varies between runs because the tasks run in parallel. That interleaving is the visible proof that sharding works. To rerun one day with different data, override settings at execution time instead of redeploying. Check gcloud run jobs execute --help on your CLI version for the exact override flags, since they have evolved across releases.
Step 12: Schedule the Job With Cloud Scheduler
Cloud Scheduler calls the Cloud Run Admin API’s :run method on a cron schedule. Two pieces must line up. The scheduler account needs permission to run this specific job, and the HTTP target must use an OAuth token, not an OIDC token, because the Admin API is a Google API endpoint, not your own service.
gcloud run jobs add-iam-policy-binding sales-digest
--region "$REGION"
--member="serviceAccount:${SCHED_SA}"
--role="roles/run.invoker"
gcloud scheduler jobs create http sales-digest-daily
--location "$REGION"
--schedule "0 6 * * *"
--time-zone "Etc/UTC"
--uri "https://run.googleapis.com/v2/projects/${PROJECT_ID}/locations/${REGION}/jobs/sales-digest:run"
--http-method POST
--oauth-service-account-email "$SCHED_SA"
gcloud scheduler jobs run sales-digest-daily --location "$REGION"The last command forces an immediate trigger so you do not wait until 06:00 UTC to find out whether the wiring works. After about ten seconds, run the executions list again. A new execution should appear, started by the scheduler identity. If the scheduler job shows a 403, jump to the troubleshooting table below. The usual culprit is the missing run.invoker binding or a wrong URI region.
Step 13: Verify the Reports, Add an Alert, and Clean Up
Close the loop by checking the data, not just the exit code. List the report folder and add the three task totals together. With the seeded sample data and a three-task execution, the combined revenue across tasks should match the total your plain Python run reported.
gcloud storage ls "gs://${BUCKET}/reports/"
gcloud storage cat "gs://${BUCKET}/reports/2026-11-05/task-*.json"
| jq -s '{rows: (map(.rows) | add), revenue: (map(.revenue_total) | add)}'Expect the full row count of your dataset and a revenue value matching your earlier runs (the jq tool is optional, any JSON reader works). Then add a failure alert. In Cloud Monitoring, create a log-based alert on resource.type="cloud_run_job" with severity>=ERROR and route it to email or chat. A silent failed batch is worse than a loud one, and the structured ERROR line from Step 6 makes this alert trivial.
When you finish experimenting, delete the project to stop all charges and remove every resource in one step.
gcloud scheduler jobs delete sales-digest-daily --location "$REGION" --quiet
gcloud run jobs delete sales-digest --region "$REGION" --quiet
gcloud storage rm --recursive "gs://${BUCKET}"
gcloud projects delete "$PROJECT_ID" --quietCommon Pitfalls With Google Cloud Run Jobs
Most failed first deployments trace back to a short list of mistakes. Read these before your first scheduled run, because each one fails quietly and costs a night of missing reports.
- Running as the default Compute Engine service account. It often holds broad Editor rights on older projects. A bug or a poisoned dependency then inherits the keys to everything. Always pass
--service-accountwith a purpose-built identity, as Step 9 does. - Writing non-idempotent output. Cloud Run retries failed tasks, so a task that appends rows to a table will duplicate data after a retry. Write whole files to deterministic paths, or use upserts keyed on a stable ID.
- Confusing tasks with parallelism. Tasks are how many copies one execution runs in total. Parallelism is how many run at the same moment. Ten tasks at parallelism two finish in five waves.
- Ignoring exit codes. Catching every exception and exiting 0 makes the platform report a green run over a broken one. Log the error, then exit non-zero.
- Using a service where a job belongs. A service must answer HTTP on a port and times out on long requests. If your work has an end, deploy a job. If you want progressive rollouts for an HTTP workload instead, our Cloud Run canary deployment walkthrough covers traffic splitting for services.
- Forgetting time zones. A schedule without
--time-zonemay not run when you expect. Set UTC explicitly and convert in your head once, not every morning. - Trusting
:latestsecrets blindly. A bad secret version reaches the next execution instantly. Pin versions for anything that can break a pipeline.
A related trap is cost surprise from retries and long timeouts. Every retry bills its own runtime, so a task that fails after 14 minutes of a 15-minute timeout can triple its cost with two retries. Set a sane timeout, fail fast on bad input, and alert on repeated failures rather than letting retries mask a real defect.
Troubleshooting: 10 Fixes for Failed Cloud Run Jobs
When something breaks, read the failing task’s logs first, then match the symptom below. The table lists the ten problems readers hit most often when building this project, with the cause and the fix.
| # | Symptom | Likely cause | Fix |
|---|---|---|---|
| 1 | Deploy fails with PERMISSION_DENIED during the build |
The Cloud Build identity lacks rights to push or read source | Grant the build service account Artifact Registry Writer and Storage access, then redeploy |
| 2 | Build error: base image python:3.15-slim not found |
Tag not published yet for your architecture or mirror | Switch the first Dockerfile line to python:3.14-slim and rebuild |
| 3 | Task exits with code 1 and a KeyError: 'BUCKET' |
The BUCKET environment variable never reached the container |
Add --set-env-vars "BUCKET=..." or run gcloud run jobs update |
| 4 | Local run raises DefaultCredentialsError |
No application-default credentials on your machine | Run gcloud auth application-default login |
| 5 | Logs show 403 on storage.objects.list |
The runner account has no role on the bucket | Re-run the Step 4 binding for roles/storage.objectUser |
| 6 | Execution fails to start with a secret permission error | The runner account cannot read the secret | Bind roles/secretmanager.secretAccessor on that secret |
| 7 | Scheduler job shows PERMISSION_DENIED |
The scheduler account lacks run.invoker on the job |
Add the Step 12 binding and rerun the scheduler job |
| 8 | Scheduler returns NOT_FOUND |
The URI has a wrong project, region, or job name | Compare the URI against gcloud run jobs describe sales-digest |
| 9 | Task dies with a memory limit message | A file or a dictionary outgrew 512 MiB | Raise --memory, or stream the file line by line |
| 10 | Task ends at the timeout with no report | Too many files per task for the timeout | Raise --task-timeout or increase --tasks so each shard shrinks |
Two more problems appear often enough to mention. If logs seem to arrive late or never, confirm PYTHONUNBUFFERED=1 sits in the Dockerfile and that you print with flush=True. If a report holds duplicate rows after a retry, your task writes append-style output somewhere else, so move to deterministic overwrite paths. For anything strange, gcloud run jobs executions describe EXECUTION_NAME --region "$REGION" prints the per-task status and the final condition message, which usually names the failing step in plain words.
A debugging routine that saves time
Work from the outside in. First confirm the execution started, using the executions list. Then check task counts: how many succeeded, failed, or never ran. Next read the logs of one failed task, filtered by its task number. Last, reproduce the failure locally with the same environment variables and the same container image. Following that order prevents the classic mistake of editing code to fix what is actually an IAM problem.
What a Run Costs and How to Control It
Cloud Run keeps its pay-per-use model for jobs. You pay for the CPU and memory a task holds while it runs, plus small charges from neighboring products such as Cloud Build, Artifact Registry storage, Cloud Storage, and Cloud Scheduler. Google’s pricing page holds the current rates by region, and those rates change, so check it before you size a budget. This article quotes no per-second prices for that reason.
The sample project runs three tasks for well under a minute each, so one execution consumes a tiny fraction of any monthly allowance. What matters is how the levers scale once your data grows.
| Lever | Effect on cost | Effect on speed | When to change it |
|---|---|---|---|
--tasks |
More tasks, more total vCPU-seconds if work is fixed | Faster, until overhead dominates | Data splits cleanly into shards |
--parallelism |
Neutral for total work | Controls waves | Downstream systems throttle you |
--cpu |
Higher per second | Helps CPU-bound code only | Profiling shows pegged CPU |
--memory |
Higher per second | Prevents out-of-memory kills | Tasks die with memory errors |
--max-retries |
Each retry bills a full attempt | Slower on repeated failure | Lower it for expensive tasks |
--task-timeout |
Caps the worst case per task | Fails stuck tasks early | Always set it |
Watch spend with the billing tools you already have. Our comparison of AWS Cost Explorer, Azure Cost Management, and Google Cloud Billing shows how each provider surfaces daily cost, and it helps you set a budget alert on the project from day one. Teams comparing container platforms for batch work should also read the Fargate, Cloud Run, and Container Apps price comparison before they choose where jobs live.
One more cost idea is worth a sentence. Google has reportedly announced billing caps for Cloud Run that would deactivate resources after a monthly spend limit. Treat that as planned, not available, until you see it in the documentation for your project.
Advanced Tips: GPUs, Checkpoints, and Choosing the Right Runtime
Try the GPU Preview on a toy workload first
The October 30 Preview lets a job run with GPUs for up to 12 hours, which opens batch inference, embedding generation, and fine-tuning runs that previously needed a cluster. Because it is Preview, expect changing flags, limited regions, and no production service-level promise. Start with a container that prints the GPU it sees, confirm quota, and only then move real work. Earlier Cloud Run GPU launches added hardware in stages, according to Google’s updates. Check the current job documentation for the supported types in your region.
Checkpoint long tasks so retries resume instead of restart
A task that runs for hours should not start from zero after a failure. Write progress markers to Cloud Storage and skip finished work on the next attempt. The pattern needs only a few lines in the worker.
def already_done(client, bucket_name, report_date, blob_name):
marker = client.bucket(bucket_name).blob(
f"state/{report_date}/{blob_name.replace('/', '_')}.done")
return marker.exists()
def mark_done(client, bucket_name, report_date, blob_name):
client.bucket(bucket_name).blob(
f"state/{report_date}/{blob_name.replace('/', '_')}.done"
).upload_from_string("ok")Call already_done before processing each file and mark_done after the result is stored. A retried task, which sees CLOUD_RUN_TASK_ATTEMPT above zero, then skips completed files. Keep the marker write after the output write so a crash between them redoes work instead of losing it.
Match the runtime to the workload
Jobs are not the answer to everything. Pull-based, continuously running consumers fit worker pools, an existing Cloud Run capability for queue- or stream-driven workloads rather than HTTP services. Teams that need custom schedulers, GPUs at scale today, or tight control over networking may still prefer a cluster, and our Google Kubernetes Engine setup guide covers that path. Teams on Microsoft’s cloud can build the same batch pattern on the platform in our Azure Container Apps setup guide. The skills transfer, since the model of a container, a trigger, and an identity is the same everywhere.
Move to the managed Python 3.15 runtime when it graduates
Google’s Python 3.15 runtime is Preview as of the November 2 release notes. Once it reaches general availability, you can drop the Dockerfile and deploy from source with the managed builder, as long as you do not need system packages. Until then, the Dockerfile route keeps you on the newest stable interpreter without waiting for the platform.
Frequently Asked Questions
What is the difference between a Cloud Run job and a Cloud Run service?
A service listens for HTTP requests and scales with traffic. A job runs a container to completion and exits, which suits batch work, reports, migrations, and scheduled tasks. Jobs do not need to listen on a port.
How do I run a Cloud Run job on a schedule?
Create a Cloud Scheduler HTTP job that calls the Cloud Run Admin API’s :run endpoint for your job. Use a dedicated service account with the run.invoker role on that job and authenticate with an OAuth token. Step 12 shows the exact commands.
Can Cloud Run jobs use GPUs?
Yes, in Preview. Google introduced GPU-enabled jobs on October 30, 2026, with executions allowed to run for up to 12 hours. Preview features can change, so test before you depend on them.
Does Cloud Run support Python 3.15?
The managed Python 3.15 runtime is in Preview. You can run Python 3.15 today by building your own container from the official python:3.15-slim image, as this tutorial does. Python 3.15.2 was the newest stable release as of October 26, 2026.
How do parallel tasks know which data to process?
Cloud Run sets CLOUD_RUN_TASK_INDEX and CLOUD_RUN_TASK_COUNT in every container. Your code uses them to pick its slice of the work, for example by taking files where the file position modulo the task count equals the task index.
What happens when a task fails?
Cloud Run retries the failed task up to the --max-retries value you set. If the task still fails, the execution reports a failure. Because retries repeat work, design tasks to overwrite their output instead of appending to it.
How much does a Cloud Run job cost?
You pay for the CPU and memory a task uses while it runs, with no charge between executions, plus small charges from related services. Rates vary by region and change over time, so use Google’s pricing page and calculator for current figures.
How do I delete everything created in this tutorial?
Delete the Cloud Scheduler job, the Cloud Run job, and the bucket, then delete the project. Deleting the project removes every resource in it and stops billing, as shown at the end of Step 13.