Background for students new to Git, GitHub, and Docker
The CI/CD pipeline sits on top of several tools that professional developers use every day. If these are new to you, start here. Understanding the purpose of each tool makes the pipeline feel logical rather than overwhelming.
Version control — Git
git commit -m "Add Stripe checkout endpoint". This snapshot is stored permanently in your project's history.
main. Developers work on short-lived branches like feature/stripe-checkout.
GitHub — the shared home for code
GLSChineseHub repository contains your FastAPI backend, Next.js frontend, and all the configuration files that make the CI/CD pipeline work.
main). Before the merge happens, teammates can review the changes, leave comments, and approve or reject. The name "Pull Request" means you are asking the team to "pull" your changes in.
.yml files) that tell GitHub to automatically run tasks whenever something happens — like "whenever someone pushes to the develop branch, run all the tests." This is the engine of the CI/CD pipeline.
Docker — packaging your application
docker-compose.yml defines them and starts them all with one command: docker-compose up.
Why do we use .yml sometimes and .md other times?
This is one of the most common points of confusion for new developers. Both are plain text files — but they serve completely different purposes and are read by completely different tools.
In CI/CD, GitHub Actions reads your
.yml files to know what steps to run, in what order, and under what conditions. The computer executes these instructions literally.# for headings and ** for bold. GitHub automatically renders .md files as formatted pages.In a repo,
.md files are documentation — they explain the project to humans, not to computers..yml. If a human reads it to understand something → .md. A README.md tells a new developer what the project is. A ci.yml tells GitHub Actions what to do when code is pushed.
Other file types you will encounter in a CI/CD repo:
| Extension | Name | Purpose | Read by |
|---|---|---|---|
| .yml / .yaml | YAML | Automation instructions, configuration | GitHub Actions, Docker Compose, pre-commit |
| .md | Markdown | Human-readable documentation | Developers, GitHub (renders as web page) |
| Dockerfile | Dockerfile | Instructions for building a Docker image | Docker engine |
| .env | Env file | Secret values for local development | Your app at runtime — never committed to Git |
| .toml | TOML | Python tool configuration (ruff, black, mypy) | Python tooling |
| .json | JSON | Structured data, Node.js config | npm, APIs, many tools |
The three principles that hold everything together
.env, CI uses GitHub Secrets, Cloud uses Secret Manager or Vault. No secret ever travels as a value baked into an image layer.Local Development
The developer's machine is where the feedback loop must be fastest. Everything that can break in CI should surface here first.
Run the same Docker image locally via docker-compose up. Local source code is mounted as a volume for hot reload. Env vars come from a .env file — gitignored, never committed.
Scripts that run automatically before git commit finalizes. If any hook fails, the commit is blocked — code never reaches GitHub.
Run pytest and npm test before pushing. The same tests that run in CI should run locally — no surprises.
Pre-commit hooks — the tool breakdown
| Tool | Language | Type | What it does |
|---|---|---|---|
| black | Python | Formatter | Auto-rewrites code to consistent style. Opinionated — no config needed. |
| ruff | Python | Linter | Replaces flake8 + isort. 100× faster (Rust). Flags unused imports, bad patterns. |
| mypy | Python | Type checker | Catches type errors that linters miss. Slower — often run in CI not commit. |
| eslint | JS/TS | Linter | Rules + auto-fix for JavaScript and TypeScript. Catches bugs and bad patterns. |
| prettier | JS/TS/JSON | Formatter | Opinionated style rewriter for the JS ecosystem. |
| tsc --noEmit | TypeScript | Type checker | Checks type annotations without emitting compiled output. |
| detect-secrets | Any | Security | Scans every staged file for API keys, tokens, and private key patterns. |
Pre-commit config (.pre-commit-config.yaml)
repos: - repo: https://github.com/psf/black rev: 24.3.0 hooks: - id: black - repo: https://github.com/astral-sh/ruff-pre-commit rev: v0.4.1 hooks: - id: ruff args: [--fix] - repo: https://github.com/pre-commit/pre-commit-hooks rev: v4.6.0 hooks: - id: detect-private-key - id: trailing-whitespace - id: end-of-file-fixer - repo: https://github.com/Yelp/detect-secrets rev: v1.4.0 hooks: - id: detect-secrets
GitHub — Policy Enforcement
GitHub is your policy enforcement layer, not just storage. Branch protection rules, PR gates, and Actions triggers form the quality gate between a developer's machine and the build system.
Branch structure
Branch protection rules
Require PR before merge · 1+ approvals · status checks must pass · branches must be up to date · no bypass allowed.
Status checks must pass (CI green) · allow direct push for solo developer · no unreviewed merges from external contributors.
Settings → Secrets and Variables → Actions. Scope secrets to environments — create staging and production environments, restrict which branches deploy to each.
GitHub Actions trigger (.github/workflows/ci.yml)
on: push: branches: [develop] # every push to develop triggers CI pull_request: branches: [main, develop] # every PR targeting either branch
CI Build — GitHub Actions
CI has one job: produce a verified, immutable artifact. Two parallel jobs run simultaneously — backend (Python/FastAPI) and frontend (Next.js). Total CI time equals whichever takes longer.
services: block spins up a real Postgres container alongside the test runner on the same Docker network. Your test code connects to localhost:5432 as if Postgres were local. Without a health check, tests may start before Postgres is ready and fail with a connection error rather than a real test failure.Complete ci.yml
name: CI on: push: branches: [develop] pull_request: branches: [main, develop] jobs: backend: name: Python / FastAPI runs-on: ubuntu-latest services: postgres: image: postgres:15 env: POSTGRES_USER: testuser POSTGRES_PASSWORD: testpass POSTGRES_DB: gls_test ports: - 5432:5432 options: >- --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 env: DATABASE_URL: postgresql://testuser:testpass@localhost:5432/gls_test ENVIRONMENT: test steps: - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: "3.11" cache: pip - name: Install dependencies run: pip install -r requirements.txt ruff black pytest pytest-asyncio httpx - name: Run migrations run: alembic upgrade head - name: Lint (ruff) run: ruff check . - name: Format check (black) run: black --check . # --check fails if files need reformatting - name: Run tests run: pytest tests/ -v frontend: name: Next.js / TypeScript runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up Node uses: actions/setup-node@v4 with: node-version: "20" cache: npm - name: Install dependencies working-directory: ./frontend run: npm ci # clean lockfile install — faster, deterministic - name: Lint (ESLint) working-directory: ./frontend run: npm run lint - name: Type check working-directory: ./frontend run: npx tsc --noEmit - name: Build check working-directory: ./frontend run: npm run build env: NEXT_PUBLIC_API_URL: http://localhost:8000
Step-by-step breakdown — backend
requirements.txt hash. pip only re-downloads packages when that file changes. Saves 30–60 seconds per run.--check flag is critical. Without it, black would reformat files in the runner and then pass — useless. With it, CI fails if any file would be reformatted, forcing the developer to run black locally first.-v gives verbose output so failed test names are visible in the CI log.Step-by-step breakdown — frontend
npm ci deletes node_modules and installs exactly what's in package-lock.json. No resolution, no version negotiation. Fails if lockfile is out of sync with package.json.Docker Registry
The registry is your artifact store and the single source of truth for what gets deployed. Images are immutable once pushed — a SHA-tagged image is never rebuilt.
Tagging strategy — SHA tag vs Semver tag
A Docker image tag is a label attached to a specific image, so you can refer to it by name later. The same image can carry multiple tags simultaneously — like a product that has both a serial number and a model name printed on the box. Understanding the difference between SHA and semver tags is essential for safe deployments and reliable rollback.
One image can carry multiple tags at the same time. When you cut a release, CI applies both — the SHA for machine traceability, and the semver for human readability. They both point to the exact same image bytes.
docker build \
-t .../backend:abc12ef \ # SHA tag — always
-t .../backend:v1.4.2 \ # Semver tag — on release only
./backend
# Both tags point to identical image bytes in the registry:
.../backend:abc12ef → sha256:f3a9b2c1...
.../backend:v1.4.2 → sha256:f3a9b2c1... # same digest
latest is a mutable tag — it silently moves to a new image every time you push. If your Cloud Run service is configured to use backend:latest, then a bad push at 2am automatically becomes production. Rollback becomes impossible because the tag no longer points to the old image.
--image .../backend:latest
--image .../backend:abc12ef
Quick reference — all three tag types
| Tag type | Example | Mutable? | Applied when | Used by |
|---|---|---|---|---|
| SHA tag | backend:abc12ef |
No — immutable | Every CI push | Cloud Run deploy, rollback, debugging |
| Semver tag | backend:v1.4.2 |
No — immutable | On official git release tag only | Stakeholders, changelogs, milestone tracking |
| latest tag | backend:latest |
Yes — dangerous | Every push (auto) | Local dev only — never production |
Build and push job (deploy.yml)
jobs: build-and-push: name: Build + push images runs-on: ubuntu-latest outputs: sha: ${{ steps.tag.outputs.SHA }} steps: - uses: actions/checkout@v4 - id: auth uses: google-github-actions/auth@v2 with: workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }} service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }} - name: Configure Docker for Artifact Registry run: gcloud auth configure-docker asia-east1-docker.pkg.dev --quiet - name: Set image tag id: tag run: echo "SHA=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT - name: Build backend image run: | docker build \ -t asia-east1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/images/backend:${{ steps.tag.outputs.SHA }} \ ./backend - name: Push backend image run: | docker push \ asia-east1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/images/backend:${{ steps.tag.outputs.SHA }} - name: Build frontend image run: | docker build \ -t asia-east1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/images/frontend:${{ steps.tag.outputs.SHA }} \ ./frontend \ --build-arg NEXT_PUBLIC_API_URL=${{ secrets.STAGING_API_URL }} - name: Push frontend image run: | docker push \ asia-east1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/images/frontend:${{ steps.tag.outputs.SHA }}
GCP Artifact Registry setup (one-time)
# Create the repository gcloud artifacts repositories create images \ --repository-format=docker \ --location=asia-east1 \ --description="GLS_CH_Hub container images" # Enable vulnerability scanning gcloud services enable containerscanning.googleapis.com # Set retention policy — keep last 10 images per service gcloud artifacts repositories set-cleanup-policies images \ --location=asia-east1 \ --policy='[{"name":"keep-last-10","action":{"type":"Keep"},"condition":{"tagState":"tagged","mostRecentVersions":{"keepCount":10}}}]'
Cloud Deployment — Cloud Run
Deployment happens in two hops: staging first, then production. The same image that passed staging goes to production — not a different build. Traffic splitting enables safe canary deployments with instant rollback.
How Cloud Run revisions work
A revision is defined by image SHA + env vars + secret versions. Change any of the three and you get a new revision. Old revisions stay available at 0% traffic.
Route traffic percentages to specific revisions. Deploy new revision at 0%, canary to 10%, watch metrics, promote to 100%. Blast radius contained at each step.
Old revision never deleted — just at 0% traffic. Rollback = one command pointing 100% traffic back to previous revision. No rebuild, no retest, under 30 seconds.
Staging deploy + smoke test
deploy-staging: name: Deploy to staging needs: build-and-push runs-on: ubuntu-latest steps: - id: auth uses: google-github-actions/auth@v2 with: workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }} service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }} - name: Deploy backend to staging run: | gcloud run deploy gls-backend-staging \ --image asia-east1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/images/backend:${{ needs.build-and-push.outputs.sha }} \ --region asia-east1 \ --platform managed \ --set-secrets DATABASE_URL=staging-db-url:latest,STRIPE_SECRET_KEY=stripe-key-staging:latest \ --set-env-vars ENVIRONMENT=staging \ --allow-unauthenticated - name: Smoke test staging run: | sleep 10 curl --fail https://gls-backend-staging-xxxx.run.app/health curl --fail https://gls-frontend-staging-xxxx.run.app/
Production canary deploy
deploy-production: name: Deploy to production needs: deploy-staging runs-on: ubuntu-latest environment: production # requires manual approval in GitHub steps: - id: auth uses: google-github-actions/auth@v2 with: workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }} service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }} # Deploy new revision with NO traffic yet - name: Deploy to prod (no traffic) run: | gcloud run deploy gls-backend-prod \ --image asia-east1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/images/backend:${{ needs.build-and-push.outputs.sha }} \ --region asia-east1 \ --no-traffic \ --tag canary \ --set-secrets DATABASE_URL=prod-db-url:latest,STRIPE_SECRET_KEY=stripe-key-prod:latest,PINECONE_API_KEY=pinecone-key:latest \ --set-env-vars ENVIRONMENT=production # Send 10% to new revision — canary phase - name: Canary — 10% traffic run: | gcloud run services update-traffic gls-backend-prod \ --region asia-east1 \ --to-tags canary=10 - name: Monitor canary (60s) run: sleep 60 # Promote fully - name: Promote to 100% run: | gcloud run services update-traffic gls-backend-prod \ --region asia-east1 \ --to-tags canary=100
Rollback command
# Point 100% of traffic back to any previous revision immediately gcloud run services update-traffic gls-backend-prod \ --region asia-east1 \ --to-revisions=PREVIOUS-REVISION-NAME=100
Required health endpoint (FastAPI)
@app.get("/health") def health(): # Simple, fast, no auth, no DB call # curl --fail will exit non-zero if this returns anything other than 200 return {"status": "ok", "env": settings.ENVIRONMENT}
--set-secrets flag syntax stripe-key-prod:latest means Cloud Run pulls the latest version from Secret Manager at container start. Your app reads secrets with os.environ.get("STRIPE_SECRET_KEY") — no SDK needed, no secret baked into the image.Secrets across all five stages
| Context | Secrets source | How it works |
|---|---|---|
| Local dev | .env file | gitignored, on your machine only, loaded by docker-compose |
| GitHub Actions CI | GitHub Secrets | ${{ secrets.NAME }} — scoped to repo or environment |
| Docker image | Build args (NEXT_PUBLIC_* only) | Baked at build time — only for Next.js public vars. Never for API keys. |
| Cloud Run staging | GCP Secret Manager | --set-secrets KEY=secret-name:latest — injected at container start |
| Cloud Run production | GCP Secret Manager | Same pattern — separate secret names/values from staging |
Recommended repo layout
GLSChineseHub/ ├── .github/ │ ├── workflows/ │ │ ├── ci.yml ← Stage 3: runs on push/PR │ │ └── deploy.yml ← Stages 4+5: runs on merge to main │ └── pull_request_template.md ├── backend/ ← FastAPI app │ ├── Dockerfile │ ├── requirements.txt │ └── main.py ├── frontend/ ← Next.js app │ ├── Dockerfile │ └── package.json ├── .pre-commit-config.yaml ← Stage 1 hooks ├── docker-compose.yml ← Stage 1 local dev └── .env.example ← template — never commit .env itself