Build Triggers & Configuration Management

Tại sao quan trọng

Mỗi build bắt đầu từ trigger — một event mà tự động kick off build. Trigger có thể từ Git push, manual CLI submission, scheduled task, hoặc webhook. Sau trigger, Cloud Build cần biết cần làm gì — đó là công việc của cloudbuild.yaml.

Configuration strategy ảnh hưởng tới:

• Build performance: Caching strategy ngắn/ dài build time bao nhiêu
• Reproducibility: Substitution variables có được interpolate đúng không
• Maintenance burden: Quản lý multiple branches/environments có phức tạp không

Chương này dạy bạn cách tối ưu trigger setup, master substitution variables, và implement effective caching.

---

Build Triggers: Khi nào Builds Kick Off

Trigger Types

1. Cloud Source Repositories

gcloud builds triggers create cloud-source-repositories \
  --repo=my-repo \
  --branch-pattern="^main$" \
  --name=main-build

Trigger khi: Commit pushed to main branch ở Cloud Source Repos

2. GitHub (Connected)

gcloud builds triggers create github \
  --repo-owner=myorg \
  --repo-name=my-app \
  --branch-pattern="^main$" \
  --build-config=cloudbuild.yaml \
  --name=github-main-build

Trigger khi: PR merged / commit pushed to GitHub

3. GitLab

Similar to GitHub — Cloud Build connects via OAuth to GitLab

4. Bitbucket

gcloud builds triggers create bitbucket \
  --repo-owner=myorg \
  --repo-name=my-app \
  --branch-pattern=".*" \
  --name=bitbucket-build

5. Scheduled Triggers (cron-like)

gcloud builds triggers create scheduled \
  --schedule="0 2 * * *" \
  --timezone="America/Los_Angeles" \
  --build-config=cloudbuild.yaml \
  --name=nightly-build

Trigger khi: Clock hits specified time (CRON syntax)

6. Manual Trigger (CLI)

gcloud builds submit --config=cloudbuild.yaml

Trigger khi: Developer manually invokes via CLI

7. Webhook

# Create trigger
gcloud builds triggers create webhook \
  --name=webhook-build

# Output: webhook URL
# https://cloudbuild.googleapis.com/v1/projects/{PROJECT_ID}/triggers/{TRIGGER_ID}/webhook

Use URL ở external system (Jenkins, GitLab, custom app) để trigger builds

Trigger Configuration

Filename patterns:

gcloud builds triggers create cloud-source-repositories \
  --repo=my-repo \
  --included-files-filter="backend/**" \
  # Only trigger if changes ở backend/ folder
  --ignored-files-filter="*.md,docs/**"
  # Don't trigger if only markdown changed

Substitutions:

gcloud builds triggers create github \
  --repo-owner=myorg \
  --repo-name=my-app \
  --substitutions="_ENVIRONMENT=dev,_REGISTRY=us-central1-docker.pkg.dev"
  # Available as ${_ENVIRONMENT}, ${_REGISTRY} ở cloudbuild.yaml

---

cloudbuild.yaml: Build Configuration Schema

Minimal Example

steps:
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/app:$SHORT_SHA', '.']

images:
  - 'gcr.io/$PROJECT_ID/app:$SHORT_SHA'

Điều gì xảy ra:

1. Execute step: docker build -t gcr.io/{PROJECT_ID}/app:{SHORT_SHA} .
2. Push image gcr.io/{PROJECT_ID}/app:{SHORT_SHA} to registry
3. Return result

Step Definition

steps:
  - name: 'gcr.io/cloud-builders/docker'          # Container image
    args: ['build', '.']                            # Command arguments
    env:                                            # Environment variables
      - 'DOCKER_BUILDKIT=1'
    dir: './backend'                                # Working directory
    serviceAccount: 'deployer@...'                  # Which service account
    entrypoint: '/bin/bash'                         # Override entrypoint
    id: 'docker-build'                              # Step ID (for ordering)
    waitFor: ['-']                                  # Wait for (default: previous step)
    timeout: '1200s'                                # Per-step timeout
    allowFailure: false                             # Continue even if fails

Substitution Variables

Built-in (auto-populated):

steps:
  - name: 'gcr.io/cloud-builders/docker'
    args:
      - 'build'
      - '-t'
      - 'gcr.io/${PROJECT_ID}/app:${SHORT_SHA}'
      - '.'

Custom (user-defined):

substitutions:
  _IMAGE_REPO: 'us-central1-docker.pkg.dev/my-project/images'
  _ENVIRONMENT: 'production'

steps:
  - name: 'gcr.io/cloud-builders/docker'
    args:
      - 'build'
      - '-t'
      - '${_IMAGE_REPO}/app:${SHORT_SHA}'
      - '.'
    env:
      - 'BUILD_ENV=${_ENVIRONMENT}'

Accessing from CLI:

gcloud builds submit \
  --substitutions=_ENVIRONMENT=staging,_REPLICAS=3

Variable interpolation rules:

• ${VAR} — interpolate VAR
• ${VAR:-default} — use default nếu VAR không set
• Nested variables không supported: ${${NESTED}} tidak work

Build Options

options:
  # Which worker pool to use
  workerPool: 'projects/PROJECT_ID/locations/REGION/workerPools/POOL_NAME'
  
  # Log streaming
  logging: CLOUD_LOGGING_ONLY  # or GCS_ONLY, CLOUD_LOGGING_AND_GCS
  
  # Default service account (for all steps)
  defaultServiceAccount: 'projects/PROJECT_ID/serviceAccounts/builder@...'
  
  # Machine type for default pool
  machineType: 'N1_HIGHCPU_8'
  
  # Build environment variables
  env:
    - 'GOLANG_VERSION=1.20'
  
  # Enable SLSA provenance
  requestedVerifyOption: VERIFIED

---

Build Caching Strategies

Layer Caching (Built-in Docker Feature)

FROM ubuntu:22.04

# Layer 1: System dependencies (slow, rarely changes)
RUN apt-get update && apt-get install -y \
    curl wget git

# Layer 2: Application dependencies (medium speed)
COPY requirements.txt .
RUN pip install -r requirements.txt

# Layer 3: Application code (fast, changes frequently)
COPY . .
RUN python setup.py build

How it works:

Docker caches layers. If layer N hasn't changed, Docker skips rebuild.

In Cloud Build:

steps:
  - name: 'gcr.io/cloud-builders/docker'
    args:
      - 'build'
      - '--cache-from'
      - 'gcr.io/$PROJECT_ID/app:latest'  # Use previous image as cache
      - '-t'
      - 'gcr.io/$PROJECT_ID/app:$SHORT_SHA'
      - '.'

First build: Takes 5-10 minutes (no cache) Second build: Takes 30-60 seconds (layers cached from first build)

Custom Artifact Caching (GCS-based)

Dùng GCS để cache non-Docker artifacts (node_modules, .m2 repos, etc.):

steps:
  - name: 'node:18'
    id: 'install-deps'
    entrypoint: 'bash'
    args:
      - '-c'
      - |
        # Restore from cache
        gsutil -m cp -r gs://my-build-cache/node_modules . 2>/dev/null || true
        
        # Install dependencies
        npm ci
        
        # Upload to cache
        gsutil -m cp -r node_modules gs://my-build-cache/
  
  - name: 'node:18'
    id: 'run-tests'
    args: ['npm', 'test']

Trade-off:

• Upload/download time: 30-60 seconds per cache hit/miss
• Cost: GCS storage + egress bandwidth
• Benefit: Dependencies not re-downloaded every build

Build Artifact Caching

# Cache intermediate outputs
artifacts:
  objects:
    location: 'gs://my-build-artifacts/$BUILD_ID'
    paths: ['dist/**/*', 'build/**/*']

After build, artifacts automatically uploaded to GCS. Next build can restore:

steps:
  - name: 'gcr.io/cloud-builders/gcs-fetcher'
    args:
      - 'cp'
      - 'gs://my-build-artifacts/$PREVIOUS_BUILD_ID/dist'
      - './dist'

---

Advanced: Conditional Execution & Ordered Steps

Step Ordering by ID

steps:
  - name: 'node:18'
    id: 'unit-tests'
    args: ['npm', 'test:unit']
  
  - name: 'node:18'
    id: 'integration-tests'
    args: ['npm', 'test:integration']
    waitFor: ['unit-tests']  # Run after unit-tests completes

  - name: 'gcr.io/cloud-builders/docker'
    id: 'build-image'
    args: ['build', '-t', 'app:latest', '.']
    waitFor: ['integration-tests']  # Run only if both tests pass

waitFor semantics:

• No waitFor → waits for previous step
• waitFor: ['-'] → run in parallel with all other steps
• waitFor: ['step-id'] → wait for specific step

Conditional Execution (Workaround)

Cloud Build không native support if statements. Workaround:

steps:
  - name: 'gcr.io/cloud-builders/git'
    id: 'check-branch'
    entrypoint: 'bash'
    args:
      - '-c'
      - |
        if [ "$BRANCH_NAME" = "main" ]; then
          echo "Building for production"
          exit 0
        else
          echo "Skipping prod build for non-main branch"
          exit 0  # Still exit 0 to continue
        fi
  
  - name: 'gcr.io/cloud-builders/docker'
    entrypoint: 'bash'
    args:
      - '-c'
      - |
        if [ "$BRANCH_NAME" = "main" ]; then
          docker push gcr.io/$PROJECT_ID/app:latest
        else
          echo "Skipping push for branch: $BRANCH_NAME"
        fi

---

Real-World: Multi-Environment Build Pipeline

substitutions:
  _REGISTRY: 'us-central1-docker.pkg.dev/my-project/images'
  _DEV_ENV: 'dev'
  _PROD_ENV: 'prod'

steps:
  # Source verification
  - name: 'gcr.io/cloud-builders/git'
    id: 'fetch-source'
    args: ['show', 'HEAD:--format=%H']
  
  # Build image
  - name: 'gcr.io/cloud-builders/docker'
    id: 'build-image'
    args:
      - 'build'
      - '--cache-from=${_REGISTRY}/app:latest'
      - '-t'
      - '${_REGISTRY}/app:${SHORT_SHA}'
      - '.'
  
  # Push to dev registry
  - name: 'gcr.io/cloud-builders/docker'
    id: 'push-dev'
    args: ['push', '${_REGISTRY}/app:${SHORT_SHA}']
  
  # Run tests against dev image
  - name: 'gcr.io/cloud-builders/docker'
    id: 'run-tests'
    args:
      - 'run'
      - '${_REGISTRY}/app:${SHORT_SHA}'
      - 'npm test'
  
  # Deploy to dev (all branches)
  - name: 'gcr.io/cloud-builders/kubectl'
    id: 'deploy-dev'
    args:
      - 'set'
      - 'image'
      - 'deployment/app'
      - 'app=${_REGISTRY}/app:${SHORT_SHA}'
      - '--namespace=dev'
    env:
      - 'CLOUDSDK_CONTAINER_CLUSTER=dev-cluster'
  
  # Tag latest if main branch
  - name: 'gcr.io/cloud-builders/docker'
    id: 'tag-latest'
    entrypoint: 'bash'
    args:
      - '-c'
      - |
        if [ "$BRANCH_NAME" = "main" ]; then
          docker tag ${_REGISTRY}/app:${SHORT_SHA} ${_REGISTRY}/app:latest
          docker push ${_REGISTRY}/app:latest
        else
          echo "Skipping latest tag for non-main branch"
        fi
  
  # Deploy to prod (only main)
  - name: 'gcr.io/cloud-builders/kubectl'
    id: 'deploy-prod'
    entrypoint: 'bash'
    args:
      - '-c'
      - |
        if [ "$BRANCH_NAME" = "main" ]; then
          gcloud container clusters get-credentials prod-cluster
          kubectl set image deployment/app app=${_REGISTRY}/app:latest --namespace=prod
        else
          echo "Skipping prod deployment for non-main branch"
        fi

images:
  - '${_REGISTRY}/app:${SHORT_SHA}'
  - '${_REGISTRY}/app:latest'  # Only if main

timeout: '1800s'

---

Troubleshooting Configuration Issues

"Substitution Variable Not Found"

ERROR: (gcloud.builds.submit) User provided substitution '_ENVIRONMENT' is not in [...]

Solution: Define ở substitutions: section

substitutions:
  _ENVIRONMENT: 'prod'

"Build step timed out"

Step takes longer than timeout value.

Solution: Increase timeout hoặc optimize step

timeout: '3600s'  # 1 hour instead of 10 minutes

"Cached layer not used"

You expected --cache-from to speed up build, nhưng nó cũng lâu.

Possible cause: Base image changed, or cache image doesn't exist

Solution:

# Ensure previous image pushed to registry
gcloud builds submit --config=cloudbuild.yaml

# Check if image exists
gcloud artifacts docker images list --repository=images

# If not, first build takes full time (rebuild cache for future)

---

References

• Cloud Build Triggers
• Build Configuration File Schema
• Substitute Variable Values
• Speeding Up Builds