Budget Alerts: Programmatic Controls & Automation

Cloud Billing budgets cho phép bạn set cost thresholds và trigger actions khi vượt quá. Không chỉ email alerts, bạn có thể automate responses — disable billing, scale down resources, notify Slack, v.v.

---

Budget Basics

Manual Budget Creation (Console)

Billing → Budgets & Alerts → Create Budget
  1. Select billing account
  2. Set amount ($10,000/month)
  3. Set threshold (80%, 100%)
  4. Add recipients (email)

Programmatic Budget API

from google.cloud.billing_v1 import BudgetServiceClient, Budget

client = BudgetServiceClient()

budget = Budget(
    display_name="Q3 Production Budget",
    billing_account="billingAccounts/123456",
    budget_filter={
        "projects": ["projects/my-project"]
    },
    amount={"specified_amount": {"currency_code": "USD", "units": 10000}},
    threshold_rules=[
        {"percent": 50},
        {"percent": 90},
        {"percent": 100},
        {"percent": 110}
    ]
)

response = client.create_budget(
    name="billingAccounts/123456",
    budget=budget
)

---

Budget Mechanics

Threshold Rules

Budget: $10,000/month

Threshold rules:
  - 50% ($5,000): Warn (email)
  - 80% ($8,000): Alert (email)
  - 100% ($10,000): Critical (email + notification)
  - 110% ($11,000): Overage (email + notification)

Metric Types

1. Current Spend
   - Actual cost so far this month
   - Updated ~24h delayed (not real-time)

2. Forecasted Spend
   - Projection of month-end cost
   - Based on trend (linear extrapolation)
   - Less accurate for variable workloads

---

Pub/Sub Notifications: Programmatic Alerts

Setup Pub/Sub Integration

# 1. Create Pub/Sub topic
gcloud pubsub topics create billing-alerts

# 2. Create Budget with Pub/Sub notifications
gcloud billing budgets create \
  --billing-account=ACCOUNT_ID \
  --display-name="Cost Alert" \
  --budget-amount=10000 \
  --threshold-rule=percent=100 \
  --pubsub-topic=projects/PROJECT_ID/topics/billing-alerts

Message Format

When budget threshold exceeded, Pub/Sub message:

{
  "budgetDisplayName": "Q3 Production Budget",
  "alertThresholdExceeded": 1.0,
  "forecastedSpend": 12000.50,
  "costAmount": 10100.25,
  "budgetAmount": 10000,
  "currency": "USD",
  "timestamp": "2026-06-25T14:30:00Z"
}

Cloud Function to Process Alert

import json
import functions_framework
from slack_sdk import WebClient

slack_client = WebClient(token=os.environ.get("SLACK_BOT_TOKEN"))

@functions_framework.cloud_event
def alert_budget_exceeded(cloud_event):
    message = json.loads(base64.b64decode(cloud_event.data["message"]["data"]))
    
    budget_name = message.get("budgetDisplayName")
    cost_amount = message.get("costAmount")
    budget_amount = message.get("budgetAmount")
    threshold = message.get("alertThresholdExceeded")
    
    # Send Slack alert
    slack_client.chat_postMessage(
        channel="#billing-alerts",
        text=f":warning: Budget Alert!\n"
             f"Budget: {budget_name}\n"
             f"Spent: ${cost_amount:.2f} / ${budget_amount:.2f}\n"
             f"Threshold exceeded: {threshold*100:.0f}%"
    )
    
    # If critical, take action
    if cost_amount > budget_amount * 1.2:  # 20% overage
        disable_non_production_resources()
        slack_client.chat_postMessage(
            channel="#billing-alerts",
            text=":stop_sign: Cost critical! Disabled dev/test resources."
        )

---

Automated Cost Control

Pattern 1: Disable Billing on Project (Nuclear Option)

from google.cloud import billing_v1

billing_client = billing_v1.BillingAccountsClient()

# Disable billing on project when budget exceeded
def disable_billing_on_project(project_id):
    result = billing_client.update_billing_account_patch(
        name=f"projects/{project_id}",
        billing_account_name=None  # Disassociate billing
    )
    print(f"Billing disabled on {project_id}")

Caveat: This stops all services (VM, database, etc). Use carefully.

Pattern 2: Scale Down Resources

import google.cloud.compute_v1 as compute

def scale_down_dev_instances():
    """Stop dev/test VMs to save cost"""
    client = compute.InstancesClient()
    
    zones = ['us-central1-a', 'us-central1-b']
    for zone in zones:
        for instance in client.list(project=PROJECT_ID, zone=zone):
            if instance.labels.get('env') == 'dev':
                print(f"Stopping {instance.name} (dev instance)")
                client.stop(project=PROJECT_ID, zone=zone, resource=instance.name)

Pattern 3: Reduce Autoscaler Capacity

import google.cloud.container_v1 as container

def reduce_gke_cluster_capacity():
    """Reduce GKE cluster size"""
    client = container.ClusterManagerClient()
    
    cluster = client.get_cluster(
        name=f"projects/{PROJECT_ID}/zones/{ZONE}/clusters/{CLUSTER_NAME}"
    )
    
    # Reduce node pool size by 50%
    for node_pool in cluster.node_pools:
        if node_pool.initial_node_count > 3:
            update = {
                "nodePool": node_pool.name,
                "initialNodeCount": max(3, node_pool.initial_node_count // 2)
            }
            # Apply update...

---

Budget Alerts in Production

Multi-Tier Budget Strategy

Organization level:
  - Budget: $100k/month
  - Threshold: 100%
  - Action: Notify ops team, freeze new resources

Project level (prod):
  - Budget: $50k/month
  - Threshold: 90%, 100%, 110%
  - Action: Alert via Slack, notify team lead

Project level (dev):
  - Budget: $10k/month
  - Threshold: 100%
  - Action: Disable non-critical resources, notify team

Team level (optional):
  - Budget: $5k/month per team
  - Threshold: 100%
  - Action: Chargeback, notify finance

Escalation Logic

Cost at 80% budget → WARNING email
Cost at 100% budget → CRITICAL email + Slack + page oncall
Cost at 110% budget → CRITICAL + stop dev resources

Example implementation:
```python
def escalate_budget_alert(cost, budget):
    percent = cost / budget
    
    if percent > 1.1:
        disable_dev_resources()
        page_oncall()
    elif percent > 1.0:
        notify_slack_critical()
    elif percent > 0.8:
        send_email_warning()

---

Forecasting & Budget Adjustment

Dynamic Budget Adjustment

from google.cloud import billing_v1

def adjust_budget_monthly():
    """Adjust budget based on actual spend trend"""
    
    # Query last 3 months spend
    spend_history = query_billing_export(
        date_range=last_90_days
    )
    
    avg_spend = sum(spend_history) / len(spend_history)
    trend = (spend_history[-1] - spend_history[0]) / len(spend_history)
    
    # Forecast next month
    forecasted_spend = avg_spend + trend
    
    # Set budget with 10% buffer
    new_budget = forecasted_spend * 1.1
    
    # Update budget
    client = billing_v1.BudgetServiceClient()
    client.update_budget(
        budget=Budget(..., amount={"units": int(new_budget)})
    )
    
    print(f"Budget adjusted to ${new_budget:.0f}")

---

Tracking Budget vs Actual

Budget Compliance Report

SELECT
  DATE_TRUNC(usage_start_time, MONTH) as month,
  SUM(cost) as actual_spend,
  10000 as budgeted_amount,  -- From budget API
  ROUND((SUM(cost) / 10000) * 100, 1) as percent_of_budget
FROM `project.billing_dataset.gcp_billing_export_v1`
GROUP BY month
ORDER BY month DESC;

Output:

month     | actual_spend | budgeted | percent_of_budget
2026-06   | $9,500       | $10,000  | 95%
2026-05   | $11,200      | $10,000  | 112%
2026-04   | $8,900       | $10,000  | 89%

---

Anti-Patterns

Anti-Pattern 1: Overly Aggressive Budget

Mistake: Set budget = current spend (no buffer).

Impact: Constant alerts, teams desensitized.

Fix: Budget = 120% of trending spend (20% buffer).

Anti-Pattern 2: Disabling Billing as Emergency Brake

Mistake: Budget exceeded → disable billing (auto).

Impact: Production outage, data corruption, SLA violation.

Fix: Disable only non-production. Manual gate for prod.

Anti-Pattern 3: Ignoring Forecasted Spend

Mistake: Monitor only actual spend (retrospective).

Impact: Surprise at month-end when forecasted > budget.

Fix: Alert on forecasted spend crossing threshold.

---

Budget Integration with FinOps

FinOps Hub Dashboard

GCP FinOps Hub auto-integrates budget data:

Dashboard shows:
  - Budget progress (% of monthly budget consumed)
  - Forecasted spend (trend-based projection)
  - Top cost drivers (by service, project, label)
  - Optimization opportunities (recommendations)

---

References

• Cloud Billing Budget API
• Create, Edit, or Delete Budgets
• Programmatic Budget Notifications
• Disable Billing on Project