DNS Peering: Hybrid On-Premises Resolution

Tại sao điều này quan trọng

DNS peering là critical enabler cho hybrid cloud architectures. Nếu bạn có:

• On-premises data centers + GCP cloud
• Multiple cloud providers + GCP
• Corporate network + GCP resources

Thì DNS peering là infrastructure piece kế tiếp split-horizon DNS.

Production scenarios đòi hỏi DNS peering:

Scenario 1: Hybrid Enterprise
├── On-prem: dc.corp.internal (AD-integrated DNS)
├── GCP: gcp.corp.internal (Cloud DNS)
└── Need: On-prem servers access GCP services via DNS
           GCP services access on-prem resources via DNS

Scenario 2: Multi-Cloud
├── AWS: aws.corp.internal
├── GCP: gcp.corp.internal
└── Need: Resources across clouds resolve each other

Scenario 3: Gradual Migration
├── On-prem: legacy.example.com (BIND)
├── GCP: prod.example.com (Cloud DNS)
└── Need: During migration, both need work, no hardcoded IPs

Cost of getting DNS wrong:

• Manual IP hardcoding (anti-pattern)
• DNS timeouts causing application failures
• Operational complexity (on-call debugging)
• Security gaps (firewall rules based on domain names don't work)

DNS Peering Fundamentals

What is DNS Peering?

DNS peering = authorized cross-project DNS resolution. Cho phép VPC trong một project resolve zones từ VPC trong project khác.

Project A: VPC A
  ├── Private Zone "prod.internal.com"
  └── Bound to: VPC A

Project B: VPC B
  └── Cannot resolve prod.internal.com (initially)
      
With DNS Peering:
  Project B → VPC B can now resolve prod.internal.com
              (via Project A's zone)

DNS Peering vs Forwarding

Aspect	Peering	Forwarding
Mechanism	Zone replication/linking	Query redirection
Setup	Between GCP VPCs	To external nameservers
Performance	Cached, low latency	Per-query, higher latency
Scope	Entire zone	Specific queries
Use Case	Multi-VPC in GCP	On-prem/external DNS

Implementation: Inbound Peering

Architecture

Hub VPC (shared DNS infrastructure):
├── Private Zone "prod.internal.example.com"
├── Inbound Server (129.154.169.252)
│   └── Listens for peering queries

Spoke VPC (service projects):
├── Resolves prod.internal.example.com
├── Via inbound server 129.154.169.252
└── Gets answers from Hub zone

Setup Steps

# Step 1: Hub Project - Create inbound server
gcloud dns inbound-policies create hub-inbound \
  --networks=projects/HUB_PROJECT/global/networks/hub-vpc \
  --enable-inbound-forwarding

# Step 2: Get inbound server IP
gcloud compute addresses describe inbound-server-ip \
  --region=us-central1 \
  --project=HUB_PROJECT
# Returns: 129.154.169.252

# Step 3: Spoke Project - Create forwarding zone
gcloud dns managed-zones create peered-zone \
  --dns-name=prod.internal.example.com \
  --visibility=private \
  --networks=projects/SPOKE_PROJECT/global/networks/spoke-vpc

# Step 4: Add forwarding target
gcloud dns managed-zones update peered-zone \
  --inbound-forwarding-servers=129.154.169.252 \
  --project=SPOKE_PROJECT

# Step 5: Verify from spoke VPC
# SSH to VM in spoke VPC, then:
nslookup service.prod.internal.example.com
# Should resolve to IP from hub's zone

Terraform Implementation

# Hub Project
resource "google_dns_managed_zone" "hub_prod" {
  project  = "hub-project"
  name     = "hub-prod-zone"
  dns_name = "prod.internal.example.com."
  
  private_visibility_config {
    networks_list {
      network_url = google_compute_network.hub_vpc.id
    }
  }
}

resource "google_dns_inbound_forwarding_policy" "hub_policy" {
  project = "hub-project"
  name    = "hub-inbound-policy"
  
  networks {
    network_url = google_compute_network.hub_vpc.id
  }
  
  inbound_forwarding_servers {
    address = "129.154.169.252"
  }
}

# Spoke Project
resource "google_dns_managed_zone" "spoke_peered" {
  project  = "spoke-project"
  name     = "spoke-peered-zone"
  dns_name = "prod.internal.example.com."
  
  private_visibility_config {
    networks_list {
      network_url = google_compute_network.spoke_vpc.id
    }
  }
  
  peering_config {
    target_network {
      network_url = "projects/hub-project/global/networks/hub-vpc"
    }
  }
}

Implementation: Outbound Peering (Bidirectional)

Nếu both projects cần resolve each other zones:

Hub VPC:
  ├── Zone "prod.internal.example.com"
  └── Outbound forwarding to 129.154.169.253 (spoke inbound)

Spoke VPC:
  ├── Zone "staging.internal.example.com"
  └── Outbound forwarding to 129.154.169.252 (hub inbound)

Result: Bidirectional resolution

Setup

# Hub Project - Create outbound policy
gcloud dns outbound-policies create hub-outbound \
  --networks=projects/HUB_PROJECT/global/networks/hub-vpc \
  --outbound-forwarding-servers=129.154.169.253  # Spoke's inbound

# Hub Project - Create forwarding zone untuk spoke domain
gcloud dns managed-zones create hub-staging-peered \
  --dns-name=staging.internal.example.com \
  --visibility=private \
  --networks=projects/HUB_PROJECT/global/networks/hub-vpc

gcloud dns managed-zones update hub-staging-peered \
  --inbound-forwarding-servers=129.154.169.253

# Spoke Project - Similar setup (reverse)
gcloud dns inbound-policies create spoke-inbound \
  --networks=projects/SPOKE_PROJECT/global/networks/spoke-vpc \
  --enable-inbound-forwarding

gcloud dns outbound-policies create spoke-outbound \
  --networks=projects/SPOKE_PROJECT/global/networks/spoke-vpc \
  --outbound-forwarding-servers=129.154.169.252  # Hub's inbound

Hybrid: On-Premises Integration Pattern

Architecture: Hub-Spoke + On-Prem

On-Premises Data Center
├── DNS Server 192.168.1.10
├── Zone "corp.internal"
└── Services: AD, databases, legacy apps

GCP Hub Project
├── Private Zone "gcp.internal.com"
├── Inbound Server 129.154.169.252 (GCP → spoke peering)
├── Outbound Forwarding to 192.168.1.10 (GCP → on-prem)
└── Cloud Interconnect/VPN link to on-prem

GCP Spoke Projects
├── VPCs peering with Hub
├── Can resolve gcp.internal.com (via hub peering)
├── Can resolve corp.internal (via hub → on-prem)
└── Shared VPC for network connectivity

Setup: Cloud DNS + On-Prem DNS

# Hub Project - Forwarding zone untuk on-prem domain
gcloud dns managed-zones create onprem-forwarding \
  --dns-name=corp.internal \
  --visibility=private \
  --networks=projects/HUB_PROJECT/global/networks/hub-vpc

gcloud dns managed-zones update onprem-forwarding \
  --inbound-forwarding-servers=192.168.1.10  # On-prem DNS

# Hub Project - Inbound policy (for spoke peering)
gcloud dns inbound-policies create hub-inbound \
  --networks=projects/HUB_PROJECT/global/networks/hub-vpc \
  --enable-inbound-forwarding

# Spoke Project - Peering policy
gcloud dns managed-zones create spoke-peered \
  --dns-name=gcp.internal.com \
  --visibility=private \
  --networks=projects/SPOKE_PROJECT/global/networks/spoke-vpc

gcloud dns managed-zones update spoke-peered \
  --inbound-forwarding-servers=129.154.169.252  # Hub's inbound

# Spoke Project - Also need forward to on-prem via hub
gcloud dns managed-zones create spoke-onprem \
  --dns-name=corp.internal \
  --visibility=private \
  --networks=projects/SPOKE_PROJECT/global/networks/spoke-vpc

gcloud dns managed-zones update spoke-onprem \
  --inbound-forwarding-servers=129.154.169.252  # Ask hub, which forwards to on-prem

Resolution Flow: End-to-End

Spoke VPC Resource:
  Query: db.corp.internal

  Step 1: Spoke VPC resolver
  Step 2: Check spoke-onprem zone
  Step 3: Forwarding target: 129.154.169.252 (hub)
  Step 4: Hub receives query
  Step 5: Check hub-onprem zone  
  Step 6: Forwarding target: 192.168.1.10 (on-prem DNS)
  Step 7: On-prem DNS responds: 192.168.1.100
  Step 8: Response propagates back: spoke → hub → spoke resource
  Step 9: Result: 192.168.1.100

Latency: ~5-10ms per hop (acceptable, under 50ms typical).

Production Patterns

Pattern 1: Hub-and-Spoke DNS

Central Hub Project:
├── All DNS infrastructure
├── Private zones for shared services
├── Inbound peering endpoints
└── Firewall rules allow spoke → hub traffic

Spoke Projects (dev, staging, prod):
├── Peer with hub for shared zones
├── Local private zones for team services
└── Outbound to hub for external zone access

Benefits:

• ✓ Centralized DNS management
• ✓ Consistent zone naming
• ✓ Single point of security policy
• ✓ Easy to add/remove teams

Pattern 2: Hybrid Gradually Migration

Phase 1: On-Prem Only
├── All services on-prem
├── All DNS resolution on-prem
└── No GCP integration

Phase 2: Cloud Bursting
├── New services in GCP
├── On-prem DNS forwards to GCP DNS
├── GCP DNS forwards to on-prem DNS
└── Bidirectional resolution working

Phase 3: Cloud Primary
├── Most services in GCP
├── GCP DNS primary
├── On-prem DNS deprecated (but still reachable)

Phase 4: Cloud Only
├── All services in GCP
├── On-prem DNS shutdown

Pattern 3: Multi-Cloud

GCP
├── Zone gcp.services.internal
├── Inbound server 129.154.169.252

AWS
├── Route 53 zone aws.services.internal
├── Conditional forwarder to GCP inbound

Result: Seamless cross-cloud resolution

Troubleshooting

Issue 1: Spoke Cannot Resolve Hub Zone

# From spoke VPC VM:
nslookup service.prod.internal.example.com

# Debugging:
1. Check peering zone exists in spoke project
   gcloud dns managed-zones list --project=spoke-project

2. Check inbound forwarding target
   gcloud dns managed-zones describe peered-zone \
     --project=spoke-project

3. Check hub inbound server IP
   gcloud compute addresses describe inbound-server-ip \
     --region=us-central1 \
     --project=hub-project

4. Test connectivity (VPN/Interconnect must be active)
   gcloud compute ssh VM --zone=us-central1-a --tunnel-through-iap
   ping 129.154.169.252  # Should work

5. Check firewall rules
   gcloud compute firewall-rules list \
     --filter="sourceRanges:spoke-vpc-subnet" \
     --format="table(name,allowed[].IPProtocol,allowed[].ports)"

Issue 2: Resolution Loops (Infinite Forwarding)

Scenario:
  Zone A forwards to Server B
  Server B forwards back to Zone A
  → Infinite loop

Debug:
  tcpdump -i eth0 port 53
  → See query bouncing back-and-forth

Solution:
  Review forwarding targets
  Ensure not circular: A → B → A

Issue 3: On-Prem Cannot Reach GCP Services

# From on-prem DNS server:
nslookup service.gcp.internal.example.com

# Debugging:
1. On-prem DNS must be configured to forward unknown zones
   Example BIND config:
   
   zone "gcp.internal.example.com" {
     type forward;
     forward only;
     forwarders { 129.154.169.252; };
   };

2. Network connectivity must allow DNS queries (port 53)
   firewall rules on GCP side must allow on-prem CIDR

3. Verify VPN/Interconnect status
   gcloud compute interconnects list
   gcloud compute vpn-tunnels list

Performance Considerations

Latency

Single-VPC resolution:     ~1-2ms (cached)
Peering resolution (1 hop): ~5-10ms (1 forwarding)
Hybrid resolution (2 hops): ~10-20ms (hub → on-prem)

Typical: Still acceptable, under SLA budgets.

Caching

Peering doesn't prevent caching:

Spoke VM resolves service.prod.internal.example.com
  → Hub forwards → Gets answer
  → Result cached locally
  → Next query hit local cache (instant)

TTL determines cache expiration.

Query Load

If hub is centralized DNS point for 100 spoke projects:
  100 projects × 1000 queries/sec = 100k QPS to hub

Risk: Hub becomes bottleneck.

Mitigation:
  1. Cloud DNS handles millions QPS (usually not issue)
  2. Use regional peering if available (not all regions support)
  3. Consider separate hub per region (us-central1 hub, eu-west1 hub)

Security Considerations

Access Control

# Only specific spokes can peer with hub
gcloud dns inbound-policies create hub-restricted \
  --networks=projects/HUB/global/networks/hub \
  --enable-inbound-forwarding

# Hub zone IAM: Only hub admins can modify
gcloud dns managed-zones add-iam-policy-binding hub-prod-zone \
  --member=group:dns-admins@company.com \
  --role=roles/dns.admin

DNS Hijacking Prevention

On-prem DNS server could be compromised:
  Query: service.example.com → 192.168.1.10 (on-prem) → returns malicious IP

Mitigation:
  1. DNSSEC validation (if enabled)
  2. Monitor DNS queries for anomalies
  3. Use VPN/Interconnect (encrypted)
  4. Regularly audit DNS records

Monitoring & Logging

# Enable DNS query logging
gcloud dns policies create log-policy \
  --description="Log all queries" \
  --enable-logging \
  --log-network=projects/HUB/global/networks/hub-vpc

# View logs
gcloud logging read "resource.type=dns_policy" \
  --limit=50 \
  --format=json | jq '.[] | {timestamp: .timestamp, queryName: .payload_proto.query_name}'

# Alert on peering query failures
# (Setup via Cloud Monitoring)

Checklists

Pre-Deployment

• [ ] Hub and spoke VPCs ready
• [ ] VPN/Interconnect between hub and on-prem (if hybrid)
• [ ] Network routes between VPCs configured
• [ ] Firewall rules allow DNS traffic (port 53)

Implementation

• [ ] Inbound peering policy created on hub
• [ ] Peering zones created on spokes
• [ ] Forwarding zones created (if on-prem)
• [ ] Testing from spoke → hub → on-prem (end-to-end)

Post-Deployment

• [ ] Query logging enabled
• [ ] Monitoring alerts configured
• [ ] Failover tested (if high availability needed)
• [ ] Documentation updated
• [ ] Team trained on troubleshooting

References

• GCP DNS Peering Documentation
• Cloud DNS Inbound Forwarding
• Hybrid Connectivity Best Practices
• VPN DNS Configuration