GameDay Script Builder

# Game Day Script: Database primary failure
**Date:** 2026-06-09
**Status:** PLANNED

## Scenario
Database primary failure

## Component Under Test
PostgreSQL primary node

## Hypothesis
> Failover completes in < 30 seconds and no requests are lost

## Pre-conditions
- [ ] Notify stakeholders
- [ ] Verify monitoring dashboards
- [ ] Record baseline metrics
- [ ] Ensure rollback plan is ready
- [ ] Have runbook open

## Failure Injection
```bash
kill -9 on postgres process
```

## Observation Points
- Error rate (SLO dashboard)
- Response latency (P95, P99)
- Alert firing time
- Failover completion time

## Success Criteria
Error rate < 0.1% and recovery < 30s

## Rollback Procedure
```bash
Restart postgres service and promote replica
```

## Post-Game Analysis
- [ ] Document actual vs expected
- [ ] File improvement tickets
- [ ] Update runbooks
- [ ] Share results with wider team