turash/bugulma/backend/TEST_ISOLATION.md

Test Isolation & Production Data Safety

✅ Production Data Protection

How Tests Are Isolated

Testcontainers provides superior isolation by giving each test its own PostgreSQL container:

1. Container Isolation: Each test gets a dedicated PostgreSQL + PostGIS container

   • No shared databases between tests
   • Complete process isolation (separate containers)
   • Automatic cleanup when test finishes

2. Connection Flow:

   Test → Starts PostgreSQL container → Runs migrations → Executes test → Stops & removes container
   

3. Production Database: Zero risk - containers are completely separate from any production setup

Verification

Check that only production database exists:

docker exec turash-postgres psql -U turash -d postgres -c "SELECT datname FROM pg_database WHERE datname NOT IN ('postgres', 'template0', 'template1');"

Expected output:

 datname
---------
 turash

No test databases should appear - they're created and destroyed automatically.

🔒 Safety Measures

1. Database Credentials

• Tests use the same PostgreSQL server but connect to postgres database (admin)
• Production database turash is never specified in test configuration
• User turash has CREATEDB privileges (required for pgtestdb)

2. Automatic Cleanup

• pgtestdb automatically drops test databases after each test
• Uses Go's testing.TB.Cleanup() mechanism
• Even if a test crashes, cleanup runs

3. Template Database Reuse

• First test creates a template database with migrations
• Subsequent tests clone from template (fast, milliseconds)
• Template is reused across test runs
• Only recreated if migrations change (Hash() method detects changes)

📦 Backup Strategy

Automatic Backups

Before running tests or making database changes, create a backup using the Cobra CLI:

Dev Mode (Recommended for Local Development)

# Create backup using Docker Compose configuration
make db-backup

# Or directly
go run ./cmd/backup --dev

Environment Variables

# Create backup using environment variables
make db-backup-env

# Or directly
go run ./cmd/backup

Connection String

# Create backup using connection string
go run ./cmd/backup --conn "postgres://user:pass@host:port/db"

Backup Features

• ✅ Timestamped: Each backup has unique timestamp
• ✅ Compressed: Uses gzip compression (saves ~70% space)
• ✅ Auto-cleanup: Keeps last 10 backups automatically (configurable)
• ✅ Safety: Creates backup before restore operations
• ✅ Multiple Sources: Supports dev mode, env vars, or connection string

Backup Location

Backups are stored in ./backups/ directory (configurable):

backups/
├── turash_backup_20250124_120000.sql.gz
├── turash_backup_20250124_130000.sql.gz
└── ...

Restore from Backup

# Restore database (dev mode)
make db-restore BACKUP=backups/turash_backup_20250124_120000.sql.gz

# Or directly
go run ./cmd/backup restore backups/turash_backup_20250124_120000.sql.gz --dev

# Using environment variables
go run ./cmd/backup restore backups/turash_backup_20250124_120000.sql.gz

# Using connection string
go run ./cmd/backup restore backups/turash_backup_20250124_120000.sql.gz --conn "postgres://..."

⚠️ Warning: Restore will REPLACE all data. A safety backup is created automatically before restore.

🧪 Test Execution Flow

What Happens When You Run Tests

1. Test Starts: SetupTestDBWithTestcontainers(t) is called
2. Container Creation: Spins up isolated PostgreSQL + PostGIS container
3. Database Setup: Creates fresh database with all migrations applied
4. PostGIS Setup: Enables PostGIS extension and runs spatial migrations
5. Test Database: Clones template → pgtestdb_<random>
6. Test Execution: Your test code runs against cloned database
7. Cleanup: Test database is automatically dropped
8. Production: turash database never touched

Performance

• First Test: ~2-5 seconds (creates template)
• Subsequent Tests: ~50-200ms (clones template)
• Parallel Tests: Safe - each gets unique database

🔍 Monitoring & Verification

Check Test Databases (During Test Run)

# While tests are running, you might see temporary databases
docker exec turash-postgres psql -U turash -d postgres -c "SELECT datname FROM pg_database WHERE datname LIKE 'pgtestdb%';"

These should disappear after tests complete.

Verify Production Database Integrity

# Check production database exists and is healthy
docker exec turash-postgres psql -U turash -d turash -c "SELECT COUNT(*) FROM organizations;"

# Check no test databases remain
docker exec turash-postgres psql -U turash -d postgres -c "SELECT datname FROM pg_database WHERE datname LIKE 'pgtestdb%';"
# Should return: (0 rows)

📋 Best Practices

1. ✅ Always backup before major changes: make db-backup
2. ✅ Run tests frequently: They're isolated and safe
3. ✅ Monitor backup directory: Ensure backups are being created
4. ✅ Verify isolation: Check that only turash database exists after tests
5. ✅ Use environment variables: Override defaults if needed for CI/CD

🚨 Troubleshooting

Test Databases Not Cleaning Up

If you see lingering pgtestdb_* databases:

# List them
docker exec turash-postgres psql -U turash -d postgres -c "SELECT datname FROM pg_database WHERE datname LIKE 'pgtestdb%';"

# Manually drop (if needed)
docker exec turash-postgres psql -U turash -d postgres -c "DROP DATABASE IF EXISTS pgtestdb_<name>;"

Permission Errors

If tests fail with permission errors:

# Grant CREATEDB privilege (already granted for turash user)
docker exec turash-postgres psql -U turash -d postgres -c "ALTER USER turash CREATEDB;"

Template Database Issues

If template database is corrupted:

# Drop template databases (they'll be recreated)
docker exec turash-postgres psql -U turash -d postgres -c "DROP DATABASE IF EXISTS pgtestdb_template_*;"

📚 References

• pgtestdb GitHub - Official documentation
• PostgreSQL CREATE DATABASE
• Testing Best Practices