# Test Isolation & Production Data Safety

## ✅ **Production Data Protection**

### How Tests Are Isolated

**Testcontainers** provides superior isolation by giving each test its own PostgreSQL container:

1. **Container Isolation**: Each test gets a dedicated PostgreSQL + PostGIS container
   - No shared databases between tests
   - Complete process isolation (separate containers)
   - Automatic cleanup when test finishes

2. **Connection Flow**:
   ```
   Test → Starts PostgreSQL container → Runs migrations → Executes test → Stops & removes container
   ```

3. **Production Database**: Zero risk - containers are completely separate from any production setup

### Verification

Check that only production database exists:
```bash
docker exec turash-postgres psql -U turash -d postgres -c "SELECT datname FROM pg_database WHERE datname NOT IN ('postgres', 'template0', 'template1');"
```

Expected output:
```
 datname
---------
 turash
```

**No test databases should appear** - they're created and destroyed automatically.

## 🔒 **Safety Measures**

### 1. Database Credentials
- Tests use the same PostgreSQL server but connect to `postgres` database (admin)
- Production database `turash` is never specified in test configuration
- User `turash` has `CREATEDB` privileges (required for pgtestdb)

### 2. Automatic Cleanup
- pgtestdb automatically drops test databases after each test
- Uses Go's `testing.TB.Cleanup()` mechanism
- Even if a test crashes, cleanup runs

### 3. Template Database Reuse
- First test creates a template database with migrations
- Subsequent tests clone from template (fast, milliseconds)
- Template is reused across test runs
- Only recreated if migrations change (Hash() method detects changes)

## 📦 **Backup Strategy**

### Automatic Backups

Before running tests or making database changes, create a backup using the Cobra CLI:

#### Dev Mode (Recommended for Local Development)

```bash
# Create backup using Docker Compose configuration
make db-backup

# Or directly
go run ./cmd/backup --dev
```

#### Environment Variables

```bash
# Create backup using environment variables
make db-backup-env

# Or directly
go run ./cmd/backup
```

#### Connection String

```bash
# Create backup using connection string
go run ./cmd/backup --conn "postgres://user:pass@host:port/db"
```

### Backup Features

- ✅ **Timestamped**: Each backup has unique timestamp
- ✅ **Compressed**: Uses gzip compression (saves ~70% space)
- ✅ **Auto-cleanup**: Keeps last 10 backups automatically (configurable)
- ✅ **Safety**: Creates backup before restore operations
- ✅ **Multiple Sources**: Supports dev mode, env vars, or connection string

### Backup Location

Backups are stored in `./backups/` directory (configurable):
```
backups/
├── turash_backup_20250124_120000.sql.gz
├── turash_backup_20250124_130000.sql.gz
└── ...
```

### Restore from Backup

```bash
# Restore database (dev mode)
make db-restore BACKUP=backups/turash_backup_20250124_120000.sql.gz

# Or directly
go run ./cmd/backup restore backups/turash_backup_20250124_120000.sql.gz --dev

# Using environment variables
go run ./cmd/backup restore backups/turash_backup_20250124_120000.sql.gz

# Using connection string
go run ./cmd/backup restore backups/turash_backup_20250124_120000.sql.gz --conn "postgres://..."
```

**⚠️ Warning**: Restore will REPLACE all data. A safety backup is created automatically before restore.

## 🧪 **Test Execution Flow**

### What Happens When You Run Tests

1. **Test Starts**: `SetupTestDBWithTestcontainers(t)` is called
2. **Container Creation**: Spins up isolated PostgreSQL + PostGIS container
3. **Database Setup**: Creates fresh database with all migrations applied
4. **PostGIS Setup**: Enables PostGIS extension and runs spatial migrations
5. **Test Database**: Clones template → `pgtestdb_<random>`
6. **Test Execution**: Your test code runs against cloned database
7. **Cleanup**: Test database is automatically dropped
8. **Production**: `turash` database never touched

### Performance

- **First Test**: ~2-5 seconds (creates template)
- **Subsequent Tests**: ~50-200ms (clones template)
- **Parallel Tests**: Safe - each gets unique database

## 🔍 **Monitoring & Verification**

### Check Test Databases (During Test Run)

```bash
# While tests are running, you might see temporary databases
docker exec turash-postgres psql -U turash -d postgres -c "SELECT datname FROM pg_database WHERE datname LIKE 'pgtestdb%';"
```

These should disappear after tests complete.

### Verify Production Database Integrity

```bash
# Check production database exists and is healthy
docker exec turash-postgres psql -U turash -d turash -c "SELECT COUNT(*) FROM organizations;"

# Check no test databases remain
docker exec turash-postgres psql -U turash -d postgres -c "SELECT datname FROM pg_database WHERE datname LIKE 'pgtestdb%';"
# Should return: (0 rows)
```

## 📋 **Best Practices**

1. ✅ **Always backup before major changes**: `make db-backup`
2. ✅ **Run tests frequently**: They're isolated and safe
3. ✅ **Monitor backup directory**: Ensure backups are being created
4. ✅ **Verify isolation**: Check that only `turash` database exists after tests
5. ✅ **Use environment variables**: Override defaults if needed for CI/CD

## 🚨 **Troubleshooting**

### Test Databases Not Cleaning Up

If you see lingering `pgtestdb_*` databases:

```bash
# List them
docker exec turash-postgres psql -U turash -d postgres -c "SELECT datname FROM pg_database WHERE datname LIKE 'pgtestdb%';"

# Manually drop (if needed)
docker exec turash-postgres psql -U turash -d postgres -c "DROP DATABASE IF EXISTS pgtestdb_<name>;"
```

### Permission Errors

If tests fail with permission errors:

```bash
# Grant CREATEDB privilege (already granted for turash user)
docker exec turash-postgres psql -U turash -d postgres -c "ALTER USER turash CREATEDB;"
```

### Template Database Issues

If template database is corrupted:

```bash
# Drop template databases (they'll be recreated)
docker exec turash-postgres psql -U turash -d postgres -c "DROP DATABASE IF EXISTS pgtestdb_template_*;"
```

## 📚 **References**

- [pgtestdb GitHub](https://github.com/peterldowns/pgtestdb) - Official documentation
- [PostgreSQL CREATE DATABASE](https://www.postgresql.org/docs/current/sql-createdatabase.html)
- [Testing Best Practices](https://www.postgresql.org/developer/testing/)