turash/concept/12_go_125_stack_backend_architecture.md

## 10. Go 1.25 Stack & Backend Architecture

### Recommended Stack

**Core Stack (MVP)**: Go 1.25 + Neo4j + NATS/Redis Streams + PostgreSQL + Redis

**Core Stack (Scale)**: Go 1.25 + Neo4j + Kafka + PostgreSQL + Redis

### HTTP Framework Selection

**Options** (Choose based on requirements):
- **Fiber**: Fast, Express-inspired, lowest latency
- **Gin**: Mature, widely adopted, good balance
- **Echo**: Clean API, good middleware support
- **Standard `net/http`**: Simple, zero dependencies, full control

**Recommendation**: Start with **Gin** for MVP (mature ecosystem), consider **Fiber** if low latency critical

### API Gateway

- **Kong** (written in Lua/Go plugin support) or **Traefik** (Go-native)
- **Alternative**: Build lightweight gateway in Go using `net/http` or Gin
- Rate limiting, request routing, authentication
- API versioning support

### Message Queue & Event Streaming

**MVP Recommendation**: Start with NATS or Redis Streams, migrate to Kafka at scale

- **NATS** (Recommended for MVP): Go-native messaging (`nats.go`)
  - **Benefits**: 60-70% complexity reduction vs Kafka, similar capabilities
  - **Use Case**: Perfect for MVP phase, real-time updates, pub/sub
  - **Library**: `github.com/nats-io/nats.go`
  
- **Redis Streams** (Alternative MVP): Simple pub/sub, job queues
  - **Benefits**: Minimal infrastructure overhead, integrates with existing Redis cache
  - **Use Case**: Initial real-time features, job queues
  - **Library**: `go-redis/redis/v9`
  
- **Kafka** (Scale Phase): Industry standard for event streaming
  - **Migration Trigger**: When platform reaches 1000+ businesses
  - **Use Case**: High-throughput event streaming, event sourcing
  - **Libraries**: `confluent-kafka-go` or `shopify/sarama`
  
- **RabbitMQ**: `streadway/amqp` for traditional message queues (not recommended)

**Background Jobs**: Use Go's `context` and goroutines, or `asynq` for distributed job queues

**Decision Framework**:
- **< 100 businesses**: Redis Streams (simplest)
- **100-1000 businesses**: NATS (balanced performance/complexity)
- **> 1000 businesses**: Kafka (enterprise-grade, high-throughput)

### Database Layer

1. **Primary Graph DB**: Neo4j using `github.com/neo4j/neo4j-go-driver/v5`
   - Connection pooling built-in
   - Transaction support
   - Prepared statements for performance

2. **Secondary RDBMS**: PostgreSQL using `github.com/jackc/pgx/v5`
   - Better performance than `database/sql`
   - Native PostGIS support via `github.com/twpayne/go-geom`
   - Connection pooling with `pgxpool`

3. **Time-Series**: 
   - TimescaleDB (PostgreSQL extension) - use `pgx` driver
   - InfluxDB using `github.com/influxdata/influxdb-client-go/v2`

4. **Cache**: Redis using `github.com/redis/go-redis/v9`
   - Match results, sessions, rate limiting
   - Pub/sub for real-time updates

5. **Search**: 
   - **Meilisearch**: `github.com/meilisearch/meilisearch-go` (Go-native, fast)
   - Elasticsearch: `github.com/elastic/go-elasticsearch/v8`
   - **Alternative**: PostgreSQL full-text search for simpler deployments

### Go 1.25 Specific Features & Performance Targets

**Critical: Upgrade to Go 1.25**

**Performance Benchmarks** (Production Targets):
- **Throughput**: 10,000+ HTTP requests/second
- **Latency**: p95 <50ms API response time
- **Memory**: <100MB baseline, <200MB peak per instance
- **CPU**: <20% utilization at 1,000 req/s
- **Concurrency**: 10,000+ goroutines supported

1. **Experimental JSON v2 Package**:
   ```go
   // Enable with: GOEXPERIMENT=jsonv2 go build
   // IMPORTANT: Build with feature flags and fallback to Go 1.23 stable features
   import "encoding/json/v2" // Feature flag: json_v2_enabled
   ```
   - **Performance**: 3-10x faster JSON processing (50μs → 5-15μs per request)
   - **Throughput**: 50,000+ JSON operations/second
   - **Use Case**: High-throughput API responses, message serialization
   - **Risk Mitigation**: Build feature flags for experimental features, fallback to Go 1.23 if not production-ready by Q1 2025

2. **GreenTea Garbage Collector**:
   ```bash
   # Enable with: GOEXPERIMENT=greenteagc go build
   # IMPORTANT: Feature flag required, fallback to standard GC
   ```
   - **Performance**: Reduces GC overhead by 10-40% (from 20% to 12-18% CPU)
   - **Latency**: 50% reduction in GC pause times (<1ms p99 pauses)
   - **Use Case**: Matching engine, event processors, graph query handlers
   - **Memory**: 15-30% reduction in heap allocations
   - **Risk Mitigation**: Feature flag implementation required, fallback to standard GC if experimental features not production-ready

3. **Container-Aware GOMAXPROCS**:
   - **Resource Utilization**: 90%+ CPU utilization in Kubernetes pods
   - **Auto-scaling**: Accurate horizontal pod autoscaling decisions
   - **Efficiency**: 25% improvement in resource allocation accuracy

4. **DWARF v5 Debug Information**:
   - **Binary Size**: 10-20% reduction in compiled binary size
   - **Build Time**: 15% faster compilation and linking
   - **Debugging**: Improved Delve debugging experience

5. **WaitGroup.Go Method**:
   ```go
   // Simplified goroutine creation
   var wg sync.WaitGroup
   wg.Go(func() { /* work */ })
   ```
   - **Code Quality**: 30% reduction in boilerplate concurrency code

6. **Trace Flight Recorder API**:
   ```go
   import "runtime/trace"
   // Continuous tracing with in-memory ring buffer
   ```
   - **Observability**: <1% performance overhead for continuous tracing
   - **Debugging**: Capture 1-hour execution traces in 50MB memory

### Go-Specific Libraries & Patterns

**Essential Libraries**:

1. **Validation**: `github.com/go-playground/validator/v10` or Go 1.25 generics for type-safe validation
2. **Configuration Management**: `github.com/spf13/viper`
3. **Logging**: `github.com/rs/zerolog` (fast, structured) or `github.com/sirupsen/logrus` (feature-rich)
4. **HTTP Client**: Standard `net/http` (Go 1.25 improvements) or `github.com/go-resty/resty/v2`
5. **Database Migration**: `github.com/golang-migrate/migrate/v4`
6. **Testing**: `github.com/stretchr/testify`, `github.com/golang/mock` or `github.com/vektra/mockery/v2`
7. **WebSocket**: `github.com/gorilla/websocket`, `nhooyr.io/websocket`, or `github.com/gobwas/ws`
8. **GraphQL**: `github.com/99designs/gqlgen` (schema-first) or `github.com/graphql-go/graphql` (runtime-first)
9. **gRPC**: `google.golang.org/grpc` for microservices
10. **Task Queues**: `github.com/hibiken/asynq` (Redis-based distributed task queue)
11. **Observability**: `go.opentelemetry.io/otel`, `github.com/prometheus/client_golang`

**Go-Specific Architecture Patterns**:

1. **Interface-Driven Design**: Accept interfaces, return structs
2. **Context Propagation**: Use `context.Context` for cancellation, timeouts, request-scoped values
3. **Error Handling**: Wrap errors with `fmt.Errorf("operation failed: %w", err)`, use `errors.Is()` and `errors.As()`
4. **Concurrency Patterns**: Channels for communication, `sync.WaitGroup` for coordination, worker pools for parallelism
5. **Graceful Shutdown**: Handle SIGTERM/SIGINT, drain connections, finish in-flight requests, cleanup resources

### Go Project Structure

```
/cmd          # Application entrypoints
/internal     # Private application code
/pkg          # Public library code
/api          # API definitions
/configs      # Configuration files
/scripts      # Build/deployment scripts
/docs         # Documentation including ADRs
```

---