## 10. Go 1.25 Stack & Backend Architecture ### Recommended Stack **Core Stack (MVP)**: Go 1.25 + Neo4j + NATS/Redis Streams + PostgreSQL + Redis **Core Stack (Scale)**: Go 1.25 + Neo4j + Kafka + PostgreSQL + Redis ### HTTP Framework Selection **Options** (Choose based on requirements): - **Fiber**: Fast, Express-inspired, lowest latency - **Gin**: Mature, widely adopted, good balance - **Echo**: Clean API, good middleware support - **Standard `net/http`**: Simple, zero dependencies, full control **Recommendation**: Start with **Gin** for MVP (mature ecosystem), consider **Fiber** if low latency critical ### API Gateway - **Kong** (written in Lua/Go plugin support) or **Traefik** (Go-native) - **Alternative**: Build lightweight gateway in Go using `net/http` or Gin - Rate limiting, request routing, authentication - API versioning support ### Message Queue & Event Streaming **MVP Recommendation**: Start with NATS or Redis Streams, migrate to Kafka at scale - **NATS** (Recommended for MVP): Go-native messaging (`nats.go`) - **Benefits**: 60-70% complexity reduction vs Kafka, similar capabilities - **Use Case**: Perfect for MVP phase, real-time updates, pub/sub - **Library**: `github.com/nats-io/nats.go` - **Redis Streams** (Alternative MVP): Simple pub/sub, job queues - **Benefits**: Minimal infrastructure overhead, integrates with existing Redis cache - **Use Case**: Initial real-time features, job queues - **Library**: `go-redis/redis/v9` - **Kafka** (Scale Phase): Industry standard for event streaming - **Migration Trigger**: When platform reaches 1000+ businesses - **Use Case**: High-throughput event streaming, event sourcing - **Libraries**: `confluent-kafka-go` or `shopify/sarama` - **RabbitMQ**: `streadway/amqp` for traditional message queues (not recommended) **Background Jobs**: Use Go's `context` and goroutines, or `asynq` for distributed job queues **Decision Framework**: - **< 100 businesses**: Redis Streams (simplest) - **100-1000 businesses**: NATS (balanced performance/complexity) - **> 1000 businesses**: Kafka (enterprise-grade, high-throughput) ### Database Layer 1. **Primary Graph DB**: Neo4j using `github.com/neo4j/neo4j-go-driver/v5` - Connection pooling built-in - Transaction support - Prepared statements for performance 2. **Secondary RDBMS**: PostgreSQL using `github.com/jackc/pgx/v5` - Better performance than `database/sql` - Native PostGIS support via `github.com/twpayne/go-geom` - Connection pooling with `pgxpool` 3. **Time-Series**: - TimescaleDB (PostgreSQL extension) - use `pgx` driver - InfluxDB using `github.com/influxdata/influxdb-client-go/v2` 4. **Cache**: Redis using `github.com/redis/go-redis/v9` - Match results, sessions, rate limiting - Pub/sub for real-time updates 5. **Search**: - **Meilisearch**: `github.com/meilisearch/meilisearch-go` (Go-native, fast) - Elasticsearch: `github.com/elastic/go-elasticsearch/v8` - **Alternative**: PostgreSQL full-text search for simpler deployments ### Go 1.25 Specific Features & Performance Targets **Critical: Upgrade to Go 1.25** **Performance Benchmarks** (Production Targets): - **Throughput**: 10,000+ HTTP requests/second - **Latency**: p95 <50ms API response time - **Memory**: <100MB baseline, <200MB peak per instance - **CPU**: <20% utilization at 1,000 req/s - **Concurrency**: 10,000+ goroutines supported 1. **Experimental JSON v2 Package**: ```go // Enable with: GOEXPERIMENT=jsonv2 go build // IMPORTANT: Build with feature flags and fallback to Go 1.23 stable features import "encoding/json/v2" // Feature flag: json_v2_enabled ``` - **Performance**: 3-10x faster JSON processing (50μs → 5-15μs per request) - **Throughput**: 50,000+ JSON operations/second - **Use Case**: High-throughput API responses, message serialization - **Risk Mitigation**: Build feature flags for experimental features, fallback to Go 1.23 if not production-ready by Q1 2025 2. **GreenTea Garbage Collector**: ```bash # Enable with: GOEXPERIMENT=greenteagc go build # IMPORTANT: Feature flag required, fallback to standard GC ``` - **Performance**: Reduces GC overhead by 10-40% (from 20% to 12-18% CPU) - **Latency**: 50% reduction in GC pause times (<1ms p99 pauses) - **Use Case**: Matching engine, event processors, graph query handlers - **Memory**: 15-30% reduction in heap allocations - **Risk Mitigation**: Feature flag implementation required, fallback to standard GC if experimental features not production-ready 3. **Container-Aware GOMAXPROCS**: - **Resource Utilization**: 90%+ CPU utilization in Kubernetes pods - **Auto-scaling**: Accurate horizontal pod autoscaling decisions - **Efficiency**: 25% improvement in resource allocation accuracy 4. **DWARF v5 Debug Information**: - **Binary Size**: 10-20% reduction in compiled binary size - **Build Time**: 15% faster compilation and linking - **Debugging**: Improved Delve debugging experience 5. **WaitGroup.Go Method**: ```go // Simplified goroutine creation var wg sync.WaitGroup wg.Go(func() { /* work */ }) ``` - **Code Quality**: 30% reduction in boilerplate concurrency code 6. **Trace Flight Recorder API**: ```go import "runtime/trace" // Continuous tracing with in-memory ring buffer ``` - **Observability**: <1% performance overhead for continuous tracing - **Debugging**: Capture 1-hour execution traces in 50MB memory ### Go-Specific Libraries & Patterns **Essential Libraries**: 1. **Validation**: `github.com/go-playground/validator/v10` or Go 1.25 generics for type-safe validation 2. **Configuration Management**: `github.com/spf13/viper` 3. **Logging**: `github.com/rs/zerolog` (fast, structured) or `github.com/sirupsen/logrus` (feature-rich) 4. **HTTP Client**: Standard `net/http` (Go 1.25 improvements) or `github.com/go-resty/resty/v2` 5. **Database Migration**: `github.com/golang-migrate/migrate/v4` 6. **Testing**: `github.com/stretchr/testify`, `github.com/golang/mock` or `github.com/vektra/mockery/v2` 7. **WebSocket**: `github.com/gorilla/websocket`, `nhooyr.io/websocket`, or `github.com/gobwas/ws` 8. **GraphQL**: `github.com/99designs/gqlgen` (schema-first) or `github.com/graphql-go/graphql` (runtime-first) 9. **gRPC**: `google.golang.org/grpc` for microservices 10. **Task Queues**: `github.com/hibiken/asynq` (Redis-based distributed task queue) 11. **Observability**: `go.opentelemetry.io/otel`, `github.com/prometheus/client_golang` **Go-Specific Architecture Patterns**: 1. **Interface-Driven Design**: Accept interfaces, return structs 2. **Context Propagation**: Use `context.Context` for cancellation, timeouts, request-scoped values 3. **Error Handling**: Wrap errors with `fmt.Errorf("operation failed: %w", err)`, use `errors.Is()` and `errors.As()` 4. **Concurrency Patterns**: Channels for communication, `sync.WaitGroup` for coordination, worker pools for parallelism 5. **Graceful Shutdown**: Handle SIGTERM/SIGINT, drain connections, finish in-flight requests, cleanup resources ### Go Project Structure ``` /cmd # Application entrypoints /internal # Private application code /pkg # Public library code /api # API definitions /configs # Configuration files /scripts # Build/deployment scripts /docs # Documentation including ADRs ``` ---