turash/docs/concept/29_technical_architecture_diagrams.md
Damir Mukimov 000eab4740
Major repository reorganization and missing backend endpoints implementation
Repository Structure:
- Move files from cluttered root directory into organized structure
- Create archive/ for archived data and scraper results
- Create bugulma/ for the complete application (frontend + backend)
- Create data/ for sample datasets and reference materials
- Create docs/ for comprehensive documentation structure
- Create scripts/ for utility scripts and API tools

Backend Implementation:
- Implement 3 missing backend endpoints identified in gap analysis:
  * GET /api/v1/organizations/{id}/matching/direct - Direct symbiosis matches
  * GET /api/v1/users/me/organizations - User organizations
  * POST /api/v1/proposals/{id}/status - Update proposal status
- Add complete proposal domain model, repository, and service layers
- Create database migration for proposals table
- Fix CLI server command registration issue

API Documentation:
- Add comprehensive proposals.md API documentation
- Update README.md with Users and Proposals API sections
- Document all request/response formats, error codes, and business rules

Code Quality:
- Follow existing Go backend architecture patterns
- Add proper error handling and validation
- Match frontend expected response schemas
- Maintain clean separation of concerns (handler -> service -> repository)
2025-11-25 06:01:16 +01:00

606 lines
22 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# 29. Technical Architecture Diagrams
## System Context Diagram
```mermaid
graph TB
subgraph "External Systems"
ERP[ERP Systems<br/>SAP, Oracle]
SCADA[SCADA Systems<br/>OPC UA, Modbus]
IoT[IoT Devices<br/>Sensors, Meters]
UTIL[Utility APIs<br/>Energy, Water]
GOV[Government Data<br/>Building Permits, NACE]
end
subgraph "Turash Platform"
API[API Gateway<br/>Kong/Traefik]
AUTH[Authentication Service<br/>JWT, OAuth2]
subgraph "Application Layer"
WEB[Web Frontend<br/>Next.js + React]
MOBILE[Mobile PWA<br/>Progressive Web App]
ADMIN[Admin Dashboard<br/>Enterprise Features]
end
subgraph "Service Layer"
MATCH[Matching Engine<br/>Graph Algorithms]
ECON[Economic Calculator<br/>NPV, ROI Models]
NOTIF[Notification Service<br/>WebSocket, Email]
SEARCH[Search Service<br/>Full-text, Spatial]
end
subgraph "Data Layer"
NEO4J[(Neo4j Graph DB<br/>Relationships, Entities)]
POSTGRES[(PostgreSQL + PostGIS<br/>Spatial, Time-series)]
REDIS[(Redis Cache<br/>Sessions, Matches)]
end
subgraph "Infrastructure"
K8S[Kubernetes Cluster<br/>AWS EKS/GCP GKE]
MQ["Message Queue<br/>NATS/Redis Streams (MVP)<br/>Kafka (Scale)"]
CDN[CDN<br/>CloudFront/Cloudflare]
MONITOR[Monitoring Stack<br/>Prometheus + Grafana]
end
end
subgraph "Users & Partners"
SME[SME Businesses<br/>Resource Providers]
ENTERPRISE[Enterprise Companies<br/>Multi-site Operations]
MUNI[Municipalities<br/>City Governments]
UTIL_PART[Utility Partners<br/>Data & Distribution]
FACIL[Facilitators<br/>Engineers, Consultants]
end
%% Connections
ERP --> API
SCADA --> API
IoT --> API
UTIL --> API
GOV --> API
API --> AUTH
AUTH --> WEB
AUTH --> MOBILE
AUTH --> ADMIN
WEB --> MATCH
MOBILE --> MATCH
ADMIN --> MATCH
MATCH --> ECON
MATCH --> NOTIF
MATCH --> SEARCH
MATCH --> NEO4J
ECON --> POSTGRES
SEARCH --> REDIS
NEO4J --> K8S
POSTGRES --> K8S
REDIS --> K8S
NOTIF --> MQ
MQ --> MONITOR
SME --> WEB
ENTERPRISE --> ADMIN
MUNI --> API
UTIL_PART --> API
FACIL --> WEB
```
## Application Architecture (C4 Container Diagram)
```mermaid
graph TB
subgraph "User Interfaces"
WEB_BROWSER[Web Browser<br/>React + Next.js]
MOBILE_APP[Mobile PWA<br/>React + Service Worker]
ADMIN_PORTAL[Admin Portal<br/>Enterprise Dashboard]
end
subgraph "API Gateway"
KONG["API Gateway<br/>Kong/Traefik<br/>Rate Limiting, Auth"]
end
subgraph "Microservices"
AUTH_SERVICE["Authentication Service<br/>Go 1.25 + JWT<br/>User Management"]
BUSINESS_SERVICE["Business Service<br/>Go 1.25 + Neo4j<br/>CRUD Operations"]
MATCHING_SERVICE["Matching Service<br/>Go 1.25 + Graph Algorithms<br/>Core Engine + Hybrid Automation"]
ECONOMIC_SERVICE["Economic Service<br/>Go 1.25 + PostgreSQL<br/>Calculations"]
NOTIFICATION_SERVICE["Notification Service<br/>Go 1.25 + Redis<br/>WebSocket/Email"]
INGESTION_SERVICE["Ingestion Service<br/>Go 1.25 + NATS/Redis Streams<br/>Data Processing"]
SEARCH_SERVICE["Search Service<br/>Go 1.25 + Elasticsearch<br/>Full-text Search"]
ANALYTICS_SERVICE["Analytics Service<br/>Go 1.25 + TimescaleDB<br/>Reporting"]
DATA_QUALITY_SERVICE["Data Quality Service<br/>Go 1.25 + PostgreSQL<br/>Scoring & Incentives"]
FACILITATOR_SERVICE["Facilitator Service<br/>Go 1.25 + Neo4j<br/>Marketplace & Routing"]
end
subgraph "Data Stores"
NEO4J["(Neo4j Cluster<br/>Graph Database<br/>Relationships)"]
POSTGRES["(PostgreSQL Cluster<br/>Spatial Database<br/>Business Data)"]
REDIS["(Redis Cluster<br/>Cache + Sessions<br/>Performance)"]
NATS["(NATS/Redis Streams<br/>Event Streaming (MVP)<br/>Async Processing)"]
KAFKA["(Kafka Cluster<br/>Event Streaming (Scale)<br/>Migration at 1000+ businesses)"]
ELASTIC["(Elasticsearch<br/>Search Index<br/>Full-text)"]
MINIO["(MinIO/S3<br/>Object Storage<br/>Files & Assets)"]
end
subgraph "Infrastructure Services"
PROMETHEUS[Prometheus<br/>Metrics Collection]
GRAFANA[Grafana<br/>Visualization]
ALERTMANAGER[AlertManager<br/>Incident Management]
LOGGING[Loki + ELK<br/>Log Aggregation]
VAULT[HashiCorp Vault<br/>Secrets Management]
end
%% Internal connections
WEB_BROWSER --> KONG
MOBILE_APP --> KONG
ADMIN_PORTAL --> KONG
KONG --> AUTH_SERVICE
AUTH_SERVICE --> BUSINESS_SERVICE
BUSINESS_SERVICE --> MATCHING_SERVICE
BUSINESS_SERVICE --> DATA_QUALITY_SERVICE
MATCHING_SERVICE --> ECONOMIC_SERVICE
MATCHING_SERVICE --> NOTIFICATION_SERVICE
MATCHING_SERVICE --> FACILITATOR_SERVICE
BUSINESS_SERVICE --> INGESTION_SERVICE
BUSINESS_SERVICE --> SEARCH_SERVICE
BUSINESS_SERVICE --> ANALYTICS_SERVICE
BUSINESS_SERVICE --> NEO4J
MATCHING_SERVICE --> NEO4J
FACILITATOR_SERVICE --> NEO4J
ECONOMIC_SERVICE --> POSTGRES
DATA_QUALITY_SERVICE --> POSTGRES
NOTIFICATION_SERVICE --> REDIS
INGESTION_SERVICE --> NATS
SEARCH_SERVICE --> ELASTIC
ANALYTICS_SERVICE --> MINIO
MATCHING_SERVICE --> PROMETHEUS
PROMETHEUS --> GRAFANA
PROMETHEUS --> ALERTMANAGER
ALL_SERVICES --> LOGGING
ALL_SERVICES --> VAULT
```
## Data Architecture Diagram
```mermaid
graph TD
subgraph "Data Sources"
ERP["ERP Systems<br/>SAP, Oracle<br/>Transactional Data"]
SCADA["SCADA Systems<br/>OPC UA, Modbus<br/>Real-time Process Data"]
IOT_SENSORS["IoT Sensors<br/>Temperature, Flow, Pressure<br/>Time-series Data"]
MANUAL["Manual Entry<br/>Web Forms, CSV Upload<br/>User-Provided Data"]
PUBLIC_DATA["Public Datasets<br/>Government, Utilities<br/>Reference Data"]
end
subgraph "Ingestion Layer"
API_INGEST["API Ingestion<br/>REST/WebSocket<br/>Real-time"]
BATCH_INGEST["Batch Ingestion<br/>CSV, XML, JSON<br/>Scheduled"]
STREAM_INGEST["Stream Processing<br/>NATS/Redis Streams/MQTT<br/>Event-driven"]
WEBHOOKS["Webhook Ingestion<br/>ERP/SCADA Push<br/>Event-triggered"]
end
subgraph "Processing Layer"
VALIDATION["Data Validation<br/>Schema, Business Rules<br/>Quality Gates"]
NORMALIZATION["Data Normalization<br/>Unit Conversion, Standardization<br/>Data Harmonization"]
ENRICHMENT["Data Enrichment<br/>Geocoding, Classification<br/>Context Addition"]
QUALITY_SCORING["Data Quality Scoring<br/>Precision Levels: Rough/Estimated/Measured<br/>Trust Scoring & Incentives"]
DEDUPLICATION["Deduplication<br/>Entity Resolution<br/>Data Quality"]
end
subgraph "Storage Layer"
subgraph "Graph Database (Neo4j)"
BUSINESS_NODES["Business Nodes<br/>Companies, Legal Entities<br/>Properties: name, industry, size"]
SITE_NODES["Site Nodes<br/>Physical Locations<br/>Properties: address, coordinates, type"]
RESOURCE_NODES["Resource Flow Nodes<br/>Energy, Water, Waste<br/>Properties: type, amount, quality"]
SERVICE_NODES["Service Nodes<br/>Maintenance, Consulting<br/>Properties: domain, capacity, rate"]
end
subgraph "Relational Database (PostgreSQL)"
BUSINESS_TABLES["Business Tables<br/>Normalized Business Data<br/>Indexes: name, industry, location"]
SITE_TABLES["Site Tables<br/>Spatial Site Data<br/>PostGIS: coordinates, boundaries"]
RESOURCE_TABLES["Resource Tables<br/>Time-series Flows<br/>Partitioned by time, indexed by location"]
TRANSACTION_TABLES["Transaction Tables<br/>Match Records, Contracts<br/>Audit trail, compliance"]
end
subgraph "Cache Layer (Redis)"
MATCH_CACHE["Match Results Cache<br/>TTL: 15 minutes<br/>Key: location + resource_type"]
SESSION_CACHE["Session Cache<br/>JWT tokens, user state<br/>TTL: 24 hours"]
COMPUTATION_CACHE["Computation Cache<br/>Economic calculations<br/>TTL: 1 hour"]
end
end
subgraph "Access Layer"
GRAPH_API["Graph API<br/>Cypher Queries<br/>Relationship Traversal"]
REST_API["REST API<br/>CRUD Operations<br/>Standard HTTP"]
GRAPHQL_API["GraphQL API<br/>Flexible Queries<br/>Client-driven"]
WEBSOCKET_API["WebSocket API<br/>Real-time Updates<br/>Live Notifications"]
BULK_API["Bulk API<br/>Data Export/Import<br/>Batch Operations"]
end
subgraph "Consumption Layer"
WEB_APP["Web Application<br/>React + Mapbox<br/>User Interface"]
MOBILE_APP["Mobile PWA<br/>Progressive Web App<br/>Field Operations"]
API_CLIENTS["API Clients<br/>ERP Integration<br/>Third-party Systems"]
ANALYTICS["Analytics Platform<br/>Business Intelligence<br/>Reporting & Insights"]
ML_MODELS["ML Models<br/>Recommendation Engine<br/>Predictive Analytics"]
end
%% Data Flow
ERP --> API_INGEST
SCADA --> STREAM_INGEST
IOT_SENSORS --> STREAM_INGEST
MANUAL --> BATCH_INGEST
PUBLIC_DATA --> WEBHOOKS
API_INGEST --> VALIDATION
BATCH_INGEST --> VALIDATION
STREAM_INGEST --> VALIDATION
WEBHOOKS --> VALIDATION
VALIDATION --> NORMALIZATION
NORMALIZATION --> ENRICHMENT
ENRICHMENT --> QUALITY_SCORING
QUALITY_SCORING --> DEDUPLICATION
DEDUPLICATION --> BUSINESS_NODES
DEDUPLICATION --> SITE_NODES
DEDUPLICATION --> RESOURCE_NODES
DEDUPLICATION --> SERVICE_NODES
DEDUPLICATION --> BUSINESS_TABLES
DEDUPLICATION --> SITE_TABLES
DEDUPLICATION --> RESOURCE_TABLES
DEDUPLICATION --> TRANSACTION_TABLES
BUSINESS_NODES --> GRAPH_API
SITE_NODES --> GRAPH_API
RESOURCE_NODES --> GRAPH_API
SERVICE_NODES --> GRAPH_API
BUSINESS_TABLES --> REST_API
SITE_TABLES --> REST_API
RESOURCE_TABLES --> REST_API
TRANSACTION_TABLES --> REST_API
GRAPH_API --> GRAPHQL_API
REST_API --> GRAPHQL_API
GRAPHQL_API --> WEBSOCKET_API
GRAPHQL_API --> BULK_API
WEBSOCKET_API --> WEB_APP
WEBSOCKET_API --> MOBILE_APP
GRAPHQL_API --> API_CLIENTS
BULK_API --> ANALYTICS
REST_API --> ML_MODELS
%% Cache connections
MATCH_CACHE -.-> GRAPHQL_API
SESSION_CACHE -.-> WEBSOCKET_API
COMPUTATION_CACHE -.-> REST_API
```
## Matching Engine Architecture
```mermaid
graph TD
subgraph "Input Processing"
QUERY_IN[Resource Query<br/>Location, Type, Preferences]
CONTEXT_IN[Business Context<br/>Industry, Size, Capabilities]
CONSTRAINTS_IN[Matching Constraints<br/>Distance, Quality, Cost]
end
subgraph "Pre-filtering Stage"
SPATIAL_FILTER["Spatial Filter<br/>PostGIS Query<br/>Radius Search"]
BASIC_FILTER["Basic Filter<br/>Resource Type Match<br/>Availability Windows"]
QUALITY_FILTER["Quality Filter<br/>Grade/Purity Match<br/>Hazard Compatibility"]
end
subgraph "Candidate Selection"
GEO_INDEX["Geospatial Index<br/>R-tree/B-tree<br/>Location-based"]
RESOURCE_INDEX["Resource Index<br/>Composite Index<br/>Type + Direction + Location"]
TEMPORAL_INDEX["Temporal Index<br/>Time-window Index<br/>Availability Overlap"]
end
subgraph "Compatibility Scoring"
QUALITY_SCORER["Quality Scorer<br/>Temperature, Pressure, Purity<br/>Weighted Compatibility"]
TEMPORAL_SCORER["Temporal Scorer<br/>Availability Overlap<br/>Seasonal Factors"]
QUANTITY_SCORER["Quantity Scorer<br/>Supply/Demand Match<br/>Variability Adjustment"]
ECONOMIC_SCORER["Economic Scorer<br/>Cost Arbitrage<br/>Transport Costs"]
TRUST_SCORER["Trust Scorer<br/>Data Precision<br/>Source Validation"]
end
subgraph "Optimization Engine"
WEIGHTED_SUM["Weighted Sum<br/>Multi-criteria Scoring<br/>Configurable Weights"]
RANKING_ENGINE["Ranking Engine<br/>Top-N Selection<br/>Diversity Consideration"]
HYBRID_ROUTER["Hybrid Automation Router<br/>80% Automated, 20% Facilitated<br/>Match Complexity Scoring"]
CONSTRAINT_CHECKER["Constraint Checker<br/>Business Rules<br/>Regulatory Compliance"]
FALLBACK_ENGINE["Fallback Engine<br/>Relaxation Strategies<br/>Alternative Matches"]
end
subgraph "Result Processing"
ECONOMIC_CALCULATION["Economic Calculation<br/>NPV, Payback, ROI<br/>Detailed Projections"]
RISK_ASSESSMENT["Risk Assessment<br/>Technical, Market, Regulatory<br/>Probability Scoring"]
PARTNER_PACKETS["Partner Packet Generation<br/>Technical Specs, Contacts<br/>Legal Templates"]
NOTIFICATION_ENGINE["Notification Engine<br/>WebSocket Push<br/>Email/SMS Alerts"]
end
subgraph "Caching & Persistence"
RESULT_CACHE["Result Cache<br/>Redis TTL Cache<br/>15-minute expiry"]
MATCH_STORAGE["Match Storage<br/>PostgreSQL<br/>Audit Trail"]
ANALYTICS_STORAGE["Analytics Storage<br/>ClickHouse/Timescale<br/>Performance Metrics"]
end
%% Data flow
QUERY_IN --> SPATIAL_FILTER
CONTEXT_IN --> BASIC_FILTER
CONSTRAINTS_IN --> QUALITY_FILTER
SPATIAL_FILTER --> GEO_INDEX
BASIC_FILTER --> RESOURCE_INDEX
QUALITY_FILTER --> TEMPORAL_INDEX
GEO_INDEX --> QUALITY_SCORER
RESOURCE_INDEX --> TEMPORAL_SCORER
TEMPORAL_INDEX --> QUANTITY_SCORER
QUALITY_SCORER --> WEIGHTED_SUM
TEMPORAL_SCORER --> WEIGHTED_SUM
QUANTITY_SCORER --> WEIGHTED_SUM
ECONOMIC_SCORER --> WEIGHTED_SUM
TRUST_SCORER --> WEIGHTED_SUM
WEIGHTED_SUM --> RANKING_ENGINE
RANKING_ENGINE --> HYBRID_ROUTER
HYBRID_ROUTER --> CONSTRAINT_CHECKER
CONSTRAINT_CHECKER --> FALLBACK_ENGINE
FALLBACK_ENGINE --> ECONOMIC_CALCULATION
ECONOMIC_CALCULATION --> RISK_ASSESSMENT
RISK_ASSESSMENT --> PARTNER_PACKETS
PARTNER_PACKETS --> NOTIFICATION_ENGINE
NOTIFICATION_ENGINE --> RESULT_CACHE
RESULT_CACHE --> MATCH_STORAGE
MATCH_STORAGE --> ANALYTICS_STORAGE
```
## Deployment Architecture (AWS)
```mermaid
graph TB
subgraph "Internet"
INTERNET[Global Users<br/>Web, Mobile, API]
end
subgraph "AWS CloudFront (CDN)"
CLOUDFRONT["CloudFront Distribution<br/>Global Edge Locations<br/>SSL Termination"]
end
subgraph "AWS Global Accelerator"
ACCELERATOR["Global Accelerator<br/>TCP/UDP Optimization<br/>Failover Routing"]
end
subgraph "AWS WAF & Shield"
WAF["Web Application Firewall<br/>OWASP Rules<br/>Rate Limiting"]
SHIELD["Shield Advanced<br/>DDoS Protection<br/>Auto Mitigation"]
end
subgraph "AWS Route 53"
ROUTE53["Route 53<br/>DNS Management<br/>Health Checks<br/>Failover"]
end
subgraph "AWS Application Load Balancer"
ALB["Application Load Balancer<br/>Layer 7 Routing<br/>SSL Offloading<br/>Health Checks"]
end
subgraph "AWS Elastic Kubernetes Service (EKS)"
EKS_CONTROL["Control Plane<br/>Managed Kubernetes<br/>Auto-scaling"]
subgraph "Worker Nodes (Auto-scaling Group)"
ON_DEMAND["On-Demand Nodes<br/>General Workloads<br/>t3.large × 3-10"]
SPOT["Spot Nodes<br/>Batch Processing<br/>c6i.xlarge × 2-20"]
end
subgraph "Kubernetes Services"
INGRESS["Ingress Controller<br/>Traefik/Nginx<br/>External Traffic"]
API_PODS["API Service Pods<br/>Go Applications<br/>Horizontal Scaling"]
WORKER_PODS["Worker Pods<br/>Matching Engine<br/>CPU Optimized"]
WEB_PODS["Web Pods<br/>Next.js SSR<br/>Memory Optimized"]
end
end
subgraph "AWS RDS (PostgreSQL)"
RDS_CLUSTER["RDS Aurora Cluster<br/>Multi-AZ<br/>PostGIS Enabled<br/>Read Replicas"]
RDS_PROXY["RDS Proxy<br/>Connection Pooling<br/>Failover Handling"]
end
subgraph "AWS ElastiCache (Redis)"
REDIS_CLUSTER["Redis Cluster<br/>Multi-AZ<br/>Backup Enabled<br/>Encryption"]
end
subgraph "AWS Neptune (Neo4j-compatible)"
NEPTUNE_CLUSTER["Neptune Cluster<br/>Graph Database<br/>Gremlin/Cypher<br/>Multi-AZ"]
end
subgraph "AWS MSK (Kafka)"
KAFKA_CLUSTER["MSK Cluster<br/>Event Streaming (Scale Phase)<br/>Multi-AZ, Migration at 1000+ businesses<br/>Auto-scaling"]
end
subgraph "Message Queue (MVP)"
NATS_CLUSTER["NATS/Redis Streams<br/>Event Streaming (MVP)<br/>Simple, Cost-effective<br/>Kubernetes-native"]
end
subgraph "AWS OpenSearch"
OPENSEARCH["OpenSearch Domain<br/>Full-text Search<br/>Multi-AZ<br/>KNN for ML"]
end
subgraph "AWS S3 & Glacier"
S3_BUCKETS["S3 Buckets<br/>Data Lake<br/>Backup Storage<br/>CDN Origin"]
GLACIER["Glacier Deep Archive<br/>Long-term Backup<br/>Compliance Storage"]
end
subgraph "AWS CloudWatch & X-Ray"
CLOUDWATCH["CloudWatch<br/>Metrics & Logs<br/>Alarms & Dashboards"]
XRAY["X-Ray<br/>Distributed Tracing<br/>Performance Insights"]
end
subgraph "AWS Systems Manager"
SSM["Parameter Store<br/>Configuration Management<br/>Secrets (non-sensitive)"]
end
subgraph "AWS Secrets Manager"
SECRETS_MANAGER["Secrets Manager<br/>Database Passwords<br/>API Keys<br/>Auto-rotation"]
end
subgraph "AWS Backup"
BACKUP_SERVICE["Backup Service<br/>Automated Backups<br/>Cross-region Replication<br/>Retention Policies"]
end
%% Traffic flow
INTERNET --> CLOUDFRONT
CLOUDFRONT --> ACCELERATOR
ACCELERATOR --> WAF
WAF --> SHIELD
SHIELD --> ROUTE53
ROUTE53 --> ALB
ALB --> INGRESS
INGRESS --> API_PODS
INGRESS --> WEB_PODS
API_PODS --> WORKER_PODS
API_PODS --> RDS_PROXY
RDS_PROXY --> RDS_CLUSTER
API_PODS --> REDIS_CLUSTER
WORKER_PODS --> REDIS_CLUSTER
API_PODS --> NEPTUNE_CLUSTER
WORKER_PODS --> NEPTUNE_CLUSTER
API_PODS --> NATS_CLUSTER
WORKER_PODS --> NATS_CLUSTER
API_PODS --> KAFKA_CLUSTER
WORKER_PODS --> KAFKA_CLUSTER
API_PODS --> OPENSEARCH
ALL_SERVICES --> S3_BUCKETS
S3_BUCKETS --> GLACIER
ALL_SERVICES --> CLOUDWATCH
ALL_SERVICES --> XRAY
ALL_SERVICES --> SSM
ALL_SERVICES --> SECRETS_MANAGER
RDS_CLUSTER --> BACKUP_SERVICE
NEPTUNE_CLUSTER --> BACKUP_SERVICE
REDIS_CLUSTER --> BACKUP_SERVICE
```
## CI/CD Pipeline Architecture
```mermaid
graph LR
subgraph "Developer Workstations"
LOCAL_DEV["Local Development<br/>Docker Compose<br/>Kind Cluster"]
end
subgraph "GitHub Repository"
MAIN_BRANCH[main Branch<br/>Production Code]
FEATURE_BRANCHES[feature/* Branches<br/>Development]
RELEASE_BRANCHES[release/* Branches<br/>Staging]
end
subgraph "GitHub Actions"
LINT_JOB["Lint & Format<br/>golangci-lint<br/>ESLint"]
TEST_JOB["Unit & Integration Tests<br/>Go 1.25 test (with feature flags)<br/>Jest, Coverage Reports"]
BUILD_JOB["Build & Containerize<br/>Go 1.25 build (fallback to 1.23)<br/>Docker Build, Multi-arch Images"]
SECURITY_JOB["Security Scan<br/>Trivy, Snyk<br/>Vulnerability Check"]
DEPLOY_DEV_JOB["Deploy to Development<br/>Kind Cluster<br/>Integration Tests"]
DEPLOY_STAGING_JOB["Deploy to Staging<br/>EKS Staging<br/>E2E Tests"]
DEPLOY_PROD_JOB["Deploy to Production<br/>EKS Production<br/>Blue-Green Deployment"]
end
subgraph "Development Environment"
DEV_CLUSTER["Kind Cluster<br/>Local Kubernetes<br/>Full Stack"]
DEV_MONITORING["Development Monitoring<br/>Prometheus + Grafana<br/>Local Stack"]
end
subgraph "Staging Environment"
STAGING_EKS["EKS Staging Cluster<br/>Full Infrastructure<br/>Production-like"]
STAGING_DB["Staging Databases<br/>Neo4j, PostgreSQL, Redis<br/>Test Data"]
STAGING_MONITORING["Staging Monitoring<br/>CloudWatch, X-Ray<br/>Alert Testing"]
end
subgraph "Production Environment"
PROD_EKS["EKS Production Cluster<br/>High Availability<br/>Auto-scaling"]
PROD_DB["Production Databases<br/>Multi-AZ, Backups<br/>Real Data"]
PROD_CDN["CloudFront CDN<br/>Global Distribution<br/>Edge Locations"]
PROD_MONITORING["Production Monitoring<br/>24/7 Alerting<br/>Incident Response"]
end
subgraph "Quality Gates"
CODE_REVIEW["Code Review<br/>Pull Request Review<br/>Architecture Review"]
SECURITY_REVIEW["Security Review<br/>Dependency Check<br/>Secrets Scanning"]
PERFORMANCE_TEST["Performance Test<br/>Load Testing<br/>Scalability Check"]
COMPLIANCE_CHECK["Compliance Check<br/>GDPR, SOC2<br/>Audit Logging"]
end
subgraph "Artifact Repository"
GHCR["GitHub Container Registry<br/>Docker Images<br/>Vulnerability Scanning"]
ARTIFACTS["Test Artifacts<br/>Coverage Reports<br/>Performance Metrics"]
end
%% Development flow
LOCAL_DEV --> FEATURE_BRANCHES
FEATURE_BRANCHES --> CODE_REVIEW
CODE_REVIEW --> MAIN_BRANCH
MAIN_BRANCH --> LINT_JOB
LINT_JOB --> TEST_JOB
TEST_JOB --> BUILD_JOB
BUILD_JOB --> SECURITY_JOB
SECURITY_JOB --> DEPLOY_DEV_JOB
DEPLOY_DEV_JOB --> DEV_CLUSTER
DEV_CLUSTER --> SECURITY_REVIEW
SECURITY_REVIEW --> RELEASE_BRANCHES
RELEASE_BRANCHES --> PERFORMANCE_TEST
PERFORMANCE_TEST --> DEPLOY_STAGING_JOB
DEPLOY_STAGING_JOB --> STAGING_EKS
STAGING_EKS --> COMPLIANCE_CHECK
COMPLIANCE_CHECK --> DEPLOY_PROD_JOB
DEPLOY_PROD_JOB --> PROD_EKS
%% Artifact flow
BUILD_JOB --> GHCR
TEST_JOB --> ARTIFACTS
SECURITY_JOB --> ARTIFACTS
%% Monitoring flow
DEV_CLUSTER --> DEV_MONITORING
STAGING_EKS --> STAGING_MONITORING
PROD_EKS --> PROD_MONITORING
%% Database flow
DEPLOY_DEV_JOB -.-> DEV_CLUSTER
DEPLOY_STAGING_JOB --> STAGING_DB
DEPLOY_PROD_JOB --> PROD_DB
PROD_EKS --> PROD_CDN
```