turash/mvp_concept.md
Damir Mukimov 0161394b9b
Initial commit: Turash Industrial Symbiosis Platform
- Complete platform specification and architecture
- MVP concept and business model
- Technical implementation plans and roadmaps
- Financial projections and funding strategy
- Brand identity and marketing materials
- Mathematical models for resource matching
- Competitive analysis and market research
- Go-based core models and algorithms

Turash guides businesses to optimal resource exchanges, navigating the complex landscape of industrial symbiosis and providing direction to sustainable profitability.
2025-11-25 05:34:50 +01:00

38 KiB

Sellable MVP Concept: Turash

A lean, marketable MVP approach to industrial resource matching platform.


MVP Vision: "Resource Dating App for Businesses"

The Pitch: A simple platform where local businesses declare what they consume and emit, and we find profitable matches. Start with heat (tangible, measurable), expand later.


Core MVP Value Proposition

Problem

  • Businesses waste money on utilities they could share
  • Neighbors don't know each other's resource needs
  • No simple way to find "I have X, you need X" opportunities

Solution (MVP)

  1. Simple Declaration: Business fills 5-minute form
  2. Automatic Matching: System finds compatible neighbors
  3. Clear ROI: "Connect to Business Y → Save €18k/year"

Why It Works

  • High Value: Rare but valuable matches (1-3 per year, big savings)
  • Low Friction: Simple data entry, clear results
  • Network Effects: More businesses = better matches

MVP Scope: What to Build

In Scope (Must Have)

  1. Business Registration

    • Company name, location (address), contact
    • Simple resource declaration:
      • "I consume: heat @ X°C, Y kWh/month"
      • "I emit: heat @ X°C, Y kWh/month"
      • Or: "I need cold storage for X m³"
  2. Basic Matching Engine

    • Distance-based filtering (within 5km)
    • Temperature compatibility (±10°C tolerance)
    • Simple economic scoring (estimated savings)
  3. Match Results

    • Ranked list: "You could save €X/year with Business Y"
    • Distance, estimated savings, basic compatibility
  4. Contact Introduction

    • Platform facilitates initial contact
    • Simple messaging or email intro
  5. Map View

    • Show businesses on map
    • Visualize potential connections

Out of Scope (Later)

  • Complex multi-party matching
  • Full economic optimization (MILP)
  • Real-time IoT data
  • Advanced analytics
  • Mobile apps
  • Enterprise features
  • Multiple resource types (start with heat only)

MVP Tech Stack (Horizontal Foundation)

Note

: This MVP concept now includes production-ready foundations addressing all critical gaps: authentication, authorization, input validation, structured logging (slog), error handling, connection management, graceful shutdown, rate limiting, health checks, and security headers.

Backend (Go 1.25)

Core Stack - Full Foundation:

  • HTTP Framework: Echo (clean API, built-in validation, mature)
  • Graph Database: Neo4j (foundation for graph queries from day 1)
    • Simple graph structure (Business → Site → ResourceFlow)
    • Basic Cypher queries (no complex traversals yet)
    • Single instance (community edition free)
    • Connection pooling configured
    • Indexes: ResourceFlow.type + direction, Site.id
  • Geospatial Database: PostgreSQL + PostGIS with GORM
    • GORM for simpler CRUD operations
    • PostGIS spatial queries (distance calculations)
    • Site locations with spatial indexes (GIST on location)
    • Event-driven sync from Neo4j
    • Connection pooling via GORM
  • Event Processing: Watermill (Go-native event processing library)
    • Start with in-memory Go channels (zero external deps)
    • Clean pubsub interface for easy backend switching
    • Can migrate to Redis Streams, Kafka, NATS later (same code)
    • Foundation for event-driven architecture
    • Production-ready event processing patterns
  • Cache: Redis (match results, sessions)
    • Connection pooling configured
    • Session storage for auth
  • Authentication: JWT-based auth
    • Simple JWT tokens (no OAuth2 provider needed yet)
    • Echo JWT middleware
    • Business registration creates account + JWT
  • Logging: Go's log/slog (structured logging)
    • JSON format for production
    • Text format for development
    • Log levels: DEBUG, INFO, WARN, ERROR
    • Request logging middleware
  • Validation: github.com/go-playground/validator/v10
    • Struct validation on all API endpoints
    • User-friendly error messages
  • Configuration: Environment variables
    • .env file for local development
    • Environment-specific config (dev, prod)
  • Hosting: Managed services (Railway/Render) or single server

Why Full Foundation Now:

  • Build right architecture from day 1
  • Each component simple but extensible
  • No expensive migrations later
  • Horizontal MVP: all layers present, keep each simple
  • Production-ready from launch

Frontend

  • React + Vite: Fast development
  • Mapbox GL JS: Map visualization (free tier: 50k loads/month)
  • Tailwind CSS: Rapid UI development
  • shadcn/ui: Pre-built components

Deploy: Vercel or Netlify (free tier)

Infrastructure (Complete Foundation, Simple Setup)

  • Application Host: Railway, Render, or DigitalOcean ($10-20/month)
    • Single Go binary serving API
  • Neo4j: Neo4j Aura Free tier (50k nodes) or self-hosted single instance
    • Community edition if self-hosting (free)
  • PostgreSQL + PostGIS: Railway, Supabase free tier, or Neon
  • Event Processing: Watermill with in-memory pubsub (built-in, zero cost)
    • Later: Switch to Redis Streams, Kafka, or NATS (same interface)
  • Redis: Redis Cloud free tier or Railway addon
  • Domain: $10/year

Total Cost: ~$30-70/month for MVP (foundation stack)


MVP Data Model (Horizontal Foundation)

Neo4j Graph Structure (Foundation)

// Nodes
(:Business {
  id: UUID,
  name: String,
  contact_email: String,
  contact_phone: String
})

(:Site {
  id: UUID,
  address: String,
  latitude: Float,
  longitude: Float
})

(:ResourceFlow {
  id: UUID,
  direction: "input" | "output",
  type: "heat", // MVP: only heat
  temperature_celsius: Float,
  quantity_kwh_per_month: Float,
  cost_per_kwh_euro: Float
})

// Relationships
(Business)-[:OPERATES_AT]->(Site)
(Site)-[:HOSTS]->(ResourceFlow)
(ResourceFlow)-[:MATCHABLE_TO {
  distance_km: Float,
  savings_euro_per_year: Float,
  score: Float
}]->(ResourceFlow)

Why Neo4j from Day 1:

  • Right foundation for graph queries
  • Simple structure now, add complexity later
  • No expensive migration
  • Natural fit for matching relationships

PostgreSQL + PostGIS with GORM (Geospatial)

// GORM model for PostGIS
type SiteGeo struct {
    SiteID     uuid.UUID `gorm:"type:uuid;primary_key"`
    BusinessID uuid.UUID `gorm:"type:uuid;not null;index"`
    Latitude   float64   `gorm:"not null"`
    Longitude  float64   `gorm:"not null"`
    Location   postgis.Geometry `gorm:"type:geometry(Point,4326);not null;index:idx_location"`
    UpdatedAt  time.Time `gorm:"autoUpdateTime"`
}

// GORM auto-migration
db.AutoMigrate(&SiteGeo{})

// Create spatial index
db.Exec("CREATE INDEX IF NOT EXISTS idx_sites_location ON sites_geo USING GIST(location)")

Why GORM + PostGIS:

  • GORM simplifies CRUD operations
  • PostGIS handles fast spatial queries (distance calculations)
  • Spatial indexes optimized for radius searches
  • Neo4j handles relationships, PostGIS handles geography
  • Event-driven sync keeps data in sync

Data Flow (Simple but Complete - Go Native)

User Input → API (Echo) → Auth Check → Validation → Watermill → Goroutine Worker
                                    ↓                                      ↓
                              JWT Verify                           Neo4j Write
                                    ↓                                      ↓
                              Rate Limit                          PostGIS Sync (async goroutine)
                                                                    ↓
                                                          Match Computation (goroutine)
                                                                    ↓
                                                          Redis Cache (matches)

Security & Authentication

Authentication:

  • JWT-based authentication (simple, no OAuth2 provider needed yet)
  • Business registration creates account + JWT token
  • All API endpoints require authentication (except /health and public match summaries)
  • Echo JWT middleware for token validation
  • Token stored in HTTP-only cookie or Authorization header

Authorization:

  • Role-Based Access Control (RBAC):
    • Business Owner: Can view/edit own business data, view own matches
    • Public/Viewer: Can view public match summaries (without sensitive data)
  • Ownership validation: Business can only access own data
  • API endpoints check ownership before allowing operations

API Security:

  • Rate Limiting: 100 requests/minute per IP
  • CORS: Configured for frontend domain only
  • Request Size Limits: 10MB max body size
  • HTTPS: Enforced in production
  • Security Headers: X-Content-Type-Options, X-Frame-Options, X-XSS-Protection

Implementation Example:

// JWT middleware
func JWTAuthMiddleware() echo.MiddlewareFunc {
    return middleware.JWTWithConfig(middleware.JWTConfig{
        SigningKey:  []byte(secretKey),
        TokenLookup: "header:Authorization",
        AuthScheme:  "Bearer",
    })
}

// Ownership check middleware
func RequireOwnership(businessID uuid.UUID) echo.MiddlewareFunc {
    return func(next echo.HandlerFunc) echo.HandlerFunc {
        return func(c echo.Context) error {
            userID := getUserIDFromToken(c)
            if userID != businessID {
                return echo.ErrForbidden
            }
            return next(c)
        }
    }
}

Error Handling & Validation

Error Handling:

  • Standardized Error Response Format:
type ErrorResponse struct {
    Error   string `json:"error"`
    Code    string `json:"code"`
    Message string `json:"message"`
    Details interface{} `json:"details,omitempty"`
}
  • HTTP Status Codes:
    • 400 Bad Request (validation errors)
    • 401 Unauthorized (missing/invalid token)
    • 403 Forbidden (no permission)
    • 404 Not Found
    • 500 Internal Server Error
  • Error Logging: All errors logged with context using slog
  • Error Wrapping: fmt.Errorf("operation failed: %w", err) for error context

Input Validation:

  • Struct Validation: github.com/go-playground/validator/v10
  • Validation Rules:
    • Temperature: -50°C to 500°C for heat
    • Quantity: Must be > 0
    • Email: Valid email format
    • Location: Must include latitude/longitude
  • User-Friendly Error Messages: Clear validation error messages
  • Cross-Entity Validation: Site belongs to Business, ResourceFlow belongs to Site

Implementation Example:

type ResourceFlow struct {
    TemperatureCelsius float64 `json:"temperature_celsius" validate:"required,min=-50,max=500"`
    QuantityKwhPerMonth float64 `json:"quantity_kwh_per_month" validate:"required,gt=0"`
    Direction string `json:"direction" validate:"required,oneof=input output"`
}

func createResourceFlow(c echo.Context) error {
    var rf ResourceFlow
    if err := c.Bind(&rf); err != nil {
        return echo.NewHTTPError(400, "invalid request body")
    }

    if err := validate.Struct(rf); err != nil {
        return echo.NewHTTPError(400, err.Error())
    }

    // ... create logic ...
}

Logging & Observability (slog)

Structured Logging with log/slog:

  • JSON Format: Production (machine-readable)
  • Text Format: Development (human-readable)
  • Log Levels: DEBUG, INFO, WARN, ERROR
  • Context Fields: Request ID, user ID, business ID, duration
  • Request Logging Middleware: Log all HTTP requests (method, path, status, duration)

Implementation Example:

import (
    "log/slog"
    "os"
)

// Initialize logger based on environment
var logger *slog.Logger

func initLogger(env string) {
    var handler slog.Handler

    if env == "production" {
        handler = slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
            Level: slog.LevelInfo,
        })
    } else {
        handler = slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{
            Level: slog.LevelDebug,
        })
    }

    logger = slog.New(handler)
}

// Request logging middleware
func RequestLogger() echo.MiddlewareFunc {
    return middleware.RequestLoggerWithConfig(middleware.RequestLoggerConfig{
        LogStatus:   true,
        LogURI:      true,
        LogMethod:   true,
        LogError:    true,
        LogDuration: true,
        LogValuesFunc: func(c echo.Context, v middleware.RequestLoggerValues) error {
            logger.Info("request",
                "method", v.Method,
                "uri", v.URI,
                "status", v.Status,
                "duration", v.Duration,
                "error", v.Error,
            )
            return nil
        },
    })
}

// Error logging
func handleError(c echo.Context, err error) error {
    logger.Error("api error",
        "error", err,
        "path", c.Path(),
        "method", c.Request().Method,
    )

    return c.JSON(500, ErrorResponse{
        Error: "internal_server_error",
        Message: "An error occurred",
    })
}

Metrics (Basic):

  • Request count per endpoint
  • Error rate
  • Response time (p50, p95, p99)
  • Database connection pool usage
  • Cache hit rate

Database Connection Management

Connection Pooling:

  • Neo4j: Configure max connections (default: 100), connection timeout (30s)
  • PostgreSQL (GORM): Configure max open connections (25), max idle (5), connection lifetime
  • Redis: Configure pool size (10 connections), connection timeout (5s)

Health Checks:

  • /health endpoint checks database connections
  • Startup health check: Verify all DB connections before accepting requests
  • Periodic health checks: Monitor connection health

Implementation Example:

// Health check endpoint
func healthCheck(c echo.Context) error {
    // Check Neo4j
    if err := neo4jDriver.VerifyConnectivity(); err != nil {
        return c.JSON(503, map[string]string{"status": "unhealthy", "neo4j": "down"})
    }

    // Check PostgreSQL
    sqlDB, _ := postgresDB.DB()
    if err := sqlDB.Ping(); err != nil {
        return c.JSON(503, map[string]string{"status": "unhealthy", "postgres": "down"})
    }

    // Check Redis
    if err := redisClient.Ping(context.Background()).Err(); err != nil {
        return c.JSON(503, map[string]string{"status": "unhealthy", "redis": "down"})
    }

    return c.JSON(200, map[string]string{"status": "healthy"})
}

Graceful Shutdown

Shutdown Handler:

  • Handle SIGTERM/SIGINT signals
  • Graceful HTTP server shutdown (wait up to 30s for in-flight requests)
  • Close database connections properly
  • Stop Watermill subscribers gracefully
  • Exit cleanly

Implementation Example:

func main() {
    // ... setup ...

    // Start server in goroutine
    go func() {
        if err := server.Start(":8080"); err != nil && err != http.ErrServerClosed {
            logger.Error("server error", "error", err)
        }
    }()

    // Wait for interrupt signal
    quit := make(chan os.Signal, 1)
    signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
    <-quit

    logger.Info("shutting down server")

    // Graceful shutdown
    ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()

    if err := server.Shutdown(ctx); err != nil {
        logger.Error("server shutdown error", "error", err)
    }

    // Close database connections
    neo4jDriver.Close()
    postgresDB.Close()
    redisClient.Close()

    // Stop Watermill subscribers
    ctx, cancel = context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()
    pubsub.Close()

    logger.Info("server stopped")
}

Rate Limiting

Rate Limiting:

  • IP-based rate limiting: 100 requests/minute per IP
  • Rate limit headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
  • Error response: 429 Too Many Requests

Implementation Example:

// Simple in-memory rate limiter (upgrade to Redis later)
func RateLimitMiddleware() echo.MiddlewareFunc {
    limiter := rate.NewLimiter(rate.Every(time.Minute), 100)

    return func(next echo.HandlerFunc) echo.HandlerFunc {
        return func(c echo.Context) error {
            ip := c.RealIP()
            if !limiter.Allow() {
                return c.JSON(429, ErrorResponse{
                    Error: "too_many_requests",
                    Message: "Rate limit exceeded",
                })
            }
            return next(c)
        }
    }
}

MVP Matching Algorithm (Horizontal - Simple but Complete)

Architecture: Multi-Step Pipeline (Foundation Pattern)

1. PostGIS: Fast spatial pre-filter
2. Neo4j: Graph traversal for compatibility
3. Go: Economic scoring
4. Redis: Cache results

Step 1: Spatial Pre-Filter (PostGIS via GORM)

// Using GORM with PostGIS
func FindSitesWithinRadius(ctx context.Context, db *gorm.DB, lat, lon float64, radiusKm float64) ([]SiteGeo, error) {
    var sites []SiteGeo

    query := `
        SELECT site_id, business_id, latitude, longitude
        FROM sites_geo
        WHERE ST_DWithin(
            location::geography,
            ST_MakePoint(?, ?)::geography,
            ?
        )
        ORDER BY ST_Distance(location, ST_MakePoint(?, ?))
    `

    radiusMeters := radiusKm * 1000
    err := db.WithContext(ctx).
        Raw(query, lon, lat, radiusMeters, lon, lat).
        Scan(&sites).
        Error

    return sites, err
}

Why PostGIS via GORM:

  • GORM simplifies database operations
  • Spatial indexes are optimized for radius queries
  • Filters 99% of data before Neo4j query
  • Much faster than Neo4j spatial queries at scale

Step 2: Graph Traversal (Neo4j)

// Simple graph query (foundation for complex traversals later)
MATCH (sourceFlow:ResourceFlow)-[:HOSTS]->(sourceSite:Site),
      (targetFlow:ResourceFlow)-[:HOSTS]->(targetSite:Site),
      (sourceFlow)-[:MATCHABLE_TO]->(targetFlow)
WHERE sourceFlow.direction = 'output'
  AND targetFlow.direction = 'input'
  AND sourceFlow.type = 'heat'
  AND targetFlow.type = 'heat'
  AND sourceSite.id IN $siteIds // from PostGIS filter
  AND ABS(sourceFlow.temperature_celsius - targetFlow.temperature_celsius) <= 10
RETURN sourceFlow, targetFlow,
       distance(point({longitude: sourceSite.longitude, latitude: sourceSite.latitude}),
                 point({longitude: targetSite.longitude, latitude: targetSite.latitude})) AS distance

Why Neo4j:

  • Natural graph traversal (relationships already defined)
  • Foundation for complex multi-hop queries later
  • Simple query now, add complexity as needed

Step 3: Economic Scoring (Go)

// Simple calculation (foundation for MILP optimization later)
func calculateSavings(inputFlow, outputFlow ResourceFlow, distanceKm float64) float64 {
    // Basic savings = cost difference * quantity
    savingsPerMonth := (inputFlow.CostPerKwh - outputFlow.CostPerKwh) * outputFlow.QuantityKwhPerMonth
    savingsPerYear := savingsPerMonth * 12

    // Simple transport cost (linear model, improve later)
    transportCost := distanceKm * 0.5 // €0.50/km simple model

    return savingsPerYear - (transportCost * outputFlow.QuantityKwhPerMonth * 12)
}

Why Simple Scoring:

  • Order-of-magnitude estimates find 80% of opportunities
  • Foundation pattern: simple function → complex optimization later
  • Fast to compute and understand

Step 4: Cache Results (Redis)

// Cache match results (5-minute TTL)
cacheKey := fmt.Sprintf("matches:%s", sourceFlowID)
redis.Set(cacheKey, matches, 5*time.Minute)

Foundation Pattern:

  • Each layer present but simple
  • Can enhance each independently later
  • No architectural rewrites needed

MVP User Flow

1. Business Signs Up (5 minutes)

  • Enter business name, location, contact
  • Declare one resource flow (e.g., "I emit 500 kWh/month heat @ 45°C")

2. See Matches (Instant)

  • Dashboard shows: "You could save €12,000/year with Factory B (2.3km away)"
  • Click to see details: distance, temperature match, estimated savings

3. Request Contact (One Click)

  • "I'm interested" button
  • Platform sends email to both businesses
  • They take conversation offline

4. Mark Status

  • "Contacted", "Not Interested", "Match Successful"
  • Simple feedback loop

MVP Features (Prioritized)

Phase 1: Launch (Week 1-4)

  1. Business registration form
  2. Basic matching algorithm
  3. Match results page
  4. Simple map view
  5. Contact request functionality

Phase 2: Engagement (Week 5-8)

  1. Email notifications for new matches
  2. Business profile page
  3. Match history/status tracking
  4. Basic analytics (match success rate)

Phase 3: Growth (Week 9-12)

  1. Social proof (success stories)
  2. Invite other businesses
  3. Simple ROI calculator
  4. Mobile-responsive design

MVP Go-to-Market Strategy

Target: Single Industrial Park or District

Why:

  • Concentrated geography = easier matching
  • Word-of-mouth spreads faster
  • Lower customer acquisition cost
  • Can validate with 20-50 businesses

Sales Approach

  1. Seed 5-10 Businesses

    • Offer free access
    • Get real data
    • Document first success story
  2. Cold Outreach

    • Identify 50 businesses in target area
    • Email: "We found €X savings opportunity in your area"
    • Free to join, pay only on successful match (lead fee model)
  3. Network Effects

    • First match → press release → more signups
    • Success stories on platform
    • Referral program

Pricing Model (MVP)

Free Tier:

  • List business
  • See matches
  • Request contact

Success Fee (Only if match succeeds):

  • €500-2000 per successful match
  • Or: 5% of first year savings

Why This Model:

  • No upfront cost = easier adoption
  • Businesses only pay when they get value
  • Aligns platform incentives with customer success

MVP Success Metrics

Week 1-4 (Launch)

  • Goal: 20 businesses registered
  • Goal: 5 matches found
  • Goal: 1 business requests contact

Week 5-12 (Validation)

  • Goal: 100 businesses registered
  • Goal: 20 matches found
  • Goal: 3 successful connections
  • Goal: €50k+ total savings identified

Month 4-6 (Traction)

  • Goal: 1st paying customer
  • Goal: 500 businesses
  • Goal: 10% match success rate
  • Goal: Revenue: €5k-10k/month

MVP Technical Architecture (Horizontal Foundation)

Architecture (All Layers Present, Each Simple)

┌─────────────┐
│   Frontend  │  React + Vite (Vercel/Netlify)
│  (React)    │
└──────┬──────┘
       │ HTTPS
┌──────▼──────────────────────────────┐
│   API Server (Go) - Echo            │
│   ├─ Auth Middleware (JWT)          │
│   ├─ Validation Middleware          │
│   ├─ Rate Limiting                  │
│   ├─ Request Logging (slog)         │
│   ├─ Error Handling                 │
│   ├─ HTTP Handlers                  │
│   └─ Watermill Subscribers          │
└───┬───────────┬─────────────────────┘
    │           │
┌───▼───┐   ┌──▼────┐
│Watermill│  │ Redis │  Cache, sessions
│(in-mem)│   │       │
└───┬───┘   └───────┘
    │
┌───▼──────────┬───────────┐
│              │           │
│   Neo4j      │  PostgreSQL│  Graph DB  │  PostGIS + GORM
│  (Graph)     │  (Geospatial)│
│  (with       │  (with      │
│   pooling)   │   pooling)  │
└──────────────┴───────────┘

Logging: slog (JSON for prod, text for dev)
Monitoring: /health endpoint, basic metrics
Shutdown: Graceful (SIGTERM/SIGINT handler)

Services (Horizontal MVP - All Present, Simple Config)

Application Layer:

  • Single Go Binary:
    • HTTP API (Echo)
    • JWT Authentication & Authorization
    • Input Validation (go-playground/validator)
    • Structured Logging (log/slog)
    • Error Handling (standardized responses)
    • Rate Limiting (in-memory, upgrade to Redis later)
    • CORS & Security Headers
    • Health Check Endpoint (/health)
    • Graceful Shutdown Handler
    • Event system (Watermill with in-memory pubsub)
    • Background workers (goroutines)
    • Worker pool pattern (controlled concurrency)

Data Layer:

  • Neo4j: Graph database (single instance, simple queries)
    • Connection pooling configured
    • Indexes: ResourceFlow.type + direction, Site.id
  • PostgreSQL + PostGIS with GORM: Geospatial queries (synced from Neo4j)
    • GORM for simpler CRUD operations
    • PostGIS for spatial queries
    • GIST spatial index on location
    • Connection pooling configured
  • Redis: Cache, sessions, match results
    • Connection pooling configured
    • Session storage for auth tokens
  • Event Processing: Watermill with in-memory pubsub (native Go channels under the hood, zero external deps)

Frontend:

  • React: Static site (Vercel)
  • Mapbox: Map visualization

Event Flow (Watermill - Clean Abstraction)

import (
    "log/slog"
    "github.com/ThreeDotsLabs/watermill"
    "github.com/ThreeDotsLabs/watermill/message"
    watermillMem "github.com/ThreeDotsLabs/watermill/pubsub/gochannel"
)

// Initialize Watermill with in-memory pubsub (MVP)
// Use slog adapter for Watermill
watermillLogger := watermill.NewStdLogger(false, false)
pubsub := watermillMem.NewGoChannel(watermillMem.Config{}, watermillLogger)

// Subscribe to events
messages, err := pubsub.Subscribe(ctx, "resource_flow_created")
if err != nil {
    slog.Error("failed to subscribe", "error", err)
    return err
}

// Background worker with logging
go func() {
    for msg := range messages {
        var resourceFlow ResourceFlow
        if err := json.Unmarshal(msg.Payload, &resourceFlow); err != nil {
            slog.Error("failed to unmarshal event", "error", err)
            msg.Nack()
            continue
        }

        slog.Info("processing event",
            "event", "resource_flow_created",
            "resource_flow_id", resourceFlow.ID,
        )

        // Handle event
        if err := handleResourceFlowCreated(ctx, resourceFlow); err != nil {
            slog.Error("failed to handle event", "error", err, "resource_flow_id", resourceFlow.ID)
            msg.Nack()
            continue
        }

        slog.Info("event processed", "resource_flow_id", resourceFlow.ID)
        msg.Ack()
    }
}()

// In API handler with validation and logging
func createResourceFlow(c echo.Context) error {
    var rf ResourceFlow
    if err := c.Bind(&rf); err != nil {
        slog.Warn("invalid request body", "error", err)
        return echo.NewHTTPError(400, "invalid request body")
    }

    // Validate
    if err := validate.Struct(rf); err != nil {
        slog.Warn("validation failed", "error", err)
        return echo.NewHTTPError(400, err.Error())
    }

    // Create in Neo4j with transaction
    if err := createInNeo4j(ctx, rf); err != nil {
        slog.Error("failed to create in Neo4j", "error", err)
        return echo.ErrInternalServerError
    }

    slog.Info("resource flow created",
        "resource_flow_id", rf.ID,
        "business_id", rf.BusinessID,
    )

    // Publish event via Watermill
    eventPayload, _ := json.Marshal(rf)
    msg := message.NewMessage(watermill.NewUUID(), eventPayload)
    if err := pubsub.Publish("resource_flow_created", msg); err != nil {
        slog.Error("failed to publish event", "error", err)
        // Continue - event will be retried or logged
    }

    return c.JSON(200, rf)
}

Flow:

User Action → API Handler → Auth → Validation → Logging → Watermill (in-memory) → Subscribers
                                                                                        ↓
                                                                                Neo4j Write (txn)
                                                                                        ↓
                                                                                PostGIS Sync (async)
                                                                                        ↓
                                                                                Match Computation
                                                                                        ↓
                                                                                Redis Cache
                                                                                        ↓
                                                                                Log Result (slog)

Why Watermill:

  • Clean pubsub abstraction (not tied to specific backend)
  • Start with in-memory Go channels (zero external deps)
  • Easy migration to Redis Streams, Kafka, NATS (change pubsub, keep code)
  • Production-ready patterns (middlewares, CQRS support)
  • Well-maintained and recommended in 2025
  • Perfect for MVP → production path
  • Integrated with slog for event logging

Deployment (Managed Services)

Option 1: Fully Managed (Easiest)

  • Railway/Render: Neo4j Aura, PostgreSQL, Redis
  • One-click deploy, managed backups
  • Slightly higher cost but zero ops
  • No Kafka needed - Watermill in-memory pubsub handles events

Option 2: Self-Hosted (More Control)

  • Single Server: DigitalOcean Droplet ($20/month)
  • Docker Compose: Neo4j, PostgreSQL, Redis
  • More control, lower cost, more ops
  • No Kafka needed - Watermill in-memory pubsub handles events

Why Watermill (Not Kafka) for MVP:

  • Watermill with in-memory pubsub handles all event processing
  • Zero external dependencies (uses Go channels under the hood)
  • Clean abstraction allows easy backend switching later
  • Simpler deployment and operations
  • Switch to Redis Streams/Kafka/NATS only when:
    • Need distributed processing (multiple servers)
    • Need event persistence/replay
    • Need high-throughput event streaming
  • Same code, just swap the pubsub implementation

MVP Development Timeline (Foundation Stack)

Week 1: Foundation Setup

  • Neo4j setup (schema, basic nodes/relationships, indexes)
  • PostgreSQL + PostGIS setup (GIST spatial index)
  • Redis setup (connection pooling)
  • Go project structure
  • Environment configuration (.env files, environment variables)
  • Structured logging setup (log/slog: JSON for prod, text for dev)
  • Error handling framework (standardized error responses)
  • Connection pooling configuration (Neo4j, PostgreSQL, Redis)

Week 2: Core Backend + Security

  • Go API with Echo
  • JWT authentication (middleware, token generation)
  • Authorization (RBAC: Business Owner, Public)
  • Input validation (go-playground/validator)
  • Error handling middleware
  • Request logging middleware (slog)
  • Neo4j driver integration (with connection pooling)
  • PostgreSQL + PostGIS with GORM integration (with connection pooling)
  • Health check endpoint (/health)
  • Rate limiting middleware (basic in-memory, upgrade to Redis later)
  • CORS configuration
  • Security headers middleware

Week 3: Event Processing + Data Layer

  • Event system with Watermill (in-memory pubsub)
  • Watermill subscribers setup
  • Event handlers (Neo4j write, PostGIS sync)
  • Background match computation (Watermill subscribers)
  • Message routing and error handling
  • Graceful shutdown handler (SIGTERM/SIGINT)
  • Database transaction management
  • Basic matching algorithm (all 3 steps)
  • Redis caching layer

Week 4: Frontend

  • React app setup
  • Business registration form (with validation)
  • Login/authentication flow
  • Match results page
  • Map view (Mapbox)
  • Real-time updates (polling initially, WebSocket later)
  • Error handling (display error messages)

Week 5: Integration & Testing

  • End-to-end integration
  • Unit tests (matching algorithm, validation logic)
  • Integration tests (API endpoints, database operations)
  • E2E test (business registration → match flow)
  • Email notifications
  • Contact request flow
  • Testing with seed data
  • Bug fixes

Week 6: Launch Prep + Monitoring

  • Production deployment
  • Monitoring setup (basic metrics, health checks)
  • Log aggregation (configure logging output)
  • Documentation (API docs, deployment guide)
  • Marketing materials
  • Final testing and bug fixes

Week 7: Polish & Launch (Buffer)

  • Performance testing
  • Security audit (basic)
  • Load testing (simple)
  • Documentation polish
  • Marketing materials finalization
  • Launch!

Total: ~7-8 weeks to MVP launch (includes production-ready foundations)


MVP Risks & Mitigations

Risk 1: Not Enough Businesses

Mitigation:

  • Target single area (industrial park)
  • Seed with 10 businesses yourself
  • Offer free access initially

Risk 2: No Matches Found

Mitigation:

  • Start with larger geographic radius (10km)
  • Focus on common resources (heat, water)
  • Expand matching criteria

Risk 3: Businesses Don't Engage

Mitigation:

  • Simple interface (5-minute setup)
  • Clear value proposition (€X savings)
  • Automated email reminders

Risk 4: Technical Complexity

Mitigation:

  • Use PostgreSQL (not Neo4j initially)
  • Keep algorithm simple
  • Launch fast, iterate based on feedback

MVP to Full Product Path (Vertical Scaling)

Foundation is Built - Now Scale Each Layer

Phase 1: Enhance Matching (100 businesses)

  • Current: Simple 3-step matching
  • Enhance:
    • Add temporal overlap checking (time profiles)
    • Improve economic scoring (better cost models)
    • Add quality compatibility matrix
  • No migration needed: Enhance Go algorithm

Phase 2: Complex Graph Queries (500 businesses)

  • Current: Simple Cypher queries
  • Enhance:
    • Multi-hop graph traversals (find clusters)
    • Complex relationship patterns
    • Graph-based clustering algorithms
  • No migration needed: Write more complex Cypher

Phase 3: Advanced Matching (1000 businesses)

  • Current: Simple economic scoring
  • Enhance:
    • MILP optimization for multi-party matches
    • Genetic algorithms for clustering
    • Machine learning for match ranking
  • No migration needed: Add optimization service

Phase 4: Distributed Processing (1000+ businesses, multiple servers)

  • Current: Watermill in-memory pubsub (single server)
  • Enhance:
    • Replace in-memory pubsub with Redis Streams or Kafka pubsub
    • Multiple worker servers
    • Event persistence and replay
  • Migration: Swap Watermill pubsub implementation (same code, different backend)

Phase 5: Multi-Region (Enterprise)

  • Current: Single region
  • Enhance:
    • Graph federation (multiple Neo4j instances)
    • Cross-region matching
    • Regional data residency
  • No migration needed: Add federation layer

Migration Strategy: Vertical Enhancement

  • Neo4j: Simple queries → Complex traversals (same DB)
  • PostGIS: Basic distance → Advanced spatial operations (same DB)
  • Events: Watermill in-memory → Watermill Redis Streams/Kafka (same code, swap pubsub)
  • Matching: Simple algorithm → MILP optimization (same service)
  • Workers: Single server → Multiple servers (same worker pattern)

Key Advantage: Foundation stays, we enhance each layer independently


MVP Sales Pitch (1-Minute Version)

"We help local businesses save money by finding resource matches. For example, if you emit waste heat, we find nearby businesses that need heat. One connection can save €10k-50k per year. It's free to join—we only get paid if you save money. Takes 5 minutes to list your business. Want to see if we have matches in your area?"


Key Principles for Horizontal MVP

  1. Right Foundation: All architectural layers present (Neo4j, PostGIS, Redis)
  2. Go Native First: Use Watermill with in-memory pubsub (zero external deps)
  3. Keep Each Simple: Simple queries, simple algorithms, simple setup
  4. Scale Vertically: Enhance each layer independently, no rewrites
  5. Focus on One Thing: Heat matching only (not all resources)
  6. One Geography: One industrial park (not global)
  7. Manual First: Web forms (not IoT) for data entry
  8. Clear Value: €X savings per year (not abstract concepts)
  9. Low Friction: 5-minute signup, free to try
  10. Network Effects: More users = better matches
  11. Success-Based Pricing: Only pay if you save money

Horizontal MVP Philosophy:

  • Build complete foundation now (all layers)
  • Use Go-native patterns first (Watermill in-memory pubsub)
  • Keep each layer simple initially
  • Add complexity to each layer as needed (vertical scaling)
  • Avoid architectural rewrites (horizontal changes)
  • Replace Watermill in-memory pubsub with Redis/Kafka pubsub only when needed (distributed processing)

Event Processing Pattern (Watermill):

import (
    "github.com/ThreeDotsLabs/watermill"
    "github.com/ThreeDotsLabs/watermill/message"
    watermillMem "github.com/ThreeDotsLabs/watermill/pubsub/gochannel"
    // Later: watermillRedis "github.com/ThreeDotsLabs/watermill/pubsub/redis/v2"
    // Later: watermillKafka "github.com/ThreeDotsLabs/watermill/pubsub/kafka"
)

// MVP: In-memory pubsub (uses Go channels under the hood)
logger := watermill.NewStdLogger(false, false)
pubsub := watermillMem.NewGoChannel(watermillMem.Config{}, logger)

// Later: Swap to Redis Streams (same code, different initialization)
// redisClient, _ := redis.NewClient(...)
// pubsub := watermillRedis.NewPublisher(redisClient, watermillRedis.PublisherConfig{}, logger)

// Later: Swap to Kafka (same code, different initialization)
// saramaConfig := sarama.NewConfig()
// pubsub := watermillKafka.NewPublisher(watermillKafka.PublisherConfig{}, logger)

// Same API for all backends:
pubsub.Publish("resource_flow_created", msg)
messages, _ := pubsub.Subscribe(ctx, "resource_flow_created")

Next Steps

  1. Validate Concept: Talk to 10 businesses in target area
  2. Build MVP: 5-week sprint to launch
  3. Seed Users: Get 20 businesses signed up
  4. Find First Match: Document success story
  5. Market & Grow: Use success story to acquire more users
  6. Iterate: Add features based on user feedback

Goal: Sellable MVP with proper foundation in 7-8 weeks, revenue in 3 months.

Updated Timeline: Includes production-ready foundations (auth, validation, logging, error handling, graceful shutdown).

Foundation Benefits:

  • No expensive migrations later
  • Can scale each layer independently
  • Right architecture from day 1
  • Easy to add complexity without rewrites