tercul-backend/TODO.md

TODO List for Tercul Go Application

---

[x] Performance Improvements

• COMPLETED: Add pagination to all repository list operations (High, 2d)
  • /works: Add limit/offset support to repository and resolver
  • /translations: Add limit/offset support to repository and resolver
  • /authors: Add limit/offset support to repository and resolver
  • /users: Add limit/offset support to repository and resolver
  • /collections: Add limit/offset support to repository and resolver
  • /tags: Add limit/offset support to repository and resolver
  • /categories: Add limit/offset support to repository and resolver
  • /comments: Add limit/offset support to repository and resolver
  • /search: Add limit/offset support to repository and resolver
  • Validate all endpoints for correct pagination and total count
  • Add unit tests for paginated list operations
  • Document pagination parameters in API docs
• COMPLETED: Refactor raw SQL queries to use GORM structured methods (High, 1d)
  • Identify all usages of raw SQL queries in repositories and sync jobs
  • Refactor syncEntities in syncjob/entities_sync.go to use GORM methods
  • Refactor any string-concatenated queries to parameterized GORM queries
  • Validate correctness and performance of refactored queries
  • Add unit tests for refactored query logic
  • Document query changes and migration steps
• Implement batching for Weaviate operations (Medium, 2d)
• COMPLETED: Optimize linguistic analysis algorithms (Medium, 2d)
  • Introduced clean NLP ports/adapters (LanguageDetector, SentimentProvider, KeywordProvider)
  • Integrated lingua-go (language detection) and GoVADER (sentiment) behind adapters
  • Added TF-IDF-based keyword provider (lightweight, state-free)
  • Bounded in-memory cache via LRU with config-driven capacity
  • Switched text cache keys to SHA-256 content hashes
  • Concurrent analysis: provider-aware and context-cancellable
  • Config toggles for providers and cache TTL
• COMPLETED: Add database indexes for frequently queried fields (Medium, 1d)
  • Foreign key indexes for all relationships
  • Unique indexes for constraint enforcement
  • Timestamp indexes for sorting and filtering
  • Composite indexes for complex queries
  • Linguistic analysis indexes for performance
• COMPLETED: Implement Redis caching for hot data (Medium, 2d)

[x] Security Enhancements

• COMPLETED: Implement password hashing in User model (Critical, 1d)
  • bcrypt password hashing in BeforeSave hook
  • CheckPassword method for password verification
  • Automatic password hashing on model save
• COMPLETED: Move hardcoded credentials to environment variables/config (Critical, 1d)
  • Fixed internal/cmd/enrich/main.go to use config package
  • Fixed internal/testutil/testutil.go to use config package
  • All database connections now use environment variables
• Add comprehensive input validation for all GraphQL mutations (High, 2d)
• COMPLETED: Implement rate limiting for API and background jobs (High, 2d)
  • Rate limiting middleware implemented
  • Configuration for rate limits in config package
• COMPLETED: Replace raw SQL with safe query builders to prevent SQL injection (Critical, 1d)
  • All repositories use GORM structured methods
  • No raw SQL queries in production code

[ ] Code Quality & Architecture

• Remove duplicate GraphQL folder and legacy server helper; keep single GraphQL layer under graph/ for now
• REFACTORED: Split linguistics/analyzer.go into focused components (Completed)
• COMPLETED: Clean NLP infrastructure and factory wiring
  • Ports for NLP capabilities with SRP/DRY boundaries
  • Adapters for lingua-go and GoVADER with fallbacks
  • Factory respects config toggles and wires providers
  • Repository no longer leaks GORM into services; added methods for fetching work and analysis data
  • Created linguistics/text_analyzer.go - Pure text analysis logic
  • Created linguistics/analysis_cache.go - Caching logic with multiple strategies
  • Created linguistics/analysis_repository.go - Database operations
  • Created linguistics/work_analysis_service.go - Work-specific analysis coordination
  • Created linguistics/types.go - Shared data structures
  • Created linguistics/text_utils.go - Text processing utilities
  • Created linguistics/factory.go - Component factory with dependency injection
• REFACTORED: Split main.go into focused components (Completed)
  • Created internal/app/application_builder.go - Application initialization
  • Created internal/app/server_factory.go - Server creation and configuration
  • Refactored main.go to use dependency injection and builders
• REFACTORED: Standardize repository implementation (Completed)
  • Improved BaseRepository with comprehensive error handling, validation, logging, and transaction support
  • Removed GenericRepository wrapper (unnecessary duplication)
  • Updated CachedRepository to use BaseRepository interface
  • Refactored WorkRepository and UserRepository to use BaseRepository pattern
  • Updated WorkService to use context in all repository calls
  • Fixed GraphQL resolvers to use context for WorkRepository calls
  • REFACTORED: All repositories completed! (Author, Tag, Category, Translation, Comment, Like, Bookmark, Collection, Book, Publisher, Country, Place, City, Source, Edition, UserProfile, UserSession, EmailVerification, PasswordReset, Contribution, Copyright, CopyrightClaim, Monetization, Edge)
  • COMPLETED: Updated mock repositories for testing
  • COMPLETED: Updated services to use context in repository calls
  • COMPLETED: Updated GraphQL resolvers to use context and handle pagination
  • COMPLETED: Fixed linguistics package model field mismatches
  • COMPLETED: Fixed application builder CopyrightRepository initialization
  • COMPLETED: Fixed server factory configuration and interface issues
  • COMPLETED: Removed all legacy code and interfaces
  • COMPLETED: Project builds successfully!
• COMPLETED: Add a service layer for business logic and validation (High, 2d)
  • Comprehensive validation in all service methods
  • Business logic separation from repositories
  • Input validation for all service operations
• Refactor duplicate code in sync jobs (Medium, 1d)
• COMPLETED: Improve error handling with custom error types and propagation (High, 2d)
  • Custom error types defined in BaseRepository
  • Error wrapping and propagation throughout codebase
  • Standardized error handling patterns
• Expand Weaviate client to support all models (Medium, 2d)
• Add code documentation and API docs (Medium, 2d)

[ ] Testing

• Add unit tests for all models, repositories, and services (High, 3d)
• Add integration tests for GraphQL API and background jobs (High, 3d)
• Add performance benchmarks for critical paths (Medium, 2d)
  • Added unit tests for linguistics adapters (lingua-go, GoVADER) and utilities
  • Add benchmarks for text analysis (sequential vs concurrent) and cache hit/miss rates

[x] Monitoring & Logging

• COMPLETED: Integrate a structured logging framework (Medium, 1d)
  • Structured logging implemented throughout codebase
  • Performance timing and debug logging in repositories
  • Error logging with context and structured fields
• Add monitoring for background jobs and API endpoints (Medium, 2d)
  • Add metrics for linguistics: analysis duration, cache hit/miss, provider usage

---

Next Objective Proposal

• Stabilize non-linguistics tests and interfaces (High, 2d)
  • Fix graph mocks to accept context in service interfaces
  • Update repositories tests (missing TestModel) and align with new repository interfaces
  • Update services tests to pass context and implement missing repo methods in mocks
• Add performance benchmarks and metrics for linguistics (Medium, 2d)
  • Benchmarks for AnalyzeText (provider on/off, concurrency levels)
  • Export metrics and dashboards for analysis duration and cache effectiveness
• Documentation (Medium, 1d)
  • Document NLP provider toggles and defaults in README/config docs
  • Describe SRP/DRY design and extension points for new providers

[x] Security & Auth

• COMPLETED: Implement JWT authentication and role-based authorization (High, 2d)
  • JWT token generation and validation with proper error handling
  • Role-based authorization with hierarchy (reader < contributor < reviewer < editor < admin)
  • Authentication middleware for GraphQL and HTTP with context validation
  • Login and registration mutations with comprehensive input validation
  • Password hashing with bcrypt (already implemented in User model)
  • Environment variable configuration for JWT with secure defaults
  • Comprehensive authentication service following SRP and clean code principles
  • Structured logging with proper error context and performance timing
  • Input sanitization and validation using govalidator
  • Context validation and proper error propagation
  • Integration with existing rate limiting system
  • GraphQL schema alignment with Go models
  • Comprehensive test coverage for authentication components
  • Production-ready error handling and security practices
• COMPLETED: Add rate limiting middleware (High, 1d)
  • Rate limiting middleware implemented and tested
  • Configuration-driven rate limits
• COMPLETED: Use environment variables for all sensitive config (Critical, 1d)
  • All database credentials use environment variables
  • Redis configuration uses environment variables
  • Centralized configuration management

---

TODO items include context, priority, and estimated effort. Update this list after each milestone.