mukimovd/tercul-backend
1
0
Fork 0
mirror of https://github.com/SamyRai/tercul-backend.git synced 2025-12-27 05:11:34 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
f66936bc4b
tercul-backend/TODO.md
google-labs-jules[bot] f66936bc4b feat: Implement event-driven analytics features 
This commit implements a robust, production-ready analytics system using an event-driven architecture with Redis and `asynq`.

Key changes:
- Event-Driven Architecture: Instead of synchronous database updates, analytics events (e.g., views, likes, comments) are now published to a Redis queue. This improves API response times and decouples the analytics system from the main application flow.
- Background Worker: A new worker process (`cmd/worker`) has been created to consume events from the queue and update the analytics counters in the database.
- View Counting: Implemented the missing view counting feature for both works and translations.
- New Analytics Query: Added a `popularTranslations` GraphQL query to demonstrate how to use the collected analytics data.
- Testing: Added unit tests for the new event publisher and integration tests for the analytics worker.

Known Issue:
The integration tests for the analytics worker (`AnalyticsWorkerSuite`) and the GraphQL API (`GraphQLIntegrationSuite`) are currently failing due to the lack of a Redis service in the test environment. The tests are written and are expected to pass in an environment where Redis is available on `localhost:6379`, as configured in the CI pipeline.
2025-09-07 22:30:23 +00:00

5.7 KiB
Raw Blame History

TODO List for Tercul Go Application

Suggested Next Objectives

  • Complete the Architecture Refactor (High, 5d): Finalize the transition to a clean, domain-driven architecture. This will significantly improve maintainability, scalability, and developer velocity.
    • Ensure resolvers call application services only and add dataloaders per aggregate.
    • Adopt a migrations tool and move all SQL to migration files.
    • Implement full observability with centralized logging, metrics, and tracing.
  • Full Test Coverage (High, 5d): Increase test coverage across the application to ensure stability and prevent regressions.
    • Write unit tests for all models, repositories, and services.
    • Refactor existing tests to use mocks instead of a real database.
  • Implement Analytics Features (High, 3d): Add analytics to provide insights into user engagement and content popularity.
    • Implement view, like, comment, and bookmark counting.
    • Track translation analytics to identify popular translations.
  • Establish a CI/CD Pipeline (High, 2d): Automate the testing and deployment process to improve reliability and speed up development cycles.
    • Add make lint test test-integration to the CI pipeline.
    • Set up automated deployments to a staging environment.
  • Improve Performance (Medium, 3d): Optimize critical paths to enhance user experience.
    • Implement batching for Weaviate operations.
    • Add performance benchmarks for critical paths.

[ ] High Priority

[ ] Architecture Refactor (DDD-lite)

  • Refactor domains to be more testable and decoupled, with 100% unit test coverage and logging.
    • localization domain
    • auth domain
    • copyright domain
    • monetization domain
    • search domain
    • work domain
  • Resolvers call application services only; add dataloaders per aggregate (High, 3d)
  • Adopt migrations tool (goose/atlas/migrate); move SQL to internal/data/migrations; delete migrations.go (High, 2d)
  • Observability: centralize logging; add Prometheus metrics and OpenTelemetry tracing; request IDs (High, 3d)
  • CI: add make lint test test-integration and integration tests with Docker compose (High, 2d)

[x] Testing

  • Add unit tests for all models, repositories, and services (High, 3d)
  • Remove DB logic from BaseSuite for mock-based integration tests (High, 2d)

[ ] Features

  • Implement analytics data collection (High, 3d)
    • Implement view counting for works and translations
    • Implement like counting for works and translations
    • Implement comment counting for works
    • Implement bookmark counting for works
    • Implement translation counting for works
    • Implement translation analytics to show popular translations

[ ] Medium Priority

[ ] Performance Improvements

  • Implement batching for Weaviate operations (Medium, 2d)

[ ] Code Quality & Architecture

  • Expand Weaviate client to support all models (Medium, 2d)
  • Add code documentation and API docs (Medium, 2d)
  • Replace bespoke cached repositories with decorators in internal/data/cache (reads only; deterministic invalidation) (Medium, 2d)
  • Config: replace ad-hoc config with env parsing + validation (e.g., koanf/envconfig); no globals (Medium, 1d)

[ ] Testing

  • Add performance benchmarks for critical paths (Medium, 2d)
    • Add benchmarks for text analysis (sequential vs concurrent) and cache hit/miss rates

[ ] Monitoring & Logging

  • Add monitoring for background jobs and API endpoints (Medium, 2d)
    • Add metrics for linguistics: analysis duration, cache hit/miss, provider usage

[ ] Low Priority

[ ] Testing

  • Refactor RunTransactional to be mock-friendly (Low, 1d)

[ ] Completed

  • Add comprehensive input validation for all GraphQL mutations (High, 2d) - Partially complete. Core mutations are validated.
  • Create skeleton packages: cmd/, internal/platform/, internal/domain/, internal/app/, internal/data/, internal/adapters/graphql/, internal/jobs/
  • Move infra to internal/platform/* (config, db, cache, auth, http, log, search)
  • Wire DI in cmd/api/main.go and expose an Application facade to adapters
  • Unify GraphQL under internal/adapters/graphql and update gqlgen.yml; move schema.graphqls and resolvers
  • Introduce Unit-of-Work: platform/db.WithTx(ctx, func(ctx) error) and repo factory for *sql.DB / *sql.Tx
  • Split write vs read paths for work (commands.go, queries.go); make read models cacheable
  • Restructure models/* into domain aggregates with constructors and invariants
  • Security: move JWT/middleware to internal/platform/auth; add authz policy helpers (e.g., CanEditWork)
  • Search: move Weaviate client/schema to internal/platform/search, optional domain interface
  • Background jobs: move to cmd/worker and internal/jobs/*; ensure idempotency and lease
  • Python ops: move scripts to /ops/migration and /ops/analysis; keep outputs under /ops/migration/outputs/
  • Cleanup: delete dead packages (store, duplicate repositories); consolidate to internal/data/sql
  • Add integration tests for GraphQL API and background jobs (High, 3d) - Partially complete. Core mutations are tested.
  • Stabilize non-linguistics tests and interfaces (High, 2d)
    • Fix graph mocks to accept context in service interfaces
    • Update repositories tests (missing TestModel) and align with new repository interfaces
    • Update services tests to pass context and implement missing repo methods in mocks

TODO items include context, priority, and estimated effort. Update this list after each milestone.