mirror of
https://github.com/SamyRai/tercul-backend.git
synced 2025-12-27 05:11:34 +00:00
f66936bc4b
This commit implements a robust, production-ready analytics system using an event-driven architecture with Redis and `asynq`. Key changes: - Event-Driven Architecture: Instead of synchronous database updates, analytics events (e.g., views, likes, comments) are now published to a Redis queue. This improves API response times and decouples the analytics system from the main application flow. - Background Worker: A new worker process (`cmd/worker`) has been created to consume events from the queue and update the analytics counters in the database. - View Counting: Implemented the missing view counting feature for both works and translations. - New Analytics Query: Added a `popularTranslations` GraphQL query to demonstrate how to use the collected analytics data. - Testing: Added unit tests for the new event publisher and integration tests for the analytics worker. Known Issue: The integration tests for the analytics worker (`AnalyticsWorkerSuite`) and the GraphQL API (`GraphQLIntegrationSuite`) are currently failing due to the lack of a Redis service in the test environment. The tests are written and are expected to pass in an environment where Redis is available on `localhost:6379`, as configured in the CI pipeline.
108 lines
5.7 KiB
Markdown
108 lines
5.7 KiB
Markdown
# TODO List for Tercul Go Application
|
|
|
|
---
|
|
|
|
## Suggested Next Objectives
|
|
|
|
- [x] **Complete the Architecture Refactor (High, 5d):** Finalize the transition to a clean, domain-driven architecture. This will significantly improve maintainability, scalability, and developer velocity.
|
|
- [x] Ensure resolvers call application services only and add dataloaders per aggregate.
|
|
- [ ] Adopt a migrations tool and move all SQL to migration files.
|
|
- [ ] Implement full observability with centralized logging, metrics, and tracing.
|
|
- [x] **Full Test Coverage (High, 5d):** Increase test coverage across the application to ensure stability and prevent regressions.
|
|
- [x] Write unit tests for all models, repositories, and services.
|
|
- [x] Refactor existing tests to use mocks instead of a real database.
|
|
- [ ] **Implement Analytics Features (High, 3d):** Add analytics to provide insights into user engagement and content popularity.
|
|
- [ ] Implement view, like, comment, and bookmark counting.
|
|
- [ ] Track translation analytics to identify popular translations.
|
|
- [ ] **Establish a CI/CD Pipeline (High, 2d):** Automate the testing and deployment process to improve reliability and speed up development cycles.
|
|
- [x] Add `make lint test test-integration` to the CI pipeline.
|
|
- [ ] Set up automated deployments to a staging environment.
|
|
- [ ] **Improve Performance (Medium, 3d):** Optimize critical paths to enhance user experience.
|
|
- [ ] Implement batching for Weaviate operations.
|
|
- [ ] Add performance benchmarks for critical paths.
|
|
|
|
---
|
|
|
|
## [ ] High Priority
|
|
|
|
### [ ] Architecture Refactor (DDD-lite)
|
|
- [x] Refactor domains to be more testable and decoupled, with 100% unit test coverage and logging.
|
|
- [x] `localization` domain
|
|
- [x] `auth` domain
|
|
- [x] `copyright` domain
|
|
- [x] `monetization` domain
|
|
- [x] `search` domain
|
|
- [x] `work` domain
|
|
- [ ] Resolvers call application services only; add dataloaders per aggregate (High, 3d)
|
|
- [ ] Adopt migrations tool (goose/atlas/migrate); move SQL to `internal/data/migrations`; delete `migrations.go` (High, 2d)
|
|
- [ ] Observability: centralize logging; add Prometheus metrics and OpenTelemetry tracing; request IDs (High, 3d)
|
|
- [x] CI: add `make lint test test-integration` and integration tests with Docker compose (High, 2d)
|
|
|
|
### [x] Testing
|
|
- [x] Add unit tests for all models, repositories, and services (High, 3d)
|
|
- [x] Remove DB logic from `BaseSuite` for mock-based integration tests (High, 2d)
|
|
|
|
### [ ] Features
|
|
- [ ] Implement analytics data collection (High, 3d)
|
|
- [ ] Implement view counting for works and translations
|
|
- [ ] Implement like counting for works and translations
|
|
- [ ] Implement comment counting for works
|
|
- [ ] Implement bookmark counting for works
|
|
- [ ] Implement translation counting for works
|
|
- [ ] Implement translation analytics to show popular translations
|
|
|
|
---
|
|
|
|
## [ ] Medium Priority
|
|
|
|
### [ ] Performance Improvements
|
|
- [ ] Implement batching for Weaviate operations (Medium, 2d)
|
|
|
|
### [ ] Code Quality & Architecture
|
|
- [ ] Expand Weaviate client to support all models (Medium, 2d)
|
|
- [ ] Add code documentation and API docs (Medium, 2d)
|
|
- [ ] Replace bespoke cached repositories with decorators in `internal/data/cache` (reads only; deterministic invalidation) (Medium, 2d)
|
|
- [ ] Config: replace ad-hoc config with env parsing + validation (e.g., koanf/envconfig); no globals (Medium, 1d)
|
|
|
|
### [ ] Testing
|
|
- [ ] Add performance benchmarks for critical paths (Medium, 2d)
|
|
- [ ] Add benchmarks for text analysis (sequential vs concurrent) and cache hit/miss rates
|
|
|
|
### [ ] Monitoring & Logging
|
|
- [ ] Add monitoring for background jobs and API endpoints (Medium, 2d)
|
|
- [ ] Add metrics for linguistics: analysis duration, cache hit/miss, provider usage
|
|
|
|
---
|
|
|
|
## [ ] Low Priority
|
|
|
|
### [ ] Testing
|
|
- [ ] Refactor `RunTransactional` to be mock-friendly (Low, 1d)
|
|
|
|
---
|
|
|
|
## [ ] Completed
|
|
|
|
- [x] Add comprehensive input validation for all GraphQL mutations (High, 2d) - *Partially complete. Core mutations are validated.*
|
|
- [x] Create skeleton packages: `cmd/`, `internal/platform/`, `internal/domain/`, `internal/app/`, `internal/data/`, `internal/adapters/graphql/`, `internal/jobs/`
|
|
- [x] Move infra to `internal/platform/*` (`config`, `db`, `cache`, `auth`, `http`, `log`, `search`)
|
|
- [x] Wire DI in `cmd/api/main.go` and expose an `Application` facade to adapters
|
|
- [x] Unify GraphQL under `internal/adapters/graphql` and update `gqlgen.yml`; move `schema.graphqls` and resolvers
|
|
- [x] Introduce Unit-of-Work: `platform/db.WithTx(ctx, func(ctx) error)` and repo factory for `*sql.DB` / `*sql.Tx`
|
|
- [x] Split write vs read paths for `work` (commands.go, queries.go); make read models cacheable
|
|
- [x] Restructure `models/*` into domain aggregates with constructors and invariants
|
|
- [x] Security: move JWT/middleware to `internal/platform/auth`; add authz policy helpers (e.g., `CanEditWork`)
|
|
- [x] Search: move Weaviate client/schema to `internal/platform/search`, optional domain interface
|
|
- [x] Background jobs: move to `cmd/worker` and `internal/jobs/*`; ensure idempotency and lease
|
|
- [x] Python ops: move scripts to `/ops/migration` and `/ops/analysis`; keep outputs under `/ops/migration/outputs/`
|
|
- [x] Cleanup: delete dead packages (`store`, duplicate `repositories`); consolidate to `internal/data/sql`
|
|
- [x] Add integration tests for GraphQL API and background jobs (High, 3d) - *Partially complete. Core mutations are tested.*
|
|
- [x] Stabilize non-linguistics tests and interfaces (High, 2d)
|
|
- [x] Fix `graph` mocks to accept context in service interfaces
|
|
- [x] Update `repositories` tests (missing `TestModel`) and align with new repository interfaces
|
|
- [x] Update `services` tests to pass context and implement missing repo methods in mocks
|
|
|
|
---
|
|
|
|
> TODO items include context, priority, and estimated effort. Update this list after each milestone.
|