From dcdc272990e2897c5e4db5d27a402f8ec5d6914d Mon Sep 17 00:00:00 2001 From: "google-labs-jules[bot]" <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Sat, 4 Oct 2025 23:59:30 +0000 Subject: [PATCH] docs: consolidate tasks and clean up legacy files This commit consolidates all outstanding tasks from `TODO.md`, `BUILD_ISSUES.md`, and inline `TODO` comments into a single `TASKS.md` file. This new file serves as the single source of truth for all pending development work, categorized by priority. The following legacy files have been removed to avoid confusion: - `TODO.md` - `BUILD_ISSUES.md` The `AGENTS.md` file has also been updated to remove a temporary debug log and now directs agents to the new `TASKS.md` file for guidance. --- AGENTS.md | 18 +------- BUILD_ISSUES.md | 14 ------- TASKS.md | 75 +++++++++++++++++++++++++++++++++ TODO.md | 107 ------------------------------------------------ 4 files changed, 77 insertions(+), 137 deletions(-) delete mode 100644 BUILD_ISSUES.md create mode 100644 TASKS.md delete mode 100644 TODO.md diff --git a/AGENTS.md b/AGENTS.md index ff33bf4..a9f53a8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,17 +1,3 @@ -# Agent Debugging Log +# Agent Instructions -## Issue: Integration Test Failures - -I've been encountering a series of integration test failures related to `unauthorized`, `forbidden`, and `directive binding is not implemented` errors. - -### Initial Investigation - -1. **`directive binding is not implemented` error:** This error was caused by the test server in `internal/adapters/graphql/integration_test.go` not being configured with the necessary validation directive. -2. **`unauthorized` and `forbidden` errors:** These errors were caused by tests that require authentication not being run with an authenticated user. -3. **Build Error:** My initial attempts to fix the test server setup introduced a build error in `cmd/api` due to a function signature mismatch in `NewServerWithAuth`. - -### Resolution Path - -1. **Fix Build Error:** I corrected the function signature in `cmd/api/server.go` to match the call site in `cmd/api/main.go`. This resolved the build error. -2. **Fix Test Server Setup:** I updated the `SetupSuite` function in `internal/adapters/graphql/integration_test.go` to register the `binding` directive, aligning the test server configuration with the production server. -3. **Fix Authentication in Tests:** The remaining `forbidden` errors are because the tests are not passing the authentication token for an admin user. I will now modify the failing tests to create an admin user and pass the token in the `executeGraphQL` function. \ No newline at end of file +This file is for providing instructions to AI agents. The previous content was a temporary debug log and has been cleared. Follow the instructions in `refactor.md` and the consolidated tasks in `TASKS.md`. \ No newline at end of file diff --git a/BUILD_ISSUES.md b/BUILD_ISSUES.md deleted file mode 100644 index 7f1aa8a..0000000 --- a/BUILD_ISSUES.md +++ /dev/null @@ -1,14 +0,0 @@ -# Build Issues - -This document tracks the build errors encountered during the refactoring process. - -- [ ] `internal/adapters/graphql/schema.resolvers.go:10:2: "log" imported and not used` -- [ ] `internal/adapters/graphql/schema.resolvers.go:1071:24: r.App.AuthorRepo undefined (type *app.Application has no field or method AuthorRepo)` -- [ ] `internal/adapters/graphql/schema.resolvers.go:1073:24: r.App.AuthorRepo undefined (type *app.Application has no field or method AuthorRepo)` -- [ ] `internal/adapters/graphql/schema.resolvers.go:1089:36: r.App.Localization.GetAuthorBiography undefined (type *"tercul/internal/app/localization".Service has no field or method GetAuthorBiography)` -- [ ] `internal/adapters/graphql/schema.resolvers.go:1141:22: r.App.UserRepo undefined (type *app.Application has no field or method UserRepo)` -- [ ] `internal/adapters/graphql/schema.resolvers.go:1143:24: r.App.UserRepo undefined (type *app.Application has no field or method UserRepo)` -- [ ] `internal/adapters/graphql/schema.resolvers.go:1212:20: r.App.TagRepo undefined (type *app.Application has no field or method TagRepo)` -- [ ] `internal/adapters/graphql/schema.resolvers.go:1225:32: r.App.TagRepo undefined (type *app.Application has no field or method TagRepo)` -- [ ] `internal/adapters/graphql/schema.resolvers.go:1249:25: r.App.CategoryRepo undefined (type *app.Application has no field or method CategoryRepo)` -- [ ] `internal/adapters/graphql/schema.resolvers.go:1262:32: r.App.CategoryRepo undefined (type *app.Application has no field or method CategoryRepo)` \ No newline at end of file diff --git a/TASKS.md b/TASKS.md new file mode 100644 index 0000000..215ec39 --- /dev/null +++ b/TASKS.md @@ -0,0 +1,75 @@ +# Consolidated Tasks for Tercul + +This document is the single source of truth for all outstanding development tasks. It is generated from `TODO.md`, `BUILD_ISSUES.md`, and inline `TODO` comments. + +--- + +## Urgent: Build Failures + +These issues are currently breaking the build and must be resolved before any other work can proceed. All issues are located in `internal/adapters/graphql/schema.resolvers.go`. + +- [ ] **Resolver Errors:** + - [ ] `internal/adapters/graphql/schema.resolvers.go:10:2: "log" imported and not used` + - [ ] `internal/adapters/graphql/schema.resolvers.go:1071:24: r.App.AuthorRepo undefined (type *app.Application has no field or method AuthorRepo)` + - [ ] `internal/adapters/graphql/schema.resolvers.go:1073:24: r.App.AuthorRepo undefined (type *app.Application has no field or method AuthorRepo)` + - [ ] `internal/adapters/graphql/schema.resolvers.go:1089:36: r.App.Localization.GetAuthorBiography undefined (type *"tercul/internal/app/localization".Service has no field or method GetAuthorBiography)` + - [ ] `internal/adapters/graphql/schema.resolvers.go:1141:22: r.App.UserRepo undefined (type *app.Application has no field or method UserRepo)` + - [ ] `internal/adapters/graphql/schema.resolvers.go:1143:24: r.App.UserRepo undefined (type *app.Application has no field or method UserRepo)` + - [ ] `internal/adapters/graphql/schema.resolvers.go:1212:20: r.App.TagRepo undefined (type *app.Application has no field or method TagRepo)` + - [ ] `internal/adapters/graphql/schema.resolvers.go:1225:32: r.App.TagRepo undefined (type *app.Application has no field or method TagRepo)` + - [ ] `internal/adapters/graphql/schema.resolvers.go:1249:25: r.App.CategoryRepo undefined (type *app.Application has no field or method CategoryRepo)` + - [ ] `internal/adapters/graphql/schema.resolvers.go:1262:32: r.App.CategoryRepo undefined (type *app.Application has no field or method CategoryRepo)` + +--- + +## High Priority + +### Architecture & Refactoring +- [ ] **Implement Full Observability:** + - [ ] Centralize logging across the application. + - [ ] Add Prometheus metrics for key operations. + - [ ] Add OpenTelemetry tracing for distributed request tracing. +- [ ] **Establish CI/CD Pipeline:** + - [ ] Add `make lint test test-integration` to the CI pipeline. + - [ ] Set up automated deployments to a staging environment. +- [ ] **Refactor Testing Utilities:** + - [ ] Remove database logic from `internal/testutil/testutil.go` to support mock-based integration tests. + +### Features +- [ ] **Implement Analytics Features:** + - [ ] Implement view counting for works and translations. + - [ ] Implement like counting for works and translations. + - [ ] Implement comment counting for works. + - [ ] Implement bookmark counting for works. + - [ ] Implement translation counting for works. + - [ ] Implement translation analytics to show popular translations. + - *Note: This is referenced in both `TODO.md` and `internal/jobs/linguistics/work_analysis_service.go`.* + +- [ ] **Complete Unfinished Implementations:** + - [ ] `internal/app/work/commands.go`: Implement the `MergeWork` function. + - [ ] `internal/app/search/service.go`: Get content from the translation service. + - [ ] `cmd/tools/enrich/main.go`: The tool is broken and needs to be fixed. + +--- + +## Medium Priority + +### Performance +- [ ] **Batch Weaviate Operations:** Implement batching for Weaviate client operations to improve throughput. +- [ ] **Add Performance Benchmarks:** Create benchmarks for critical API paths and business logic to monitor performance over time. + +### Code Quality & Architecture +- [ ] **Expand Weaviate Client:** Add support for all models to the Weaviate client. +- [ ] **Add Documentation:** Improve code documentation and generate API documentation. +- [ ] **Refactor Caching:** Replace bespoke cached repositories with the decorator pattern in `internal/data/cache`. +- [ ] **Improve Configuration Handling:** Replace ad-hoc config with environment variable parsing and validation (e.g., using `koanf` or `envconfig`). + +### Monitoring & Logging +- [ ] **Add Monitoring:** Add monitoring for background jobs and API endpoints. +- [ ] **Add Linguistics Metrics:** Add metrics for linguistics pipeline (analysis duration, cache hit/miss, provider usage). + +--- + +## Low Priority + +- [ ] **Refactor Transactional Runner:** Refactor `RunTransactional` in `internal/testutil/testutil.go` to be more mock-friendly. \ No newline at end of file diff --git a/TODO.md b/TODO.md deleted file mode 100644 index 629434f..0000000 --- a/TODO.md +++ /dev/null @@ -1,107 +0,0 @@ -# TODO List for Tercul Go Application - ---- - -## Suggested Next Objectives - -- [x] **Complete the Architecture Refactor (High, 5d):** Finalize the transition to a clean, domain-driven architecture. This will significantly improve maintainability, scalability, and developer velocity. - - [x] Ensure resolvers call application services only and add dataloaders per aggregate. - - [x] Adopt a migrations tool and move all SQL to migration files. - - [ ] Implement full observability with centralized logging, metrics, and tracing. -- [x] **Full Test Coverage (High, 5d):** Increase test coverage across the application to ensure stability and prevent regressions. - - [x] Write unit tests for all models, repositories, and services. - - [x] Refactor existing tests to use mocks instead of a real database. -- [ ] **Implement Analytics Features (High, 3d):** Add analytics to provide insights into user engagement and content popularity. - - [ ] Implement view, like, comment, and bookmark counting. - - [ ] Track translation analytics to identify popular translations. -- [ ] **Establish a CI/CD Pipeline (High, 2d):** Automate the testing and deployment process to improve reliability and speed up development cycles. - - [ ] Add `make lint test test-integration` to the CI pipeline. - - [ ] Set up automated deployments to a staging environment. -- [ ] **Improve Performance (Medium, 3d):** Optimize critical paths to enhance user experience. - - [ ] Implement batching for Weaviate operations. - - [ ] Add performance benchmarks for critical paths. - ---- - -## [ ] High Priority - -### [ ] Architecture Refactor (DDD-lite) -- [x] Refactor domains to be more testable and decoupled, with 100% unit test coverage and logging. - - [x] `localization` domain - - [x] `auth` domain - - [x] `copyright` domain - - [x] `monetization` domain - - [x] `search` domain - - [x] `work` domain -- [x] Resolvers call application services only; add dataloaders per aggregate (High, 3d) -- [x] Adopt migrations tool (goose/atlas/migrate); move SQL to `internal/data/migrations`; delete `migrations.go` (High, 2d) -- [ ] Observability: centralize logging; add Prometheus metrics and OpenTelemetry tracing; request IDs (High, 3d) -- [ ] CI: add `make lint test test-integration` and integration tests with Docker compose (High, 2d) - -### [x] Testing -- [x] Add unit tests for all models, repositories, and services (High, 3d) -- [x] Remove DB logic from `BaseSuite` for mock-based integration tests (High, 2d) - -### [ ] Features -- [ ] Implement analytics data collection (High, 3d) - - [ ] Implement view counting for works and translations - - [ ] Implement like counting for works and translations - - [ ] Implement comment counting for works - - [ ] Implement bookmark counting for works - - [ ] Implement translation counting for works - - [ ] Implement translation analytics to show popular translations - ---- - -## [ ] Medium Priority - -### [ ] Performance Improvements -- [ ] Implement batching for Weaviate operations (Medium, 2d) - -### [ ] Code Quality & Architecture -- [ ] Expand Weaviate client to support all models (Medium, 2d) -- [ ] Add code documentation and API docs (Medium, 2d) -- [ ] Replace bespoke cached repositories with decorators in `internal/data/cache` (reads only; deterministic invalidation) (Medium, 2d) -- [ ] Config: replace ad-hoc config with env parsing + validation (e.g., koanf/envconfig); no globals (Medium, 1d) - -### [ ] Testing -- [ ] Add performance benchmarks for critical paths (Medium, 2d) - - [ ] Add benchmarks for text analysis (sequential vs concurrent) and cache hit/miss rates - -### [ ] Monitoring & Logging -- [ ] Add monitoring for background jobs and API endpoints (Medium, 2d) - - [ ] Add metrics for linguistics: analysis duration, cache hit/miss, provider usage - ---- - -## [ ] Low Priority - -### [ ] Testing -- [ ] Refactor `RunTransactional` to be mock-friendly (Low, 1d) - ---- - -## [ ] Completed - -- [x] Add comprehensive input validation for all GraphQL mutations (High, 2d) - *Partially complete. Core mutations are validated.* -- [x] Create skeleton packages: `cmd/`, `internal/platform/`, `internal/domain/`, `internal/app/`, `internal/data/`, `internal/adapters/graphql/`, `internal/jobs/` -- [x] Move infra to `internal/platform/*` (`config`, `db`, `cache`, `auth`, `http`, `log`, `search`) -- [x] Wire DI in `cmd/api/main.go` and expose an `Application` facade to adapters -- [x] Unify GraphQL under `internal/adapters/graphql` and update `gqlgen.yml`; move `schema.graphqls` and resolvers -- [x] Introduce Unit-of-Work: `platform/db.WithTx(ctx, func(ctx) error)` and repo factory for `*sql.DB` / `*sql.Tx` -- [x] Split write vs read paths for `work` (commands.go, queries.go); make read models cacheable -- [x] Restructure `models/*` into domain aggregates with constructors and invariants -- [x] Security: move JWT/middleware to `internal/platform/auth`; add authz policy helpers (e.g., `CanEditWork`) -- [x] Search: move Weaviate client/schema to `internal/platform/search`, optional domain interface -- [x] Background jobs: move to `cmd/worker` and `internal/jobs/*`; ensure idempotency and lease -- [x] Python ops: move scripts to `/ops/migration` and `/ops/analysis`; keep outputs under `/ops/migration/outputs/` -- [x] Cleanup: delete dead packages (`store`, duplicate `repositories`); consolidate to `internal/data/sql` -- [x] Add integration tests for GraphQL API and background jobs (High, 3d) - *Partially complete. Core mutations are tested.* -- [x] Stabilize non-linguistics tests and interfaces (High, 2d) - - [x] Fix `graph` mocks to accept context in service interfaces - - [x] Update `repositories` tests (missing `TestModel`) and align with new repository interfaces - - [x] Update `services` tests to pass context and implement missing repo methods in mocks - ---- - -> TODO items include context, priority, and estimated effort. Update this list after each milestone.