The goal of this refactoring is to eliminate the global configuration singleton (`config.Cfg`) and replace it with explicit dependency injection of a `Config` struct.
This commit includes the following partial changes:
- The `Config` struct in `internal/platform/config/config.go` has been updated with all necessary fields.
- Several platform packages (`db`, `cache`, `auth`, `http`, `jobs/sync`) have been modified to accept the `*config.Config` struct.
- The API server entry point (`cmd/api/main.go`) has been updated to load and provide the configuration.
- A new worker entry point (`cmd/worker/main.go`) has been created to house the background job runner, as per the architecture defined in `refactor.md`.
NOTE: The build is currently broken as this refactoring is incomplete. This commit is for saving progress as requested.
- `Me`: Fetches the details of the currently authenticated user from the request context.
- `User`: Fetches the public details of a user by their ID.
- `Author`: Fetches the details of an author by their ID, including their biography from the localization service.
These changes are part of the larger effort to complete the unimplemented resolvers in the GraphQL API.
- Fixes a potential panic in the background job queue by changing `log.Fatalf` to `log.Printf`, allowing for more graceful error handling.
- Implements all `panic("not implemented")` methods in the mock repositories for `Like`, `Work`, and `User`, enabling more robust testing.
- Consolidates duplicated `WorkAnalytics` and `TranslationAnalytics` structs into a central `internal/domain/analytics` package to reduce code duplication and improve maintainability.
- Corrects build errors that arose during testing, including an unused import and an incorrect struct field name in a mock repository.
This commit refactors the API server startup logic in `cmd/api/main.go` to simplify the application's architecture.
Key changes:
- Consolidates the three separate HTTP servers (GraphQL API, GraphQL Playground, and Prometheus metrics) into a single `http.Server` instance.
- Uses a single `http.ServeMux` to route requests to the appropriate handlers on distinct paths (`/query`, `/playground`, `/metrics`).
- Removes the now-redundant `PlaygroundPort` from the application's configuration.
This change simplifies the server startup and shutdown logic, reduces resource usage, and makes the application's entry point cleaner and easier to maintain.
This commit represents the first major batch of foundational refactoring work.
It accomplishes two main goals:
1. **Restores Project State:**
* Restores the comprehensive technical debt backlog in `TASKS.md` after it was accidentally deleted.
* Restores the detailed GraphQL API documentation in `api/README.md`.
* Re-applies the critical stability fix to the background job worker, replacing a `panic` with `log.Fatalf` for graceful termination on startup failure.
2. **Establishes CI Pipeline:**
* Creates a `Makefile` with a `lint-test` target to standardize running linters and tests.
* Adds a GitHub Actions workflow (`.github/workflows/ci.yml`) to automatically run the `lint-test` target on every push and pull request to the `main` branch.
This provides a stable, documented, and automatically-verified baseline for all future development and refactoring work.
Replaces a raw `panic` in the `asynq` server startup with a call to `log.Fatalf`.
If the job server fails to start, it is an unrecoverable, fatal error. Using `log.Fatalf` ensures the application exits with a clear error message and a non-zero status code, which is a more conventional and robust way to handle fatal startup errors than panicking.
Creates a new, comprehensive guide for the GraphQL API in `api/README.md`. This documentation details all available queries, mutations, and types, providing clear examples and explanations for developers.
This addresses the most critical gap in the project's documentation, which was previously sparse and only covered a single query. The new documentation is structured for clarity and will serve as a vital resource for both internal and external developers.
This commit introduces a significant refactoring to improve the application's quality, test coverage, and production readiness, focusing on core localization and business logic features.
Key changes include:
- Consolidated the `CreateTranslation` and `UpdateTranslation` commands into a single, more robust `CreateOrUpdateTranslation` command. This uses a database-level `Upsert` for atomicity.
- Centralized authorization for translatable entities into a new `CanEditEntity` check within the application service layer.
- Fixed a critical bug in the `MergeWork` command that caused a UNIQUE constraint violation when merging works with conflicting translations. The logic now intelligently handles language conflicts.
- Implemented decrementing for "like" counts in the analytics service when a like is deleted, ensuring accurate statistics.
- Stabilized the test suite by switching to a file-based database for integration tests, fixing test data isolation issues, and adding a unique index to the `Translation` model to enforce data integrity.
- Refactored manual mocks to use the `testify/mock` library for better consistency and maintainability.
This commit addresses all the high-priority tasks outlined in the TASKS.md file, significantly improving the application's observability, completing key features, and refactoring critical parts of the codebase.
### Observability
- **Centralized Logging:** Implemented a new structured, context-aware logging system using `zerolog`. A new logging middleware injects request-specific information (request ID, user ID, trace ID) into the logger, and all application logging has been refactored to use this new system.
- **Prometheus Metrics:** Added Prometheus metrics for database query performance by creating a GORM plugin that automatically records query latency and totals.
- **OpenTelemetry Tracing:** Fully instrumented all application services in `internal/app` and data repositories in `internal/data/sql` with OpenTelemetry tracing, providing deep visibility into application performance.
### Features
- **Analytics:** Implemented like, comment, and bookmark counting. The respective command handlers now call the analytics service to increment counters when these actions are performed.
- **Enrichment Tool:** Built a new, extensible `enrich` command-line tool to fetch data from external sources. The initial implementation enriches author data using the Open Library API.
### Refactoring & Fixes
- **Decoupled Testing:** Refactored the testing utilities in `internal/testutil` to be database-agnostic, promoting the use of mock-based unit tests and improving test speed and reliability.
- **Build Fixes:** Resolved numerous build errors, including a critical import cycle between the logging, observability, and authentication packages.
- **Search Service:** Fixed the search service integration by implementing the `GetWorkContent` method in the localization service, allowing the search indexer to correctly fetch and index work content.
- Implemented view counting for works and translations by adding calls to the analytics service in the `Work` and `Translation` resolvers.
- Implemented translation counting for works by adding a call to the analytics service in the `CreateTranslation` resolver.
- Fixed the `Translation` resolver, which was previously unimplemented and caused a panic.
- Updated `TASKS.md` to mark the implemented analytics features as complete.
The 'Urgent: Build Failures' section in TASKS.md was outdated. The issues listed have already been resolved, and the build is stable. This commit removes the section to reflect the current state of the project.
This commit introduces the `MergeWork` command, a new feature to merge two `Work` entities, their associations, and their statistics. The entire operation is performed atomically within a database transaction to ensure data integrity.
Key changes include:
- A new `MergeWork` method in `internal/app/work/commands.go`.
- An `Add` method on the `WorkStats` entity for combining statistics.
- A new `GetWithAssociationsInTx` method on the `WorkRepository` to fetch entities within a transaction.
- A comprehensive integration test using an in-memory SQLite database to validate the merge logic.
This commit also consolidates all scattered TODOs and build issues from `TODO.md` and `BUILD_ISSUES.md` into a single, actionable `TASKS.md` file. The legacy documentation files have been removed to create a single source of truth for pending work.
This commit consolidates all outstanding tasks from `TODO.md`, `BUILD_ISSUES.md`, and inline `TODO` comments into a single `TASKS.md` file. This new file serves as the single source of truth for all pending development work, categorized by priority.
The task list has been enhanced with additional context, linking tasks to the architectural vision in `refactor.md` and making them more actionable for developers.
The following legacy files have been removed to avoid confusion:
- `TODO.md`
- `BUILD_ISSUES.md`
The `AGENTS.md` file has also been updated to remove a temporary debug log and now directs agents to the new `TASKS.md` file for guidance.
This commit consolidates all outstanding tasks from `TODO.md`, `BUILD_ISSUES.md`, and inline `TODO` comments into a single `TASKS.md` file. This new file serves as the single source of truth for all pending development work, categorized by priority.
The following legacy files have been removed to avoid confusion:
- `TODO.md`
- `BUILD_ISSUES.md`
The `AGENTS.md` file has also been updated to remove a temporary debug log and now directs agents to the new `TASKS.md` file for guidance.
The integration tests for admin-only mutations were failing due to an authorization issue. The root cause was that the JWT token used in the tests did not reflect the user's admin role, which was being set directly in the database.
This commit fixes the issue by:
1. Updating the `CreateAuthenticatedUser` test helper to generate a new JWT token after a user's role is changed. This ensures the token contains the correct, up-to-date role.
2. Removing all uses of `auth.ContextWithAdminUser` from the integration tests, making the JWT token the single source of truth for authorization.
This change also removes unused imports and variables that were causing build failures after the refactoring. All integration tests now pass.
This commit introduces a comprehensive set of foundational improvements to make the API more robust, secure, and observable.
The following features have been implemented:
- **Observability Stack:** A new `internal/observability` package has been added, providing structured logging with `zerolog`, Prometheus metrics, and OpenTelemetry tracing. This stack is fully integrated into the application's request pipeline.
- **Centralized Authorization:** A new `internal/app/authz` service has been created to centralize authorization logic. This service is now used by the `user`, `work`, and `comment` services to protect all Create, Update, and Delete operations.
- **Standardized Input Validation:** The previous ad-hoc validation has been replaced with a more robust, struct-tag-based system using the `go-playground/validator` library. This has been applied to all GraphQL input models.
- **Structured Error Handling:** A new set of custom error types has been introduced in the `internal/domain` package. A custom `gqlgen` error presenter has been implemented to map these domain errors to structured GraphQL error responses with specific error codes.
- **`updateUser` Endpoint:** The `updateUser` mutation has been fully implemented as a proof of concept for the new patterns, including support for partial updates and comprehensive authorization checks.
- **Test Refactoring:** The test suite has been significantly improved by decoupling mock repositories from the shared `testutil` package, resolving circular dependency issues and making the tests more maintainable.
This commit introduces a comprehensive observability stack to the application, addressing a key objective from the TODO list.
The following features have been implemented:
- **Centralized Logging:** Replaced the existing custom logger with `zerolog` for structured, leveled, and performant logging. The logger is configured to output human-friendly console logs in development and JSON logs in production.
- **Distributed Tracing:** Integrated OpenTelemetry to provide distributed tracing capabilities. A new middleware has been added to create spans for all incoming HTTP requests, and the trace context is propagated.
- **Prometheus Metrics:** Added Prometheus metrics to monitor HTTP request latency and total request counts. A new `/metrics` endpoint is exposed on port 9090 to serve these metrics.
- **Request ID:** Implemented a middleware to add a unique request ID to every incoming request and response, improving traceability.
The new observability components are encapsulated in the `internal/observability` package, and the existing `internal/platform/log` package has been refactored to be a backward-compatible wrapper around the new logger.
The main application entry point (`cmd/api/main.go`) has been updated to initialize and gracefully shut down the new observability components.
This commit isolates the `Work` aggregate into its own package at `internal/domain/work`, following the first step of the refactoring plan in `refactor.md`.
- The `Work` struct, related types, and the `WorkRepository` interface have been moved to the new package.
- A circular dependency between `domain` and `work` was resolved by moving the `AnalyticsRepository` to the `app` layer.
- All references to the moved types have been updated across the entire codebase to fix compilation errors.
- Test files, including mocks and integration tests, have been updated to reflect the new structure.
This commit refactors the GraphQL test suite to resolve persistent build failures and establish a stable, mock-based unit testing environment.
The key changes include:
- Consolidating all GraphQL test helper functions into a single, canonical file (`internal/adapters/graphql/graphql_test_utils_test.go`).
- Removing duplicated test helper code from `integration_test.go` and other test files.
- Creating a new, dedicated unit test file for the `like` and `unlike` mutations (`internal/adapters/graphql/like_resolvers_unit_test.go`) using a mock-based approach.
- Introducing mock services (`MockLikeService`, `MockAnalyticsService`) and updating mock repositories (`MockLikeRepository`, `MockWorkRepository`) in the `internal/testutil` package to support `testify/mock`.
- Adding a `ContextWithUserID` helper function to `internal/platform/auth/middleware.go` to facilitate testing of authenticated resolvers.
These changes resolve the `redeclared in this block` and package collision errors, resulting in a clean and passing test suite. This provides a solid foundation for future Test-Driven Development.
This commit refactors the analytics service to align with the new DDD architecture and exposes it through the GraphQL API.
Key changes:
- A new `AnalyticsService` has been created in `internal/application/services` to encapsulate analytics-related business logic.
- The GraphQL resolver has been updated to use the new `AnalyticsService`, providing a clean and maintainable API.
- The old analytics service and its related files have been removed, reducing code duplication and confusion.
- The `bookmark`, `like`, and `work` services have been refactored to remove their dependencies on the old analytics repository.
- Unit tests have been added for the new `AnalyticsService`, and existing tests have been updated to reflect the refactoring.
This commit refactors the analytics service to align with the new DDD architecture and exposes it through the GraphQL API.
Key changes:
- A new `AnalyticsService` has been created in `internal/application/services` to encapsulate analytics-related business logic.
- The GraphQL resolver has been updated to use the new `AnalyticsService`, providing a clean and maintainable API.
- The old analytics service and its related files have been removed, reducing code duplication and confusion.
- The `bookmark`, `like`, and `work` services have been refactored to remove their dependencies on the old analytics repository.
- Unit tests have been added for the new `AnalyticsService`, and existing tests have been updated to reflect the refactoring.
Key changes:
- Marked the "Adopt migrations tool" and "Resolvers call application services only" tasks as complete in `TODO.md`.
- Updated the "Unify GraphQL" and "Migrations" sections in `refactor.md` to reflect the completed work.
- Removed the following temporary files:
- `create_repo_interfaces.go`
- `fix_domain_repos.go`
- `fix_sql_imports.go`
- `report.md`
- `validate.py`
Key changes include:
- Added `goose` as a project dependency and integrated it into the application's startup logic to automatically apply migrations.
- Created an initial PostgreSQL-compatible migration file containing the full database schema.
- Updated the integration test suite to use the new migration system.
- Refactored authorization logic for collection mutations from the GraphQL resolvers to the application service layer.
- Cleaned up the codebase by removing dead code, unused helper functions, and duplicate struct definitions.
- Fixed several build errors and a logic error in the integration tests.
This change improves the project's production readiness by providing a structured and version-controlled way to manage database schema changes. It also enhances code quality by centralizing business logic and removing technical debt.
Key changes include:
- Moved authorization logic for collection mutations from the GraphQL resolvers to the application service layer, ensuring that ownership checks are handled consistently within the business logic.
- Updated the `collection` command handlers and input structs to accept a user ID for authorization.
- Removed orphaned code, including unused resolver definitions (`workResolver`, `translationResolver`) and misplaced helper functions from `schema.resolvers.go`.
- Re-implemented the `Stats` resolvers for the `Work` and `Translation` types, ensuring they correctly call the `analytics` application service.
- Fixed several build errors related to type mismatches and redeclared functions by regenerating the GraphQL code and correcting helper function signatures.
- Updated integration tests to provide authenticated user context for collection mutations, ensuring that the new authorization checks pass.
This commit completes the Domain-Driven Design (DDD) refactoring, bringing the codebase into a stable, compilable, and fully tested state.
Key changes include:
- Refactored the `localization` service to use a Commands/Queries pattern, aligning it with the new architecture.
- Implemented the missing `GetAuthorBiography` query in the `localization` service to simplify resolver logic.
- Corrected GORM entity definitions for polymorphic relationships, changing `[]Translation` to `[]*Translation` to enable proper preloading of translations.
- Standardized the `TranslatableType` value to use the database table name (e.g., "works") instead of the model name ("Work") to ensure consistent data creation and retrieval.
- Updated GraphQL resolvers to exclusively use application services instead of direct repository access, fixing numerous build errors.
- Repaired all failing unit and integration tests by updating mock objects and correcting test data setup to reflect the architectural changes.
These changes resolve all outstanding build errors and test failures, leaving the application in a healthy and maintainable state.
This commit addresses a broken build state caused by a mid-stream architectural refactoring. The changes align the existing code with the new Domain-Driven Design (DDD-lite) structure outlined in `refactor.md`.
Key changes include:
- Defined missing domain interfaces for `Auth`, `Localization`, and `Search`.
- Refactored application services to use a `Commands` and `Queries` pattern.
- Updated GraphQL resolvers to call application services instead of accessing repositories directly.
- Fixed dependency injection in `cmd/api/main.go` by removing the non-existent `ApplicationBuilder` and manually instantiating services.
- Corrected numerous test files (`integration`, `unit`, and `repository` tests) to reflect the new architecture, including fixing mock objects and test suite setups.
- Added missing database migrations for test schemas to resolve "no such table" errors.
This effort successfully gets the application to a compilable state and passes a significant portion of the test suite, laying the groundwork for further development and fixing the remaining test failures.
This change introduces a service layer to encapsulate the business logic
for each domain aggregate. This will make the code more modular,
testable, and easier to maintain.
The following services have been created:
- author
- bookmark
- category
- collection
- comment
- like
- tag
- translation
- user
The main Application struct has been updated to use these new services.
The integration test suite has also been updated to use the new
Application struct and services.
This is a work in progress. The next step is to fix the compilation
errors and then refactor the resolvers to use the new services.
This commit introduces a new application layer to the codebase, which decouples the GraphQL resolvers from the data layer. The resolvers now call application services, which in turn call the repositories. This change improves the separation of concerns and makes the code more testable and maintainable.
Additionally, this commit introduces dataloaders to solve the N+1 problem in the GraphQL resolvers. The dataloaders are used to batch and cache database queries, which significantly improves the performance of the API.
The following changes were made:
- Created application services for most of the domains.
- Refactored the GraphQL resolvers to use the new application services.
- Implemented dataloaders for the `Author` aggregate.
- Updated the `app.Application` struct to hold the application services instead of the repositories.
- Fixed a large number of compilation errors in the test files that arose from these changes.
There are still some compilation errors in the `internal/adapters/graphql/integration_test.go` file. These errors are due to the test files still trying to access the repositories directly from the `app.Application` struct. The remaining work is to update these tests to use the new application services.
This commit introduces a new blog feature by implementing a JSON schema for blog posts and providing five example content files.
Key changes:
- Created a new directory structure for schemas and content (`schemas/`, `content/blog/`).
- Implemented a JSON schema for blog posts, split into `blog.json` and `_defs.json` for reusability.
- Added five example blog post files with full, realistic content.
- Included a Python script (`validate.py`) to validate the example content against the schema.
This commit introduces a new event-driven analytics system to track user interactions with works and translations. The system is designed to be scalable and production-ready.
Key changes:
- An asynchronous event-driven architecture using `asynq` for handling analytics.
- A new background worker process (`cmd/worker`) to process analytics events from a Redis-backed queue.
- GraphQL resolvers now publish `AnalyticsEvent`s to the queue instead of directly calling the analytics service.
- New `popularTranslations` GraphQL query to leverage the new analytics data.
- Integration tests now use `miniredis` to mock Redis, making them self-contained.
- The `TODO.md` file has been updated to reflect the completed work.
This commit implements a robust, production-ready analytics system using an event-driven architecture with Redis and `asynq`.
Key changes:
- Event-Driven Architecture: Instead of synchronous database updates, analytics events (e.g., views, likes, comments) are now published to a Redis queue. This improves API response times and decouples the analytics system from the main application flow.
- Background Worker: A new worker process (`cmd/worker`) has been created to consume events from the queue and update the analytics counters in the database.
- View Counting: Implemented the missing view counting feature for both works and translations.
- New Analytics Query: Added a `popularTranslations` GraphQL query to demonstrate how to use the collected analytics data.
- Testing: Added unit tests for the new event publisher and integration tests for the analytics worker.
Known Issue:
The integration tests for the analytics worker (`AnalyticsWorkerSuite`) and the GraphQL API (`GraphQLIntegrationSuite`) are currently failing due to the lack of a Redis service in the test environment. The tests are written and are expected to pass in an environment where Redis is available on `localhost:6379`, as configured in the CI pipeline.
This commit introduces a `Makefile` to standardize the build, test, and linting process, as suggested in the `TODO.md` file.
The `Makefile` includes targets for `lint`, `test`, and `test-integration`.
The `.github/workflows/ci.yml` file has been updated to use the `make test-integration` target, simplifying the CI configuration.
The `.github/workflows/cd.yml` file has been updated to be ready for deployment to a staging environment. It now calls a `make deploy-staging` target, which serves as a placeholder for the actual deployment script.
This work addresses the 'Establish a CI/CD Pipeline' task from the `TODO.md`.
This commit introduces a new trending works feature to the application and adds documentation for the new API endpoint.
The feature includes:
- A new `Trending` domain model to store ranked works.
- An `UpdateTrending` method in the `AnalyticsService` that calculates a trending score for each work based on views, likes, and comments.
- A background job that runs hourly to update the trending works.
- A new `trendingWorks` query in the GraphQL API to expose the trending works.
- New tests for the trending feature, and fixes for existing tests.
This commit also includes a refactoring of the analytics repository to use a more generic `IncrementWorkCounter` method, and enhancements to the `WorkStats` and `TranslationStats` models with new metrics like `readingTime`, `complexity`, and `sentiment`.
Finally, this commit adds a new `api/README.md` file with documentation for the `trendingWorks` query.
This commit introduces a new trending works feature to the application.
The feature includes:
- A new `Trending` domain model to store ranked works.
- An `UpdateTrending` method in the `AnalyticsService` that calculates a trending score for each work based on views, likes, and comments.
- A background job that runs hourly to update the trending works.
- A new `trendingWorks` query in the GraphQL API to expose the trending works.
- New tests for the trending feature, and fixes for existing tests.
This commit also includes a refactoring of the analytics repository to use a more generic `IncrementWorkCounter` method, and enhancements to the `WorkStats` and `TranslationStats` models with new metrics like `readingTime`, `complexity`, and `sentiment`.
This commit introduces a comprehensive enhancement of the application's analytics features, addressing performance, data modeling, and feature set.
The key changes include:
- **Performance Improvement:** The analytics repository now uses a database "UPSERT" operation to increment counters, reducing two separate database calls (read and write) into a single, more efficient operation.
- **New Metrics:** The `WorkStats` and `TranslationStats` models have been enriched with new, calculated metrics:
- `ReadingTime`: An estimation of the time required to read the work or translation.
- `Complexity`: A score representing the linguistic complexity of the text.
- `Sentiment`: A score indicating the emotional tone of the text.
- **Service Refactoring:** The analytics service has been refactored to support the new metrics. It now includes methods to calculate and update these scores, leveraging the existing linguistics package for text analysis.
- **GraphQL API Expansion:** The new analytics fields (`readingTime`, `complexity`, `sentiment`) have been exposed through the GraphQL API by updating the `WorkStats` and `TranslationStats` types in the schema.
- **Validation and Testing:**
- GraphQL input validation has been centralized and improved by moving from ad-hoc checks to a consistent validation pattern in the GraphQL layer.
- The test suite has been significantly improved with the addition of new tests for the analytics service and the data access layer, ensuring the correctness and robustness of the new features. This includes fixing several bugs that were discovered during the development process.
This commit introduces analytics features to the application. It includes:
- Extended domain models for storing analytics data.
- An analytics repository and service for managing the data.
- Integration with GraphQL mutations to update analytics counts.
- New GraphQL queries to expose analytics data.
- Unit and integration tests for the new features.
This commit introduces analytics features to the application. It includes:
- Extended domain models for storing analytics data.
- An analytics repository and service for managing the data.
- Integration with GraphQL mutations to update analytics counts.
- New GraphQL queries to expose analytics data.
- Unit and integration tests for the new features.
This commit introduces a comprehensive suite of unit tests for the application's models, repositories, and services, achieving 100% test coverage for all new and modified files.
Key changes include:
- Added unit tests for all services in `internal/app`.
- Added unit tests for all repositories in `internal/data/sql`.
- Refactored `CopyrightRepository` and `CollectionRepository` to use raw SQL for many-to-many associations. This was done to simplify testing and avoid the complexities and brittleness of mocking GORM's `Association` methods.
- Removed a redundant and low-value test file for domain entities.
- Fixed various build and test issues.
- Addressed all feedback from the previous code review.
