690514:2019 204-rfa-approval-refactor #01
This commit is contained in:
@@ -0,0 +1,110 @@
|
||||
# Tasks: Unified AI Architecture
|
||||
|
||||
**Input**: Design documents from `/specs/300-others/301-unified-ai-arch/`
|
||||
**Prerequisites**: plan.md, spec.md, research.md, data-model.md, contracts/openapi.yaml
|
||||
**Tests**: Tests are OPTIONAL for this implementation phase unless specifically requested during PR review.
|
||||
|
||||
## Phase 1: Setup (Shared Infrastructure)
|
||||
|
||||
**Purpose**: Project initialization and basic structure
|
||||
|
||||
- [ ] T001 Initialize `AiModule` inside `backend/src/ai/ai.module.ts`
|
||||
- [ ] T002 [P] Install `qdrant-js` client dependency in the backend workspace
|
||||
- [ ] T003 Add `AI_HOST_URL`, `AI_QDRANT_URL`, `AI_N8N_SERVICE_TOKEN` to backend `.env` configuration
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Foundational (Blocking Prerequisites)
|
||||
|
||||
**Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented
|
||||
**⚠️ CRITICAL**: No user story work can begin until this phase is complete
|
||||
|
||||
- [ ] T004 Setup `QdrantService` in `backend/src/ai/qdrant.service.ts` to manage vector DB connections
|
||||
- [ ] T005 [P] Setup BullMQ infrastructure in `AiModule` (configure `AiQueueService`)
|
||||
- [ ] T006 [P] Implement `ServiceAccountGuard` to validate n8n service tokens for internal API routes
|
||||
- [ ] T007 Implement SQL Schema Deltas for `migration_review_queue` and `ai_audit_logs` in MariaDB
|
||||
- [ ] T008 Implement TypeORM base entities mapping to the created SQL tables
|
||||
|
||||
**Checkpoint**: Foundation ready - user story implementation can now begin
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: User Story 1 - Legacy Document Migration and Review (Priority: P1) 🎯 MVP
|
||||
|
||||
**Goal**: Process legacy PDFs through an AI pipeline and review extracted metadata in a staging queue before DB commit.
|
||||
**Independent Test**: Upload PDF batch via n8n/endpoint, verify they appear in the UI queue, and approve them successfully.
|
||||
|
||||
### Implementation for User Story 1
|
||||
|
||||
- [ ] T009 [P] [US1] Create `MigrationReviewRecord` TypeORM Entity in `backend/src/ai/entities/migration-review.entity.ts`
|
||||
- [ ] T010 [US1] Implement `AiIngestService` to handle batch ingestion and queue creation
|
||||
- [ ] T011 [US1] Implement `POST /api/ai/legacy-migration/ingest` in `AiController` using `ServiceAccountGuard`
|
||||
- [ ] T011b [P] [US1] Export n8n workflow definition to `backend/src/ai/workflows/folder-watcher.json` to monitor the network directory and POST to the ingest API (FR-001b)
|
||||
- [ ] T012 [US1] Implement `GET /api/ai/legacy-migration/queue` in `AiController`
|
||||
- [ ] T013 [US1] Implement `POST /api/ai/legacy-migration/queue/{publicId}/approve` with Zod/class-validator payload checking (FR-007)
|
||||
- [ ] T014 [P] [US1] Create Frontend API hooks for staging queue in `frontend/src/lib/api/ai.ts`
|
||||
- [ ] T015 [US1] Build Frontend Staging Queue Table UI in `frontend/src/app/(dashboard)/ai-staging/page.tsx`
|
||||
- [ ] T016 [US1] Implement UI Form dropdown constraints for master data fields in the approval modal (FR-012)
|
||||
- [ ] T017 [US1] Build `AiStatusBanner.tsx` component in `frontend/src/components/ai/AiStatusBanner.tsx` to handle offline graceful degradation
|
||||
|
||||
**Checkpoint**: At this point, User Story 1 should be fully functional.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: User Story 2 - RAG Conversational Q&A (Priority: P2)
|
||||
|
||||
**Goal**: Ask natural language questions about project documents with context-aware RAG answers.
|
||||
**Independent Test**: Submit a RAG query for a specific project; verify response < 10s and accurate isolation.
|
||||
|
||||
### Implementation for User Story 2
|
||||
|
||||
- [ ] T018 [P] [US2] Create BullMQ Processor `rag.processor.ts` with strict concurrency limit = 1 (FR-009)
|
||||
- [ ] T019 [US2] Implement `AiRagService` containing Ollama LLM integration logic
|
||||
- [ ] T020 [US2] Enforce `projectPublicId` filtering natively in Qdrant search payload inside `AiRagService`
|
||||
- [ ] T021 [US2] Implement `POST /api/ai/rag/query` to push jobs to BullMQ and apply rate limiting (5 per min) (FR-010)
|
||||
- [ ] T022 [US2] Add AbortController logic to backend processor to cancel LLM generation on client disconnect (FR-011)
|
||||
- [ ] T023 [P] [US2] Build `RagChatWidget.tsx` component with streaming/polling UI for queue wait status
|
||||
|
||||
**Checkpoint**: RAG capability is fully implemented and throttled safely.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: User Story 3 - AI Audit Log Management (Priority: P3)
|
||||
|
||||
**Goal**: View and manage AI audit logs for model feedback, with safe deletion capabilities.
|
||||
**Independent Test**: Generate AI suggestions, verify logs exist, and test hard delete as a System Admin.
|
||||
|
||||
### Implementation for User Story 3
|
||||
|
||||
- [ ] T024 [P] [US3] Create `AiAuditLog` TypeORM Entity in `backend/src/ai/entities/ai-audit-log.entity.ts`
|
||||
- [ ] T025 [US3] Inject Audit Log creation logic into the `/approve` endpoint (capture Human vs AI differences)
|
||||
- [ ] T026 [US3] Implement `DELETE /api/ai/audit-logs` endpoint with `@UseGuards(CaslAbilityGuard)` checking for `SYSTEM_ADMIN`
|
||||
- [ ] T027 [US3] Create BullMQ Processor `vector-deletion.processor.ts` to handle asynchronous vector cleanup (FR-008)
|
||||
- [ ] T028 [US3] Integrate `vector-deletion-queue` dispatch into the main Document Deletion service
|
||||
|
||||
**Checkpoint**: AI Audit and safe vector cleanup are complete.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Polish & Cross-Cutting Concerns
|
||||
|
||||
**Purpose**: Improvements that affect multiple user stories
|
||||
|
||||
- [ ] T029 Code cleanup and CASL RBAC matrix review for all AI endpoints
|
||||
- [ ] T030 E2E Validation of the BullMQ concurrency limit (stress test 10 concurrent requests)
|
||||
- [ ] T031 Finalize `README.md` and `quickstart.md` documentation for Desk-5439 setup
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & Execution Order
|
||||
|
||||
### Phase Dependencies
|
||||
- **Setup (Phase 1)**: Start immediately
|
||||
- **Foundational (Phase 2)**: Depends on Phase 1 - BLOCKS US1, US2, US3
|
||||
- **User Stories (Phases 3-5)**: Depend on Phase 2. Should be executed sequentially (US1 -> US2 -> US3) due to shared services, but frontend/backend tasks within each story can run in parallel.
|
||||
- **Polish (Phase 6)**: Depends on all stories completing.
|
||||
|
||||
### Parallel Opportunities
|
||||
- Database schema changes (T007) and Backend Auth Setup (T006)
|
||||
- Frontend UI components (T015, T017, T023) can be stubbed concurrently with backend API creation.
|
||||
- Entity creation (T009, T024) and BullMQ Processor setup (T018)
|
||||
Reference in New Issue
Block a user