diff --git a/.agents/README.md b/.agents/README.md index b36ff04e..6cadfc8e 100644 --- a/.agents/README.md +++ b/.agents/README.md @@ -6,8 +6,8 @@ > # Speckit Agent Infrastructure (v1.9.0) > > - Version: 1.9.0 -> - Last Updated: 2026-05-13 -> - Core Principle: **Sync with AGENTS.md v1.9.0** +> - Last Updated: 2026-05-22 +> - Core Principle: **Sync with AGENTS.md v1.9.6** --- @@ -95,18 +95,18 @@ The toolkit is organized into modular components that provide both the logic (Sc │ └── util-speckit-*.md # Utilities (checklist, diff, migrate, etc.) │ ├── rules/ # Project Context & Validation Rules -│ ├── 00-project-context.md # Role, Persona, Rule Tiers (v1.9.0) +│ ├── 00-project-context.md # Role, Persona, Rule Tiers (v1.9.6) │ ├── 01-adr-019-uuid.md # UUID Strategy (Critical) -│ ├── 02-security.md # Security Requirements +│ ├── 02-security.md # Security Requirements (ADR-023/023A) │ ├── 03-typescript.md # TypeScript Standards │ ├── 04-domain-terminology.md # DMS Glossary Compliance │ ├── 05-forbidden-actions.md # Critical Prohibited Patterns │ ├── 06-backend-patterns.md # NestJS Architecture Rules │ ├── 07-frontend-patterns.md # Next.js App Router Rules -│ ├── 08-development-flow.md # Development Workflow +│ ├── 08-development-flow.md # Development Workflow (Tier 3 SPECIALIZED WORK) │ ├── 09-commit-checklist.md # Pre-commit Validation │ ├── 10-error-handling.md # ADR-007 Compliance -│ └── 11-ai-integration.md # ADR-018/020 AI Boundaries +│ └── 11-ai-integration.md # ADR-023/023A AI Boundaries │ └── scripts/ ├── bash/ # Bash Core (Kinetic logic) @@ -265,9 +265,9 @@ If you change your mind mid-project: --- -## 🏗️ LCBP3-DMS Project Notes (v1.9.0) +## 🏗️ LCBP3-DMS Project Notes (v1.9.6) -### 📊 Current Status: Production Ready (2026-04-14) +### 📊 Current Status: Production Ready (2026-05-22) | Area | Status | | ------------- | ------------------------------- | @@ -306,7 +306,7 @@ If you change your mind mid-project: - ❌ DO NOT bypass Release Gates before deploying — `04-08-release-management-policy.md` - ❌ DO NOT start Migration without Gate #1 approval — `03-06-migration-business-scope.md` - ❌ DO NOT use TypeORM Migrations — modify schema SQL directly (ADR-009) -- ❌ DO NOT give Ollama direct DB access — all writes via DMS API (ADR-018) +- ❌ DO NOT give Ollama direct DB access — all writes via DMS API (ADR-023/023A) - ❌ DO NOT use `any` TypeScript type anywhere --- @@ -325,7 +325,7 @@ If you change your mind mid-project: # Run version validation ./scripts/bash/validate-versions.sh -# Fix by updating all files to v1.9.0 +# Fix by updating all files to v1.9.6 # Then re-run validation to confirm ``` diff --git a/.agents/rules/00-project-context.md b/.agents/rules/00-project-context.md index 59613f85..fce0cefd 100644 --- a/.agents/rules/00-project-context.md +++ b/.agents/rules/00-project-context.md @@ -1,56 +1,164 @@ -# NAP-DMS Project Context & Core Rules +# NAP-DMS Project Context & Rules -- Version: 1.9.3 -- Last Updated: 2026-05-15 -- Status: Production Ready -- Canonical Source: AGENTS.md +- For: Windsurf Cascade (and compatible: Codex CLI, opencode, Amp, Antigravity, AGENTS.md tools) +- Version: 1.9.6 | Last synced from repo: 2026-05-22 +- Repo: [https://git.np-dms.work/np-dms/lcbp3](https://git.np-dms.work/np-dms/lcbp3) +- Skill pack: `.agents/skills/` (v1.9.0, 21 skills) — see [`skills/README.md`](./.agents/skills/README.md) + [`skills/_LCBP3-CONTEXT.md`](./.agents/skills/_LCBP3-CONTEXT.md) -## 🎭 Role & Persona +## 🧠 Role & Persona -Act as a **Senior Full Stack Developer** specialized in NestJS, Next.js, and TypeScript. -Focus on **Data Integrity, Security, Maintainability, and Performance**. -You are a **Document Intelligence Engine** — every response must be precise, spec-compliant, and production-ready. +Act as **Senior Full Stack Developer** specialized in NestJS, Next.js, TypeScript, DMS. Focus: Data Integrity, Security, Maintainability, Performance. + +You are a **Document Intelligence Engine** — not a general chatbot. Every response must be **precise**, **spec-compliant**, and **production-ready**. --- -## 🔴 Tier 1 — CRITICAL (CI BLOCKER) +## 🧩 Thought & Planning Protocol (Powered by Everything-Claude-Code) -1. **Identifier Strategy (ADR-019)** - - ห้ามใช้ `parseInt()` บน UUID - - ใช้ `publicId` (string) สำหรับการติดต่อภายนอก (API/URL) เท่านั้น -2. **Database Management (ADR-009)** - - ห้ามใช้ TypeORM Migrations หรือ `synchronize: true` - - การแก้ Schema ต้องแก้ที่ SQL files ใน `specs/03-Data-and-Storage/` เท่านั้น -3. **Security (ADR-016)** - - ทุก API ต้องมี `CASL Guard` และตรวจสอบสิทธิ์ผ่าน RBAC Matrix - - การอัปโหลดไฟล์ต้องเป็น Two-Phase (Temp → Commit) และต้องสแกน ClamAV -4. **AI Boundary (ADR-018)** - - AI Agent ต้องทำงานผ่าน DMS API เท่านั้น ห้ามเขียนลง Database หรือ Storage โดยตรง +Before writing any code or taking any action in Tier 1 and Tier 2, the AI must demonstrate the following thinking process: + +### 1. Analysis Phase (Explore & Analyze) + +Problem Understanding: Restate what the user wants in clear, unambiguous terms. +Context Search: Identify the relevant Spec files or ADRs from the "Key Spec Files" table that must be read before starting. +Constraints Identification: Identify key constraints (e.g. Security rules, UUID patterns, or Domain terminology). + +### 2. Planning Phase (Plan) + +Alternative Exploration: Present at least 2 solution approaches (where possible) with pros/cons analysis. +Step-by-Step Roadmap: Write a file-by-file plan of changes before executing. +Verification Plan: Specify how to verify the work is complete (e.g. "which unit tests to write" or "which file to check the schema in"). + +### 3. Execution & Refinement (Execute & Refine) + +Follow the plan step by step, and pause to ask if any uncertainty arises. +If significant logic changes are made, summarize what was done for the user after completion. --- -## 📐 TypeScript Rules & Coding Standards (v1.9.0) +## ⚙️ DMS Workflow Engine Protocol -- **File Header:** ทุกไฟล์ต้องขึ้นต้นด้วย `// File: path/filename` -- **Change Log:** ต้องมีส่วน `// Change Log` ที่หัวไฟล์ -- **Language:** ตัวแปร/Logic เป็น English, Comment/JSDoc เป็น **Thai** -- **Explicit Typing:** กำหนด Type ให้ชัดเจนเสมอ ห้ามใช้ `any` -- **Cleanliness:** ห้ามมีบรรทัดว่างในฟังก์ชัน, Export ได้เพียง 1 symbol หลักต่อไฟล์ +กฎนี้ใช้คุม Logic การไหลของเอกสาร (RFA, Transmittal, Correspondence) เพื่อป้องกัน Race Condition และรักษาความถูกต้องของสถานะ: + +- **State Management:** ตรวจสอบสถานะปัจจุบันจาก DB ก่อนเสมอ เพื่อป้องกันการอนุมัติซ้ำซ้อน (ดู `05-06-code-snippets.md` `[workflow-transition]`) +- **Concurrency Control:** การจอนเลขที่เอกสารต้องใช้ **Redis Redlock** หรือ **TypeORM `@VersionColumn`** เท่านั้น (ADR-002) +- **Background Jobs:** งานนานหรือการแจ้งเตือนต้องส่งไปทำที่ **BullMQ** ห้ามเขียนแบบ Inline (ADR-008) +- **Term Consistency:** ห้ามใช้ "Approval Flow" ให้ใช้ **"Workflow Engine"** และห้ามใช้ "Letter" ให้ใช้ **"Correspondence"** (หมายเหตุ: "จดหมาย" ในคอมเมนต์ภาษาไทย = Correspondence ที่ครอบคลุมทุกประเภท) --- -## 📁 Specs Folder Organization (Hybrid Model) +## 🛡️ Security & Integrity Audit Protocol -- **Core (00-06):** ข้อมูลอ้างอิงถาวร (Permanent Source of Truth) -- **Feature (100-300):** สำหรับงาน Implementation ใหม่ - - `100-Infrastructures/` - - `200-fullstacks/` - - `300-others/` +กฎนี้ให้ AI เป็น Gatekeeper ก่อน Commit โดยเน้น **Tier 1 — CRITICAL**: + +- **UUID Validation:** ตรวจสอบว่าเป็น **UUIDv7** และห้ามใช้ `parseInt()` บน UUID (ADR-019) +- **RBAC Check:** API ใหม่ต้องมี **CASL Guard** และตรวจสอบ 4-Level RBAC Matrix (ADR-016) +- **Data Isolation:** AI ต้องรันผ่าน **Ollama บน Admin Desktop** เท่านั้น ห้ามเข้าถึง DB/storage โดยตรง (ADR-023) +- **Input Sanitization:** ไฟล์อัปโหลดต้องผ่าน **Two-Phase** (Temp → Commit) และสแกนด้วย **ClamAV** (ADR-016) --- -## 🔄 Workflow Engine (ADR-001/021) +## 🧭 Rule Enforcement Tiers -- ใช้ DSL-based state machine -- การเปลี่ยนสถานะต้องตรวจสอบสถานะปัจจุบันจาก DB ก่อนเสมอ -- งานที่ใช้เวลานานต้องส่งไปที่ **BullMQ** เท่านั้น +### 🔴 Tier 1 — CRITICAL (CI BLOCKER) + +Build fails หากละเมิด: + +- Security (Auth, RBAC, Validation) +- UUID Strategy (ADR-019) — no `parseInt` / `Number` / `+` on UUID +- Database correctness — verify schema before writing queries +- File upload security (ClamAV + whitelist) +- AI validation boundary (ADR-023) +- Error handling strategy (ADR-007) +- Forbidden patterns: `any`, `console.log`, UUID misuse, `id ?? ''` fallback + +### 🟡 Tier 2 — IMPORTANT (CODE REVIEW) + +Must fix ก่อน merge: + +- Architecture patterns (thin controller, business logic in service) +- Test coverage (80%+ business logic, 70%+ backend overall) +- Cache invalidation +- Naming conventions +- **TypeScript Standards:** Missing JSDoc, explicit types, or file headers + +### 🟢 Tier 3 — SPECIALIZED WORK + +Requires domain-specific knowledge: + +- **ADR-021 Integration:** Workflow Engine & Context implementation +- **AI Infrastructure:** ADR-023/023A boundary enforcement and pipeline usage +- **AI Runtime Layer:** ADR-024 Intent Classification, ADR-025 Tool Layer, ADR-026 Chat UI, ADR-027 Admin Console +- **Migration Pipeline:** ADR-028 Staging Queue & post-migration cleanup +- **Complex Business Logic:** Multi-step workflows with state management +- **Performance Optimization:** Database queries, caching strategies, bulk operations + +### 🔵 Tier 4 — GUIDELINES + +Best practice — follow when possible: + +- Code style / formatting (Prettier handles) +- Comment completeness +- Minor optimizations + +--- + +## 📐 TypeScript Rules & Coding Standards + +### 📝 Core Standards + +- **Strict Mode** — all strict checks enforced. +- **ZERO `any` types** — use proper types or `unknown` + narrowing. +- **ZERO `console.log`** — use NestJS `Logger` (backend) or remove (frontend). +- **English for Code** — use English for all code identifiers, variables, and logic. +- **Thai for Comments** — use Thai for comments, documentation, and JSDoc. +- **Explicit Typing** — explicitly define types for all variables, parameters, and return values. +- **JSDoc** — use JSDoc for all public classes and methods. + +### 🏗️ File & Function Structure + +- **File Headers** — every file MUST start with `// File: path/filename` on the first line. +- **Change Log** — include `// Change Log` at the top of the file to track modifications. +- **Single Export** — export **only one main symbol** per file. +- **Function Style** — avoid unnecessary blank lines inside functions. + +--- + +## 🚫 Forbidden Actions + +| ❌ Forbidden | ✅ Correct Approach | +| ----------------------------------------------- | ------------------------------------------------------- | +| SQL Triggers for business logic | NestJS Service methods | +| `.env` files in production | `docker-compose.yml` environment section | +| TypeORM migration files | Edit schema SQL directly (ADR-009) | +| Inventing table/column names | Verify against `lcbp3-v1.9.0-schema-02-tables.sql` | +| `any` TypeScript type | Proper types / generics | +| `console.log` in committed code | NestJS Logger (backend) / remove (frontend) | +| `req: any` in controllers | `RequestWithUser` typed interface | +| `parseInt()` on UUID values | Use UUID string directly (ADR-019) | +| Exposing INT PK in API responses | UUIDv7 (ADR-019) | +| AI accessing DB/storage directly | AI → DMS API → DB (ADR-023) | +| Direct file operations bypassing StorageService | `StorageService` for all file moves | +| Inline email/notification sending | BullMQ queue job | +| Deploying without Release Gates | Complete `04-08-release-management-policy.md` | +| AI direct cloud API calls | On-premises Ollama only (ADR-023) | +| AI outputs without human validation | Human-in-the-loop validation required (ADR-023) | +| n8n calling Ollama/Qdrant directly | n8n → DMS API → BullMQ → Ollama/Qdrant (ADR-023A) | +| Qdrant query without `projectPublicId` filter | `QdrantService.search(projectPublicId, ...)` (ADR-023A) | + +--- + +## 🚧 Out of Scope — Never Do Without Explicit Approval + +| ❌ Never Do Autonomously | ⚠️ Why Approval Is Required | +| --------------------------------------------------------------- | ---------------------------------------------------------------- | +| `DROP` or `RENAME` a column / table | Irreversible data loss — requires DBA + PM sign-off | +| Push directly to `main` / `master` branch | Bypasses CI, code review, and release gates | +| Generate or insert seed data into production database | May corrupt live data or violate business state invariants | +| Delete files from permanent storage | Files may be referenced in active documents or audit trails | +| Modify RBAC permission matrix without security team approval | Defines access control for all users — security boundary change | +| Upgrade major library versions (NestJS, Next.js, TypeORM, etc.) | Breaking changes require full regression test cycle | +| Disable or modify authentication / authorization guards | Creates unguarded endpoints — immediate security risk | +| Change Redis lock TTL or disable Redlock | Risk of document number race condition (ADR-002) | +| Create or supersede an ADR unilaterally | Architecture decisions require team consensus and review process | +| Add new columns to production tables without schema review | Must update Data Dictionary + downstream queries simultaneously | diff --git a/.agents/rules/02-security.md b/.agents/rules/02-security.md index ddc3ae76..13a10273 100644 --- a/.agents/rules/02-security.md +++ b/.agents/rules/02-security.md @@ -9,9 +9,9 @@ 5. **Password:** bcrypt 12 salt rounds, min 8 chars, rotate every 90 days 6. **Rate Limiting:** `ThrottlerGuard` on all auth endpoints 7. **File Upload:** Whitelist PDF/DWG/DOCX/XLSX/ZIP, max 50MB, ClamAV scan -8. **AI Isolation (ADR-018):** Ollama on Admin Desktop ONLY — NO direct DB/storage access +8. **AI Isolation (ADR-023/023A):** Ollama on Admin Desktop ONLY — NO direct DB/storage access; 2-model stack `gemma4:e4b Q8_0` + `nomic-embed-text`; all inference via BullMQ (`ai-realtime` / `ai-batch`) 9. **Error Handling (ADR-007):** Use layered error classification with user-friendly messages -10. **AI Integration (ADR-020):** RFA-First approach with unified pipeline architecture +10. **AI Integration (ADR-023/023A):** RFA-First approach; n8n orchestrates Migration Phase only via DMS API — never calls Ollama directly; `QdrantService.search()` requires `projectPublicId` as mandatory param 11. **AI Audit Trail:** Log all AI interactions and human validations 12. **Rate Limiting:** Apply to AI endpoints to prevent abuse @@ -26,7 +26,7 @@ - [ ] No SQL injection vulnerabilities - [ ] File upload validation (whitelist + ClamAV) - [ ] Rate limiting applied to auth endpoints -- [ ] AI boundary enforcement (ADR-023) - no direct DB/storage access +- [ ] AI boundary enforcement (ADR-023/023A) - no direct DB/storage access - [ ] AI audit logging implemented for AI interactions - [ ] AI outputs validated before use (human-in-the-loop) - [ ] Error handling follows ADR-007 layered classification diff --git a/.agents/rules/03-typescript.md b/.agents/rules/03-typescript.md index 3cda0a54..7e13e7fd 100644 --- a/.agents/rules/03-typescript.md +++ b/.agents/rules/03-typescript.md @@ -1,4 +1,4 @@ -# TypeScript Rules (v1.9.3) +# TypeScript Rules ## Core Standards diff --git a/.agents/rules/04-domain-terminology.md b/.agents/rules/04-domain-terminology.md index 0f43715c..14524d8f 100644 --- a/.agents/rules/04-domain-terminology.md +++ b/.agents/rules/04-domain-terminology.md @@ -22,14 +22,20 @@ Spec priority: **`06-Decision-Records`** > **`05-Engineering-Guidelines`** > others -| Document | Path | Use When | -| ----------------------- | ----------------------------------------------------------------- | ------------------------------- | -| **Glossary** | `specs/00-overview/00-02-glossary.md` | Verify domain terminology | -| **Schema Tables** | `specs/03-Data-and-Storage/lcbp3-v1.9.0-schema-02-tables.sql` | Before writing any query | -| **Data Dictionary** | `specs/03-Data-and-Storage/03-01-data-dictionary.md` | Field meanings + business rules | -| **Edge Cases** | `specs/01-Requirements/01-06-edge-cases-and-rules.md` | Prevent bugs in flows | -| **ADR-019 UUID** | `specs/06-Decision-Records/ADR-019-hybrid-identifier-strategy.md` | UUID-related work | -| **ADR-023 AI** | `specs/06-Decision-Records/ADR-023-unified-ai-architecture.md` | AI integration work | -| **Backend Guidelines** | `specs/05-Engineering-Guidelines/05-02-backend-guidelines.md` | NestJS patterns | -| **Frontend Guidelines** | `specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md` | Next.js patterns | -| **Testing Strategy** | `specs/05-Engineering-Guidelines/05-04-testing-strategy.md` | Coverage goals | +| Document | Path | Use When | +| ------------------------------ | --------------------------------------------------------------------------- | --------------------------------- | +| **Glossary** | `specs/00-overview/00-02-glossary.md` | Verify domain terminology | +| **Schema Tables** | `specs/03-Data-and-Storage/lcbp3-v1.9.0-schema-02-tables.sql` | Before writing any query | +| **Data Dictionary** | `specs/03-Data-and-Storage/03-01-data-dictionary.md` | Field meanings + business rules | +| **Edge Cases** | `specs/01-Requirements/01-06-edge-cases-and-rules.md` | Prevent bugs in flows | +| **ADR-019 UUID** | `specs/06-Decision-Records/ADR-019-hybrid-identifier-strategy.md` | UUID-related work | +| **ADR-023 AI** | `specs/06-Decision-Records/ADR-023-unified-ai-architecture.md` | AI integration work | +| **ADR-023A AI Model** | `specs/06-Decision-Records/ADR-023A-unified-ai-architecture.md` | 2-model stack, BullMQ 2-queue | +| **ADR-024 Intent Class.** | `specs/06-Decision-Records/ADR-024-intent-classification-strategy.md` | Pattern→LLM Fallback; Redis cache | +| **ADR-025 AI Tool Layer** | `specs/06-Decision-Records/ADR-025-ai-tool-layer-architecture.md` | Tool Registry; CASL-guarded | +| **ADR-026 Chat UI** | `specs/06-Decision-Records/ADR-026-document-chat-ui-pattern.md` | Side-panel; streaming SSE | +| **ADR-027 AI Admin Console** | `specs/06-Decision-Records/ADR-027-ai-admin-console-and-dynamic-control.md` | Dynamic control; admin-only | +| **ADR-028 Migration Refactor** | `specs/06-Decision-Records/ADR-028-migration-architecture-refactor.md` | Staging Queue; cleanup | +| **Backend Guidelines** | `specs/05-Engineering-Guidelines/05-02-backend-guidelines.md` | NestJS patterns | +| **Frontend Guidelines** | `specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md` | Next.js patterns | +| **Testing Strategy** | `specs/05-Engineering-Guidelines/05-04-testing-strategy.md` | Coverage goals | diff --git a/.agents/rules/08-development-flow.md b/.agents/rules/08-development-flow.md index 4fd9b336..7fe194f9 100644 --- a/.agents/rules/08-development-flow.md +++ b/.agents/rules/08-development-flow.md @@ -28,15 +28,82 @@ - Add minimal test if logic changed - Check forbidden patterns before commit +### 🟢 Specialized Work — ADR-021, AI Runtime Layer, Complex Logic + +**MUST complete:** + +1. **Domain Knowledge Check** - Read relevant ADRs (ADR-021, ADR-023/023A, ADR-024~028) +2. **Pattern Verification** - Check existing implementations in codebase +3. **Specialized Requirements** - Follow domain-specific patterns +4. **Complex Logic Testing** - Multi-scenario test coverage +5. **Performance Validation** - Load testing if applicable + +**For ADR-021 Integration:** + +- Read ADR-021 - Integrated workflow & step attachments +- Check ADR-001 - Unified workflow engine patterns +- Verify WorkflowEngineService - Polymorphic instance handling +- Add workflow fields - Expose workflowInstanceId, workflowState, availableActions +- Include IntegratedBanner - Frontend workflow lifecycle display +- Test workflow transitions - State changes and action validation + +**For AI Infrastructure (ADR-023/023A):** + +- Verify AI boundary enforcement - No direct DB/storage access +- Check BullMQ 2-queue setup - ai-realtime + ai-batch +- Validate Qdrant multi-tenancy - projectPublicId filter required +- Test human-in-the-loop validation workflows +- Audit AI interaction logging to ai_audit_logs + +**For AI Runtime Layer (ADR-024/025/026/027):** + +- ADR-024: Pattern Layer first (ai_intent_patterns DB + Redis cache 5 min) → LLM Fallback (gemma4:e4b, semaphore max=3) +- ADR-025: Tool Registry dispatch — AI Gateway → Tool → Business Service; ToolResult DTO must use publicId only +- ADR-026: useAiChat() hook + side-panel UI; streaming response via SSE; TanStack Query cache +- ADR-027: Admin Console — dynamic model/prompt/intent control; CASL-guarded admin-only endpoints + +**For Migration Pipeline (ADR-028):** + +- Use Staging Queue pattern — never write directly to production tables +- Post-migration cleanup process required after each batch +- Migration Validation Gates must pass before promoting to production + +**Expected output:** + +- Backend services expose specialized context fields +- Frontend components use domain-specific patterns +- Complex state management with proper validation +- Performance metrics within acceptable thresholds +- Comprehensive test coverage for edge cases + +--- + ## Context-Aware Triggers -| Request | Files to Check | Expected Response | -| -------------------- | -------------------------------------------------------------------- | --------------------------------------------------- | -| "สร้าง API ใหม่" | `05-02-backend-guidelines.md`, `lcbp3-v1.9.0-schema-02-tables.sql` | NestJS Controller + Service + DTO + CASL Guard | -| "แก้ฟอร์ม frontend" | `05-03-frontend-guidelines.md`, `01-06-edge-cases.md` | RHF+Zod + TanStack Query + Thai comments | -| "เพิ่ม field ใหม่" | `ADR-009`, `data-dictionary.md`, `lcbp3-v1.9.0-schema-02-tables.sql` | Edit SQL directly + update Data Dictionary + Entity | -| "ตรวจสอบ UUID" | `ADR-019`, `05-07-hybrid-uuid-implementation-plan.md` | UUIDv7 MariaDB native UUID + TransformInterceptor | -| "สร้าง migration" | `ADR-009`, `03-06-migration-business-scope.md` | Edit SQL schema directly + n8n workflow | -| "ตรวจสอบ permission" | `seed-permissions.sql`, `ADR-016` | CASL 4-Level RBAC matrix | -| "deploy production" | `04-08-release-management-policy.md`, `ADR-015` | Release Gates + Blue-Green strategy | -| "เพิ่ม test" | `05-04-testing-strategy.md` | Coverage goals + test patterns | +| Request | Files to Check | Expected Response | +| ----------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| "สร้าง API ใหม่" | `05-02-backend-guidelines.md`, `lcbp3-v1.9.0-schema-02-tables.sql` | NestJS Controller + Service + DTO + CASL Guard | +| "แก้ฟอร์ม frontend" | `05-03-frontend-guidelines.md`, `01-06-edge-cases.md` | RHF+Zod + TanStack Query + Thai comments | +| "เพิ่ม field ใหม่" | `ADR-009`, `data-dictionary.md`, `lcbp3-v1.9.0-schema-02-tables.sql` | Edit SQL directly + update Data Dictionary + Entity | +| "ตรวจสอบ UUID" | `ADR-019`, `05-07-hybrid-uuid-implementation-plan.md` | UUIDv7 MariaDB native UUID + TransformInterceptor | +| "สร้าง migration" | `ADR-009`, `03-06-migration-business-scope.md` | Edit SQL schema directly + n8n workflow | +| "ตรวจสอบ permission" | `seed-permissions.sql`, `ADR-016` | CASL 4-Level RBAC matrix | +| "deploy production" | `04-08-release-management-policy.md`, `ADR-015` | Release Gates + Blue-Green strategy | +| "เพิ่ม test" | `05-04-testing-strategy.md` | Coverage goals + test patterns | +| "AI integration" | ✅ | `ADR-023`, `ADR-023A`, `ADR-024`, `ADR-025` | AI boundary + 2-model stack + BullMQ queue policy + Intent/Tool Layer | +| "Error handling" | ✅ | `ADR-007` | Layered error classification + recovery | +| "File upload" | ✅ | `ADR-016`, `05-02-backend-guidelines.md`, `03-Data-and-Storage/03-03-file-storage.md` | Two-phase upload → temp → commit; ClamAV + whitelist | +| "Notifications / Queue" | ✅ | `ADR-008`, `05-02-backend-guidelines.md` | BullMQ job — never inline; check retry + dead-letter | +| "Add i18n / translate" | ✅ | `05-08-i18n-guidelines.md` | i18n keys only — no hardcoded text | +| "Workflow / DSL" | ✅ | `ADR-001`, `01-03-modules/01-03-06-unified-workflow.md` | DSL state machine + WorkflowEngineService | +| "Document numbering" | ✅ | `ADR-002`, `01-02-business-rules/01-02-02-doc-numbering-rules.md` | Redis Redlock + DB optimistic lock (double-lock) | +| "ตรวจสอบ Workflow" | ✅ | `01-06-edge-cases-and-rules.md`, `05-02-backend-guidelines.md`, `ADR-001`, `ADR-002` | เช็คการเปลี่ยน State, คิว BullMQ และการล็อกเลขที่เอกสาร | +| "Transmittal submit" | 📋 | `ADR-021`, `specs/200-fullstacks/201-transmittals-circulation/` | submit() with EC-RFA-004 validation | +| "Circulation reassign" | 📋 | `ADR-021`, `specs/200-fullstacks/201-transmittals-circulation/` | reassignRouting() with EC-CIRC-001 | +| "สร้าง workflow ใหม่" | 📋 | `ADR-001`, `ADR-021`, `specs/200-fullstacks/203-unified-workflow-engine/` | DSL workflow definition + WorkflowEngineService setup | +| "ตรวจสอบ AI boundary" | ✅ | `ADR-023`, `ADR-023A` | Verify Ollama isolation + BullMQ queues + Qdrant projectPublicId filter | +| "Intent classification" | ✅ | `ADR-024`, `specs/200-fullstacks/224-intent-classification/` | Pattern Layer → LLM Fallback; ai_intent_patterns; Redis cache 5 min | +| "AI Tool Layer" | ✅ | `ADR-025`, `specs/200-fullstacks/225-ai-tool-layer-architecture/` | Tool Registry; CASL-guarded dispatch; ToolResult publicId only | +| "Document Chat UI" | ✅ | `ADR-026`, `specs/200-fullstacks/226-document-chat-ui-pattern/` | Side-panel; useAiChat() hook; streaming SSE; TanStack Query cache | +| "AI Admin Console" | ✅ | `ADR-027`, `specs/200-fullstacks/227-ai-admin-console/` | Dynamic model/prompt/intent control; admin-only CASL endpoints | +| "Migration refactor" | ✅ | `ADR-028`, `specs/200-fullstacks/228-migration-arch-refactor/` | Staging Queue; post-migration cleanup; validation gates | diff --git a/.agents/rules/README.md b/.agents/rules/README.md index 6686617f..6935ec3b 100644 --- a/.agents/rules/README.md +++ b/.agents/rules/README.md @@ -4,13 +4,14 @@ Critical rules and guidelines for AI agents working on LCBP3-DMS. ## Version -- **Current:** v1.9.3 -- **Last Updated:** 2026-05-15 -- **Synced with:** `AGENTS.md` (v1.9.3) +- **Current:** v1.9.6 +- **Last Updated:** 2026-05-22 +- **Synced with:** `AGENTS.md` (v1.9.6) ## Purpose This directory contains rule files that define: + - Project context and role expectations - Critical Tier 1 rules (CI blockers) - Coding standards and patterns @@ -24,6 +25,7 @@ This directory contains rule files that define: ### 🔴 Tier 1 — CRITICAL (CI BLOCKER) Build fails immediately if violated: + - Security (Auth, RBAC, Validation) - UUID Strategy (ADR-019) — no `parseInt` / `Number` / `+` on UUID - Database correctness — verify schema before writing queries @@ -35,15 +37,28 @@ Build fails immediately if violated: ### 🟡 Tier 2 — IMPORTANT (CODE REVIEW) Must fix before merge: + - Architecture patterns (thin controller, business logic in service) - Test coverage (80%+ business logic, 70%+ backend overall) - Cache invalidation - Naming conventions - TypeScript Standards: Missing JSDoc, explicit types, or file headers -### 🟢 Tier 3 — GUIDELINES +### 🟢 Tier 3 — SPECIALIZED WORK + +Requires domain-specific knowledge: + +- **ADR-021 Integration:** Workflow Engine & Context implementation +- **AI Infrastructure:** ADR-023/023A boundary enforcement and pipeline usage +- **AI Runtime Layer:** ADR-024 Intent Classification, ADR-025 Tool Layer, ADR-026 Chat UI, ADR-027 Admin Console +- **Migration Pipeline:** ADR-028 Staging Queue & post-migration cleanup +- **Complex Business Logic:** Multi-step workflows with state management +- **Performance Optimization:** Database queries, caching strategies, bulk operations + +### 🔵 Tier 4 — GUIDELINES Best practice — follow when possible: + - Code style / formatting (Prettier handles) - Comment completeness - Minor optimizations @@ -52,61 +67,68 @@ Best practice — follow when possible: ### Core Rules (Tier 1 - CRITICAL) -| File | Purpose | -|------|---------| +| File | Purpose | +| ----------------------- | ------------------------------------------------------------------------------- | | `00-project-context.md` | Project context, role & persona, tier classification, specs folder organization | -| `01-adr-019-uuid.md` | UUID handling strategy — no parseInt, use publicId only | -| `02-security.md` | Security requirements, checklist, ADR-023/023A AI boundaries | +| `01-adr-019-uuid.md` | UUID handling strategy — no parseInt, use publicId only | +| `02-security.md` | Security requirements, checklist, ADR-023/023A AI boundaries | ### Coding Standards -| File | Purpose | -|------|---------| -| `03-typescript.md` | TypeScript rules, file headers, i18n guidelines | -| `06-backend-patterns.md` | NestJS patterns, UUID resolution, API response patterns | +| File | Purpose | +| ------------------------- | ------------------------------------------------------- | +| `03-typescript.md` | TypeScript rules, file headers, i18n guidelines | +| `06-backend-patterns.md` | NestJS patterns, UUID resolution, API response patterns | | `07-frontend-patterns.md` | Next.js patterns, RHF+Zod+TanStack Query, UUID handling | ### Domain & Workflow -| File | Purpose | -|------|---------| -| `04-domain-terminology.md` | DMS glossary, key spec files priority table | -| `08-development-flow.md` | Development workflow by work type (Critical/Normal/Quick Fix) | +| File | Purpose | +| -------------------------- | ------------------------------------------------------------- | +| `04-domain-terminology.md` | DMS glossary, key spec files priority table | +| `08-development-flow.md` | Development workflow by work type (Critical/Normal/Quick Fix) | ### Compliance & Architecture -| File | Purpose | -|------|---------| +| File | Purpose | +| ------------------------- | -------------------------------------------------------------- | | `05-forbidden-actions.md` | Actions that must never be done, schema changes, UUID handling | -| `09-commit-checklist.md` | Pre-commit verification, commit message format | -| `10-error-handling.md` | ADR-007 error handling strategy, layered classification | -| `11-ai-integration.md` | ADR-023/023A AI architecture, 2-model stack, BullMQ 2-queue | +| `09-commit-checklist.md` | Pre-commit verification, commit message format | +| `10-error-handling.md` | ADR-007 error handling strategy, layered classification | +| `11-ai-integration.md` | ADR-023/023A AI architecture, 2-model stack, BullMQ 2-queue | ## Key Spec Files Priority Spec priority: **`06-Decision-Records`** > **`05-Engineering-Guidelines`** > others -| Document | Path | Use When | -|----------|------|----------| -| **Glossary** | `specs/00-overview/00-02-glossary.md` | Verify domain terminology | -| **Schema Tables** | `specs/03-Data-and-Storage/lcbp3-v1.9.0-schema-02-tables.sql` | Before writing any query | -| **Data Dictionary** | `specs/03-Data-and-Storage/03-01-data-dictionary.md` | Field meanings + business rules | -| **Edge Cases** | `specs/01-Requirements/01-06-edge-cases-and-rules.md` | Prevent bugs in flows | -| **ADR-019 UUID** | `specs/06-Decision-Records/ADR-019-hybrid-identifier-strategy.md` | UUID-related work | -| **ADR-023 AI** | `specs/06-Decision-Records/ADR-023-unified-ai-architecture.md` | AI integration work | -| **Backend Guidelines** | `specs/05-Engineering-Guidelines/05-02-backend-guidelines.md` | NestJS patterns | -| **Frontend Guidelines** | `specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md` | Next.js patterns | -| **Testing Strategy** | `specs/05-Engineering-Guidelines/05-04-testing-strategy.md` | Coverage goals | +| Document | Path | Use When | +| ------------------------------ | --------------------------------------------------------------------------- | --------------------------------- | +| **Glossary** | `specs/00-overview/00-02-glossary.md` | Verify domain terminology | +| **Schema Tables** | `specs/03-Data-and-Storage/lcbp3-v1.9.0-schema-02-tables.sql` | Before writing any query | +| **Data Dictionary** | `specs/03-Data-and-Storage/03-01-data-dictionary.md` | Field meanings + business rules | +| **Edge Cases** | `specs/01-Requirements/01-06-edge-cases-and-rules.md` | Prevent bugs in flows | +| **ADR-019 UUID** | `specs/06-Decision-Records/ADR-019-hybrid-identifier-strategy.md` | UUID-related work | +| **ADR-023 AI** | `specs/06-Decision-Records/ADR-023-unified-ai-architecture.md` | AI integration work | +| **ADR-023A AI Model** | `specs/06-Decision-Records/ADR-023A-unified-ai-architecture.md` | 2-model stack, BullMQ 2-queue | +| **ADR-024 Intent Class.** | `specs/06-Decision-Records/ADR-024-intent-classification-strategy.md` | Pattern→LLM Fallback; Redis cache | +| **ADR-025 AI Tool Layer** | `specs/06-Decision-Records/ADR-025-ai-tool-layer-architecture.md` | Tool Registry; CASL-guarded | +| **ADR-026 Chat UI** | `specs/06-Decision-Records/ADR-026-document-chat-ui-pattern.md` | Side-panel; streaming SSE | +| **ADR-027 AI Admin Console** | `specs/06-Decision-Records/ADR-027-ai-admin-console-and-dynamic-control.md` | Dynamic control; admin-only | +| **ADR-028 Migration Refactor** | `specs/06-Decision-Records/ADR-028-migration-architecture-refactor.md` | Staging Queue; cleanup | +| **Backend Guidelines** | `specs/05-Engineering-Guidelines/05-02-backend-guidelines.md` | NestJS patterns | +| **Frontend Guidelines** | `specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md` | Next.js patterns | +| **Testing Strategy** | `specs/05-Engineering-Guidelines/05-04-testing-strategy.md` | Coverage goals | ## Maintenance When updating rules: 1. **Check AGENTS.md version** — Ensure rule files are synced -2. **Update version numbers** — Bump version in `00-project-context.md` and `03-typescript.md` -3. **Review ADR references** — Ensure all ADR references are current (ADR-023, ADR-023A, etc.) +2. **Update version numbers** — Bump version in `00-project-context.md` only (03-typescript.md no longer has version) +3. **Review ADR references** — Ensure all ADR references are current (ADR-023, ADR-023A, ADR-024~028) 4. **Add new forbidden actions** — When new patterns are identified as violations 5. **Update key spec files table** — When new ADRs or guidelines are added +6. **Update Tier 3 SPECIALIZED WORK** — When new domain-specific workflows are added ## Related Documents diff --git a/.windsurf/rules/00-project-context.md b/.windsurf/rules/00-project-context.md index 65fd1478..fa7c2b73 100644 --- a/.windsurf/rules/00-project-context.md +++ b/.windsurf/rules/00-project-context.md @@ -2,58 +2,102 @@ trigger: always_on --- -# NAP-DMS Project Context +# NAP-DMS Project Context & Rules -## Role & Persona +- For: Windsurf Cascade (and compatible: Codex CLI, opencode, Amp, Antigravity, AGENTS.md tools) +- Version: 1.9.6 | Last synced from repo: 2026-05-22 +- Repo: [https://git.np-dms.work/np-dms/lcbp3](https://git.np-dms.work/np-dms/lcbp3) +- Skill pack: `.agents/skills/` (v1.9.0, 21 skills) — see [`skills/README.md`](./.agents/skills/README.md) + [`skills/_LCBP3-CONTEXT.md`](./.agents/skills/_LCBP3-CONTEXT.md) -Act as a **Senior Full Stack Developer** specialized in: +## 🧠 Role & Persona -- NestJS, Next.js, TypeScript -- Document Management Systems (DMS) +Act as **Senior Full Stack Developer** specialized in NestJS, Next.js, TypeScript, DMS. Focus: Data Integrity, Security, Maintainability, Performance. -Focus: +You are a **Document Intelligence Engine** — not a general chatbot. Every response must be **precise**, **spec-compliant**, and **production-ready**. -- Data Integrity -- Security -- Maintainability -- Performance +--- -You are a **Document Intelligence Engine** — not a general chatbot. -Every response must be **precise**, **spec-compliant**, and **production-ready**. +## 🧩 Thought & Planning Protocol (Powered by Everything-Claude-Code) -## Project Information +Before writing any code or taking any action in Tier 1 and Tier 2, the AI must demonstrate the following thinking process: -- **Project:** NAP-DMS (LCBP3) -- Version: 1.9.0 -- Last Updated: 2026-05-13 -- Canonical Source: AGENTS.md -- **Stack:** NestJS + Next.js + TypeScript + MariaDB + Ollama (AI) -- **Repo:** https://git.np-dms.work/np-dms/lcbp3 +### 1. Analysis Phase (Explore & Analyze) -## Rule Enforcement Tiers +Problem Understanding: Restate what the user wants in clear, unambiguous terms. +Context Search: Identify the relevant Spec files or ADRs from the "Key Spec Files" table that must be read before starting. +Constraints Identification: Identify key constraints (e.g. Security rules, UUID patterns, or Domain terminology). + +### 2. Planning Phase (Plan) + +Alternative Exploration: Present at least 2 solution approaches (where possible) with pros/cons analysis. +Step-by-Step Roadmap: Write a file-by-file plan of changes before executing. +Verification Plan: Specify how to verify the work is complete (e.g. "which unit tests to write" or "which file to check the schema in"). + +### 3. Execution & Refinement (Execute & Refine) + +Follow the plan step by step, and pause to ask if any uncertainty arises. +If significant logic changes are made, summarize what was done for the user after completion. + +--- + +## ⚙️ DMS Workflow Engine Protocol + +กฎนี้ใช้คุม Logic การไหลของเอกสาร (RFA, Transmittal, Correspondence) เพื่อป้องกัน Race Condition และรักษาความถูกต้องของสถานะ: + +- **State Management:** ตรวจสอบสถานะปัจจุบันจาก DB ก่อนเสมอ เพื่อป้องกันการอนุมัติซ้ำซ้อน (ดู `05-06-code-snippets.md` `[workflow-transition]`) +- **Concurrency Control:** การจอนเลขที่เอกสารต้องใช้ **Redis Redlock** หรือ **TypeORM `@VersionColumn`** เท่านั้น (ADR-002) +- **Background Jobs:** งานนานหรือการแจ้งเตือนต้องส่งไปทำที่ **BullMQ** ห้ามเขียนแบบ Inline (ADR-008) +- **Term Consistency:** ห้ามใช้ "Approval Flow" ให้ใช้ **"Workflow Engine"** และห้ามใช้ "Letter" ให้ใช้ **"Correspondence"** (หมายเหตุ: "จดหมาย" ในคอมเมนต์ภาษาไทย = Correspondence ที่ครอบคลุมทุกประเภท) + +--- + +## 🛡️ Security & Integrity Audit Protocol + +กฎนี้ให้ AI เป็น Gatekeeper ก่อน Commit โดยเน้น **Tier 1 — CRITICAL**: + +- **UUID Validation:** ตรวจสอบว่าเป็น **UUIDv7** และห้ามใช้ `parseInt()` บน UUID (ADR-019) +- **RBAC Check:** API ใหม่ต้องมี **CASL Guard** และตรวจสอบ 4-Level RBAC Matrix (ADR-016) +- **Data Isolation:** AI ต้องรันผ่าน **Ollama บน Admin Desktop** เท่านั้น ห้ามเข้าถึง DB/storage โดยตรง (ADR-023) +- **Input Sanitization:** ไฟล์อัปโหลดต้องผ่าน **Two-Phase** (Temp → Commit) และสแกนด้วย **ClamAV** (ADR-016) + +--- + +## 🧭 Rule Enforcement Tiers ### 🔴 Tier 1 — CRITICAL (CI BLOCKER) -Build fails immediately if violated: +Build fails หากละเมิด: - Security (Auth, RBAC, Validation) - UUID Strategy (ADR-019) — no `parseInt` / `Number` / `+` on UUID - Database correctness — verify schema before writing queries - File upload security (ClamAV + whitelist) -- AI validation boundary (ADR-018) +- AI validation boundary (ADR-023) - Error handling strategy (ADR-007) -- Forbidden patterns: `any`, `console.log`, UUID misuse +- Forbidden patterns: `any`, `console.log`, UUID misuse, `id ?? ''` fallback ### 🟡 Tier 2 — IMPORTANT (CODE REVIEW) -Must fix before merge: +Must fix ก่อน merge: - Architecture patterns (thin controller, business logic in service) - Test coverage (80%+ business logic, 70%+ backend overall) - Cache invalidation - Naming conventions +- **TypeScript Standards:** Missing JSDoc, explicit types, or file headers -### 🟢 Tier 3 — GUIDELINES +### 🟢 Tier 3 — SPECIALIZED WORK + +Requires domain-specific knowledge: + +- **ADR-021 Integration:** Workflow Engine & Context implementation +- **AI Infrastructure:** ADR-023/023A boundary enforcement and pipeline usage +- **AI Runtime Layer:** ADR-024 Intent Classification, ADR-025 Tool Layer, ADR-026 Chat UI, ADR-027 Admin Console +- **Migration Pipeline:** ADR-028 Staging Queue & post-migration cleanup +- **Complex Business Logic:** Multi-step workflows with state management +- **Performance Optimization:** Database queries, caching strategies, bulk operations + +### 🔵 Tier 4 — GUIDELINES Best practice — follow when possible: @@ -63,13 +107,13 @@ Best practice — follow when possible: --- -## 📐 TypeScript Rules & Coding Standards (v1.9.3) +## 📐 TypeScript Rules & Coding Standards ### 📝 Core Standards - **Strict Mode** — all strict checks enforced. - **ZERO `any` types** — use proper types or `unknown` + narrowing. -- **ZERO `console.log`** — use NestJS `Logger` (backend) or remove before commit (frontend). +- **ZERO `console.log`** — use NestJS `Logger` (backend) or remove (frontend). - **English for Code** — use English for all code identifiers, variables, and logic. - **Thai for Comments** — use Thai for comments, documentation, and JSDoc. - **Explicit Typing** — explicitly define types for all variables, parameters, and return values. @@ -81,3 +125,44 @@ Best practice — follow when possible: - **Change Log** — include `// Change Log` at the top of the file to track modifications. - **Single Export** — export **only one main symbol** per file. - **Function Style** — avoid unnecessary blank lines inside functions. + +--- + +## 🚫 Forbidden Actions + +| ❌ Forbidden | ✅ Correct Approach | +| ----------------------------------------------- | ------------------------------------------------------- | +| SQL Triggers for business logic | NestJS Service methods | +| `.env` files in production | `docker-compose.yml` environment section | +| TypeORM migration files | Edit schema SQL directly (ADR-009) | +| Inventing table/column names | Verify against `lcbp3-v1.9.0-schema-02-tables.sql` | +| `any` TypeScript type | Proper types / generics | +| `console.log` in committed code | NestJS Logger (backend) / remove (frontend) | +| `req: any` in controllers | `RequestWithUser` typed interface | +| `parseInt()` on UUID values | Use UUID string directly (ADR-019) | +| Exposing INT PK in API responses | UUIDv7 (ADR-019) | +| AI accessing DB/storage directly | AI → DMS API → DB (ADR-023) | +| Direct file operations bypassing StorageService | `StorageService` for all file moves | +| Inline email/notification sending | BullMQ queue job | +| Deploying without Release Gates | Complete `04-08-release-management-policy.md` | +| AI direct cloud API calls | On-premises Ollama only (ADR-023) | +| AI outputs without human validation | Human-in-the-loop validation required (ADR-023) | +| n8n calling Ollama/Qdrant directly | n8n → DMS API → BullMQ → Ollama/Qdrant (ADR-023A) | +| Qdrant query without `projectPublicId` filter | `QdrantService.search(projectPublicId, ...)` (ADR-023A) | + +--- + +## 🚧 Out of Scope — Never Do Without Explicit Approval + +| ❌ Never Do Autonomously | ⚠️ Why Approval Is Required | +| --------------------------------------------------------------- | ---------------------------------------------------------------- | +| `DROP` or `RENAME` a column / table | Irreversible data loss — requires DBA + PM sign-off | +| Push directly to `main` / `master` branch | Bypasses CI, code review, and release gates | +| Generate or insert seed data into production database | May corrupt live data or violate business state invariants | +| Delete files from permanent storage | Files may be referenced in active documents or audit trails | +| Modify RBAC permission matrix without security team approval | Defines access control for all users — security boundary change | +| Upgrade major library versions (NestJS, Next.js, TypeORM, etc.) | Breaking changes require full regression test cycle | +| Disable or modify authentication / authorization guards | Creates unguarded endpoints — immediate security risk | +| Change Redis lock TTL or disable Redlock | Risk of document number race condition (ADR-002) | +| Create or supersede an ADR unilaterally | Architecture decisions require team consensus and review process | +| Add new columns to production tables without schema review | Must update Data Dictionary + downstream queries simultaneously | diff --git a/.windsurf/rules/02-security.md b/.windsurf/rules/02-security.md index 4e3a3981..9cb08731 100644 --- a/.windsurf/rules/02-security.md +++ b/.windsurf/rules/02-security.md @@ -13,9 +13,9 @@ trigger: always_on 5. **Password:** bcrypt 12 salt rounds, min 8 chars, rotate every 90 days 6. **Rate Limiting:** `ThrottlerGuard` on all auth endpoints 7. **File Upload:** Whitelist PDF/DWG/DOCX/XLSX/ZIP, max 50MB, ClamAV scan -8. **AI Isolation (ADR-018):** Ollama on Admin Desktop ONLY — NO direct DB/storage access +8. **AI Isolation (ADR-023/023A):** Ollama on Admin Desktop ONLY — NO direct DB/storage access; 2-model stack `gemma4:e4b Q8_0` + `nomic-embed-text`; all inference via BullMQ (`ai-realtime` / `ai-batch`) 9. **Error Handling (ADR-007):** Use layered error classification with user-friendly messages -10. **AI Integration (ADR-020):** RFA-First approach with unified pipeline architecture +10. **AI Integration (ADR-023/023A):** RFA-First approach; n8n orchestrates Migration Phase only via DMS API — never calls Ollama directly; `QdrantService.search()` requires `projectPublicId` as mandatory param 11. **AI Audit Trail:** Log all AI interactions and human validations 12. **Rate Limiting:** Apply to AI endpoints to prevent abuse @@ -30,7 +30,7 @@ trigger: always_on - [ ] No SQL injection vulnerabilities - [ ] File upload validation (whitelist + ClamAV) - [ ] Rate limiting applied to auth endpoints -- [ ] AI boundary enforcement (ADR-023) - no direct DB/storage access +- [ ] AI boundary enforcement (ADR-023/023A) - no direct DB/storage access - [ ] AI audit logging implemented for AI interactions - [ ] AI outputs validated before use (human-in-the-loop) - [ ] Error handling follows ADR-007 layered classification diff --git a/.windsurf/rules/03-typescript.md b/.windsurf/rules/03-typescript.md index 3cda0a54..11e166e8 100644 --- a/.windsurf/rules/03-typescript.md +++ b/.windsurf/rules/03-typescript.md @@ -1,4 +1,8 @@ -# TypeScript Rules (v1.9.3) +--- +trigger: always_on +--- + +# TypeScript Rules ## Core Standards diff --git a/.windsurf/rules/04-domain-terminology.md b/.windsurf/rules/04-domain-terminology.md index 6672b33b..cf3005f8 100644 --- a/.windsurf/rules/04-domain-terminology.md +++ b/.windsurf/rules/04-domain-terminology.md @@ -26,14 +26,20 @@ trigger: always_on Spec priority: **`06-Decision-Records`** > **`05-Engineering-Guidelines`** > others -| Document | Path | Use When | -| ----------------------- | ----------------------------------------------------------------- | ------------------------------- | -| **Glossary** | `specs/00-overview/00-02-glossary.md` | Verify domain terminology | -| **Schema Tables** | `specs/03-Data-and-Storage/lcbp3-v1.9.0-schema-02-tables.sql` | Before writing any query | -| **Data Dictionary** | `specs/03-Data-and-Storage/03-01-data-dictionary.md` | Field meanings + business rules | -| **Edge Cases** | `specs/01-Requirements/01-06-edge-cases-and-rules.md` | Prevent bugs in flows | -| **ADR-019 UUID** | `specs/06-Decision-Records/ADR-019-hybrid-identifier-strategy.md` | UUID-related work | -| **ADR-023 AI** | `specs/06-Decision-Records/ADR-023-unified-ai-architecture.md` | AI integration work | -| **Backend Guidelines** | `specs/05-Engineering-Guidelines/05-02-backend-guidelines.md` | NestJS patterns | -| **Frontend Guidelines** | `specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md` | Next.js patterns | -| **Testing Strategy** | `specs/05-Engineering-Guidelines/05-04-testing-strategy.md` | Coverage goals | +| Document | Path | Use When | +| ------------------------------ | --------------------------------------------------------------------------- | --------------------------------- | +| **Glossary** | `specs/00-overview/00-02-glossary.md` | Verify domain terminology | +| **Schema Tables** | `specs/03-Data-and-Storage/lcbp3-v1.9.0-schema-02-tables.sql` | Before writing any query | +| **Data Dictionary** | `specs/03-Data-and-Storage/03-01-data-dictionary.md` | Field meanings + business rules | +| **Edge Cases** | `specs/01-Requirements/01-06-edge-cases-and-rules.md` | Prevent bugs in flows | +| **ADR-019 UUID** | `specs/06-Decision-Records/ADR-019-hybrid-identifier-strategy.md` | UUID-related work | +| **ADR-023 AI** | `specs/06-Decision-Records/ADR-023-unified-ai-architecture.md` | AI integration work | +| **ADR-023A AI Model** | `specs/06-Decision-Records/ADR-023A-unified-ai-architecture.md` | 2-model stack, BullMQ 2-queue | +| **ADR-024 Intent Class.** | `specs/06-Decision-Records/ADR-024-intent-classification-strategy.md` | Pattern→LLM Fallback; Redis cache | +| **ADR-025 AI Tool Layer** | `specs/06-Decision-Records/ADR-025-ai-tool-layer-architecture.md` | Tool Registry; CASL-guarded | +| **ADR-026 Chat UI** | `specs/06-Decision-Records/ADR-026-document-chat-ui-pattern.md` | Side-panel; streaming SSE | +| **ADR-027 AI Admin Console** | `specs/06-Decision-Records/ADR-027-ai-admin-console-and-dynamic-control.md` | Dynamic control; admin-only | +| **ADR-028 Migration Refactor** | `specs/06-Decision-Records/ADR-028-migration-architecture-refactor.md` | Staging Queue; cleanup | +| **Backend Guidelines** | `specs/05-Engineering-Guidelines/05-02-backend-guidelines.md` | NestJS patterns | +| **Frontend Guidelines** | `specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md` | Next.js patterns | +| **Testing Strategy** | `specs/05-Engineering-Guidelines/05-04-testing-strategy.md` | Coverage goals | diff --git a/.windsurf/rules/08-development-flow.md b/.windsurf/rules/08-development-flow.md index f1514893..19c15cdc 100644 --- a/.windsurf/rules/08-development-flow.md +++ b/.windsurf/rules/08-development-flow.md @@ -32,15 +32,82 @@ trigger: always_on - Add minimal test if logic changed - Check forbidden patterns before commit +### 🟢 Specialized Work — ADR-021, AI Runtime Layer, Complex Logic + +**MUST complete:** + +1. **Domain Knowledge Check** - Read relevant ADRs (ADR-021, ADR-023/023A, ADR-024~028) +2. **Pattern Verification** - Check existing implementations in codebase +3. **Specialized Requirements** - Follow domain-specific patterns +4. **Complex Logic Testing** - Multi-scenario test coverage +5. **Performance Validation** - Load testing if applicable + +**For ADR-021 Integration:** + +- Read ADR-021 - Integrated workflow & step attachments +- Check ADR-001 - Unified workflow engine patterns +- Verify WorkflowEngineService - Polymorphic instance handling +- Add workflow fields - Expose workflowInstanceId, workflowState, availableActions +- Include IntegratedBanner - Frontend workflow lifecycle display +- Test workflow transitions - State changes and action validation + +**For AI Infrastructure (ADR-023/023A):** + +- Verify AI boundary enforcement - No direct DB/storage access +- Check BullMQ 2-queue setup - ai-realtime + ai-batch +- Validate Qdrant multi-tenancy - projectPublicId filter required +- Test human-in-the-loop validation workflows +- Audit AI interaction logging to ai_audit_logs + +**For AI Runtime Layer (ADR-024/025/026/027):** + +- ADR-024: Pattern Layer first (ai_intent_patterns DB + Redis cache 5 min) → LLM Fallback (gemma4:e4b, semaphore max=3) +- ADR-025: Tool Registry dispatch — AI Gateway → Tool → Business Service; ToolResult DTO must use publicId only +- ADR-026: useAiChat() hook + side-panel UI; streaming response via SSE; TanStack Query cache +- ADR-027: Admin Console — dynamic model/prompt/intent control; CASL-guarded admin-only endpoints + +**For Migration Pipeline (ADR-028):** + +- Use Staging Queue pattern — never write directly to production tables +- Post-migration cleanup process required after each batch +- Migration Validation Gates must pass before promoting to production + +**Expected output:** + +- Backend services expose specialized context fields +- Frontend components use domain-specific patterns +- Complex state management with proper validation +- Performance metrics within acceptable thresholds +- Comprehensive test coverage for edge cases + +--- + ## Context-Aware Triggers -| Request | Files to Check | Expected Response | -| -------------------- | -------------------------------------------------------------------- | --------------------------------------------------- | -| "สร้าง API ใหม่" | `05-02-backend-guidelines.md`, `lcbp3-v1.9.0-schema-02-tables.sql` | NestJS Controller + Service + DTO + CASL Guard | -| "แก้ฟอร์ม frontend" | `05-03-frontend-guidelines.md`, `01-06-edge-cases.md` | RHF+Zod + TanStack Query + Thai comments | -| "เพิ่ม field ใหม่" | `ADR-009`, `data-dictionary.md`, `lcbp3-v1.9.0-schema-02-tables.sql` | Edit SQL directly + update Data Dictionary + Entity | -| "ตรวจสอบ UUID" | `ADR-019`, `05-07-hybrid-uuid-implementation-plan.md` | UUIDv7 MariaDB native UUID + TransformInterceptor | -| "สร้าง migration" | `ADR-009`, `03-06-migration-business-scope.md` | Edit SQL schema directly + n8n workflow | -| "ตรวจสอบ permission" | `seed-permissions.sql`, `ADR-016` | CASL 4-Level RBAC matrix | -| "deploy production" | `04-08-release-management-policy.md`, `ADR-015` | Release Gates + Blue-Green strategy | -| "เพิ่ม test" | `05-04-testing-strategy.md` | Coverage goals + test patterns | +| Request | Files to Check | Expected Response | +| ----------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- | +| "สร้าง API ใหม่" | `05-02-backend-guidelines.md`, `lcbp3-v1.9.0-schema-02-tables.sql` | NestJS Controller + Service + DTO + CASL Guard | +| "แก้ฟอร์ม frontend" | `05-03-frontend-guidelines.md`, `01-06-edge-cases.md` | RHF+Zod + TanStack Query + Thai comments | +| "เพิ่ม field ใหม่" | `ADR-009`, `data-dictionary.md`, `lcbp3-v1.9.0-schema-02-tables.sql` | Edit SQL directly + update Data Dictionary + Entity | +| "ตรวจสอบ UUID" | `ADR-019`, `05-07-hybrid-uuid-implementation-plan.md` | UUIDv7 MariaDB native UUID + TransformInterceptor | +| "สร้าง migration" | `ADR-009`, `03-06-migration-business-scope.md` | Edit SQL schema directly + n8n workflow | +| "ตรวจสอบ permission" | `seed-permissions.sql`, `ADR-016` | CASL 4-Level RBAC matrix | +| "deploy production" | `04-08-release-management-policy.md`, `ADR-015` | Release Gates + Blue-Green strategy | +| "เพิ่ม test" | `05-04-testing-strategy.md` | Coverage goals + test patterns | +| "AI integration" | ✅ | `ADR-023`, `ADR-023A`, `ADR-024`, `ADR-025` | AI boundary + 2-model stack + BullMQ queue policy + Intent/Tool Layer | +| "Error handling" | ✅ | `ADR-007` | Layered error classification + recovery | +| "File upload" | ✅ | `ADR-016`, `05-02-backend-guidelines.md`, `03-Data-and-Storage/03-03-file-storage.md` | Two-phase upload → temp → commit; ClamAV + whitelist | +| "Notifications / Queue" | ✅ | `ADR-008`, `05-02-backend-guidelines.md` | BullMQ job — never inline; check retry + dead-letter | +| "Add i18n / translate" | ✅ | `05-08-i18n-guidelines.md` | i18n keys only — no hardcoded text | +| "Workflow / DSL" | ✅ | `ADR-001`, `01-03-modules/01-03-06-unified-workflow.md` | DSL state machine + WorkflowEngineService | +| "Document numbering" | ✅ | `ADR-002`, `01-02-business-rules/01-02-02-doc-numbering-rules.md` | Redis Redlock + DB optimistic lock (double-lock) | +| "ตรวจสอบ Workflow" | ✅ | `01-06-edge-cases-and-rules.md`, `05-02-backend-guidelines.md`, `ADR-001`, `ADR-002` | เช็คการเปลี่ยน State, คิว BullMQ และการล็อกเลขที่เอกสาร | +| "Transmittal submit" | 📋 | `ADR-021`, `specs/200-fullstacks/201-transmittals-circulation/` | submit() with EC-RFA-004 validation | +| "Circulation reassign" | 📋 | `ADR-021`, `specs/200-fullstacks/201-transmittals-circulation/` | reassignRouting() with EC-CIRC-001 | +| "สร้าง workflow ใหม่" | 📋 | `ADR-001`, `ADR-021`, `specs/200-fullstacks/203-unified-workflow-engine/` | DSL workflow definition + WorkflowEngineService setup | +| "ตรวจสอบ AI boundary" | ✅ | `ADR-023`, `ADR-023A` | Verify Ollama isolation + BullMQ queues + Qdrant projectPublicId filter | +| "Intent classification" | ✅ | `ADR-024`, `specs/200-fullstacks/224-intent-classification/` | Pattern Layer → LLM Fallback; ai_intent_patterns; Redis cache 5 min | +| "AI Tool Layer" | ✅ | `ADR-025`, `specs/200-fullstacks/225-ai-tool-layer-architecture/` | Tool Registry; CASL-guarded dispatch; ToolResult publicId only | +| "Document Chat UI" | ✅ | `ADR-026`, `specs/200-fullstacks/226-document-chat-ui-pattern/` | Side-panel; useAiChat() hook; streaming SSE; TanStack Query cache | +| "AI Admin Console" | ✅ | `ADR-027`, `specs/200-fullstacks/227-ai-admin-console/` | Dynamic model/prompt/intent control; admin-only CASL endpoints | +| "Migration refactor" | ✅ | `ADR-028`, `specs/200-fullstacks/228-migration-arch-refactor/` | Staging Queue; post-migration cleanup; validation gates | diff --git a/.windsurf/rules/README.md b/.windsurf/rules/README.md index 6686617f..52b8949b 100644 --- a/.windsurf/rules/README.md +++ b/.windsurf/rules/README.md @@ -1,16 +1,21 @@ +--- +trigger: always_on +--- + # LCBP3 Agent Rules Critical rules and guidelines for AI agents working on LCBP3-DMS. ## Version -- **Current:** v1.9.3 -- **Last Updated:** 2026-05-15 -- **Synced with:** `AGENTS.md` (v1.9.3) +- **Current:** v1.9.6 +- **Last Updated:** 2026-05-22 +- **Synced with:** `AGENTS.md` (v1.9.6) ## Purpose This directory contains rule files that define: + - Project context and role expectations - Critical Tier 1 rules (CI blockers) - Coding standards and patterns @@ -24,6 +29,7 @@ This directory contains rule files that define: ### 🔴 Tier 1 — CRITICAL (CI BLOCKER) Build fails immediately if violated: + - Security (Auth, RBAC, Validation) - UUID Strategy (ADR-019) — no `parseInt` / `Number` / `+` on UUID - Database correctness — verify schema before writing queries @@ -35,15 +41,28 @@ Build fails immediately if violated: ### 🟡 Tier 2 — IMPORTANT (CODE REVIEW) Must fix before merge: + - Architecture patterns (thin controller, business logic in service) - Test coverage (80%+ business logic, 70%+ backend overall) - Cache invalidation - Naming conventions - TypeScript Standards: Missing JSDoc, explicit types, or file headers -### 🟢 Tier 3 — GUIDELINES +### 🟢 Tier 3 — SPECIALIZED WORK + +Requires domain-specific knowledge: + +- **ADR-021 Integration:** Workflow Engine & Context implementation +- **AI Infrastructure:** ADR-023/023A boundary enforcement and pipeline usage +- **AI Runtime Layer:** ADR-024 Intent Classification, ADR-025 Tool Layer, ADR-026 Chat UI, ADR-027 Admin Console +- **Migration Pipeline:** ADR-028 Staging Queue & post-migration cleanup +- **Complex Business Logic:** Multi-step workflows with state management +- **Performance Optimization:** Database queries, caching strategies, bulk operations + +### 🔵 Tier 4 — GUIDELINES Best practice — follow when possible: + - Code style / formatting (Prettier handles) - Comment completeness - Minor optimizations @@ -52,61 +71,68 @@ Best practice — follow when possible: ### Core Rules (Tier 1 - CRITICAL) -| File | Purpose | -|------|---------| +| File | Purpose | +| ----------------------- | ------------------------------------------------------------------------------- | | `00-project-context.md` | Project context, role & persona, tier classification, specs folder organization | -| `01-adr-019-uuid.md` | UUID handling strategy — no parseInt, use publicId only | -| `02-security.md` | Security requirements, checklist, ADR-023/023A AI boundaries | +| `01-adr-019-uuid.md` | UUID handling strategy — no parseInt, use publicId only | +| `02-security.md` | Security requirements, checklist, ADR-023/023A AI boundaries | ### Coding Standards -| File | Purpose | -|------|---------| -| `03-typescript.md` | TypeScript rules, file headers, i18n guidelines | -| `06-backend-patterns.md` | NestJS patterns, UUID resolution, API response patterns | +| File | Purpose | +| ------------------------- | ------------------------------------------------------- | +| `03-typescript.md` | TypeScript rules, file headers, i18n guidelines | +| `06-backend-patterns.md` | NestJS patterns, UUID resolution, API response patterns | | `07-frontend-patterns.md` | Next.js patterns, RHF+Zod+TanStack Query, UUID handling | ### Domain & Workflow -| File | Purpose | -|------|---------| -| `04-domain-terminology.md` | DMS glossary, key spec files priority table | -| `08-development-flow.md` | Development workflow by work type (Critical/Normal/Quick Fix) | +| File | Purpose | +| -------------------------- | ------------------------------------------------------------- | +| `04-domain-terminology.md` | DMS glossary, key spec files priority table | +| `08-development-flow.md` | Development workflow by work type (Critical/Normal/Quick Fix) | ### Compliance & Architecture -| File | Purpose | -|------|---------| +| File | Purpose | +| ------------------------- | -------------------------------------------------------------- | | `05-forbidden-actions.md` | Actions that must never be done, schema changes, UUID handling | -| `09-commit-checklist.md` | Pre-commit verification, commit message format | -| `10-error-handling.md` | ADR-007 error handling strategy, layered classification | -| `11-ai-integration.md` | ADR-023/023A AI architecture, 2-model stack, BullMQ 2-queue | +| `09-commit-checklist.md` | Pre-commit verification, commit message format | +| `10-error-handling.md` | ADR-007 error handling strategy, layered classification | +| `11-ai-integration.md` | ADR-023/023A AI architecture, 2-model stack, BullMQ 2-queue | ## Key Spec Files Priority Spec priority: **`06-Decision-Records`** > **`05-Engineering-Guidelines`** > others -| Document | Path | Use When | -|----------|------|----------| -| **Glossary** | `specs/00-overview/00-02-glossary.md` | Verify domain terminology | -| **Schema Tables** | `specs/03-Data-and-Storage/lcbp3-v1.9.0-schema-02-tables.sql` | Before writing any query | -| **Data Dictionary** | `specs/03-Data-and-Storage/03-01-data-dictionary.md` | Field meanings + business rules | -| **Edge Cases** | `specs/01-Requirements/01-06-edge-cases-and-rules.md` | Prevent bugs in flows | -| **ADR-019 UUID** | `specs/06-Decision-Records/ADR-019-hybrid-identifier-strategy.md` | UUID-related work | -| **ADR-023 AI** | `specs/06-Decision-Records/ADR-023-unified-ai-architecture.md` | AI integration work | -| **Backend Guidelines** | `specs/05-Engineering-Guidelines/05-02-backend-guidelines.md` | NestJS patterns | -| **Frontend Guidelines** | `specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md` | Next.js patterns | -| **Testing Strategy** | `specs/05-Engineering-Guidelines/05-04-testing-strategy.md` | Coverage goals | +| Document | Path | Use When | +| ------------------------------ | --------------------------------------------------------------------------- | --------------------------------- | +| **Glossary** | `specs/00-overview/00-02-glossary.md` | Verify domain terminology | +| **Schema Tables** | `specs/03-Data-and-Storage/lcbp3-v1.9.0-schema-02-tables.sql` | Before writing any query | +| **Data Dictionary** | `specs/03-Data-and-Storage/03-01-data-dictionary.md` | Field meanings + business rules | +| **Edge Cases** | `specs/01-Requirements/01-06-edge-cases-and-rules.md` | Prevent bugs in flows | +| **ADR-019 UUID** | `specs/06-Decision-Records/ADR-019-hybrid-identifier-strategy.md` | UUID-related work | +| **ADR-023 AI** | `specs/06-Decision-Records/ADR-023-unified-ai-architecture.md` | AI integration work | +| **ADR-023A AI Model** | `specs/06-Decision-Records/ADR-023A-unified-ai-architecture.md` | 2-model stack, BullMQ 2-queue | +| **ADR-024 Intent Class.** | `specs/06-Decision-Records/ADR-024-intent-classification-strategy.md` | Pattern→LLM Fallback; Redis cache | +| **ADR-025 AI Tool Layer** | `specs/06-Decision-Records/ADR-025-ai-tool-layer-architecture.md` | Tool Registry; CASL-guarded | +| **ADR-026 Chat UI** | `specs/06-Decision-Records/ADR-026-document-chat-ui-pattern.md` | Side-panel; streaming SSE | +| **ADR-027 AI Admin Console** | `specs/06-Decision-Records/ADR-027-ai-admin-console-and-dynamic-control.md` | Dynamic control; admin-only | +| **ADR-028 Migration Refactor** | `specs/06-Decision-Records/ADR-028-migration-architecture-refactor.md` | Staging Queue; cleanup | +| **Backend Guidelines** | `specs/05-Engineering-Guidelines/05-02-backend-guidelines.md` | NestJS patterns | +| **Frontend Guidelines** | `specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md` | Next.js patterns | +| **Testing Strategy** | `specs/05-Engineering-Guidelines/05-04-testing-strategy.md` | Coverage goals | ## Maintenance When updating rules: 1. **Check AGENTS.md version** — Ensure rule files are synced -2. **Update version numbers** — Bump version in `00-project-context.md` and `03-typescript.md` -3. **Review ADR references** — Ensure all ADR references are current (ADR-023, ADR-023A, etc.) +2. **Update version numbers** — Bump version in `00-project-context.md` only (03-typescript.md no longer has version) +3. **Review ADR references** — Ensure all ADR references are current (ADR-023, ADR-023A, ADR-024~028) 4. **Add new forbidden actions** — When new patterns are identified as violations 5. **Update key spec files table** — When new ADRs or guidelines are added +6. **Update Tier 3 SPECIALIZED WORK** — When new domain-specific workflows are added ## Related Documents diff --git a/backend/src/modules/ai/ai-migration-checkpoint.service.ts b/backend/src/modules/ai/ai-migration-checkpoint.service.ts new file mode 100644 index 00000000..c65ba74f --- /dev/null +++ b/backend/src/modules/ai/ai-migration-checkpoint.service.ts @@ -0,0 +1,145 @@ +// File: src/modules/ai/ai-migration-checkpoint.service.ts +// Change Log: +// - 2026-05-23: สร้าง service จัดการ Migration Checkpoint, Queue และ Error log ผ่าน API (ADR-023A) + +import { Injectable, Logger } from '@nestjs/common'; +import { InjectRepository } from '@nestjs/typeorm'; +import { DataSource, Repository } from 'typeorm'; +import { + MigrationProgress, + MigrationProgressStatus, +} from './entities/migration-progress.entity'; +import { + MigrationReviewRecord, + MigrationReviewRecordStatus, +} from './entities/migration-review.entity'; +import { + MigrationErrorLogDto, + MigrationQueueRecordDto, + SaveCheckpointDto, +} from './dto/migration-checkpoint.dto'; + +/** Response DTO สำหรับ Checkpoint */ +export interface CheckpointResponse { + batchId: string; + lastProcessedIndex: number; + status: MigrationProgressStatus; + updatedAt: Date | null; +} + +@Injectable() +export class AiMigrationCheckpointService { + private readonly logger = new Logger(AiMigrationCheckpointService.name); + + constructor( + @InjectRepository(MigrationProgress) + private readonly progressRepo: Repository, + @InjectRepository(MigrationReviewRecord) + private readonly reviewRepo: Repository, + private readonly dataSource: DataSource + ) {} + + /** + * ดึง Checkpoint ปัจจุบันของ Batch — ถ้ายังไม่มีให้คืนค่า default + */ + async getCheckpoint(batchId: string): Promise { + const record = await this.progressRepo.findOne({ where: { batchId } }); + if (!record) { + return { + batchId, + lastProcessedIndex: 0, + status: MigrationProgressStatus.RUNNING, + updatedAt: null, + }; + } + return { + batchId: record.batchId, + lastProcessedIndex: record.lastProcessedIndex, + status: record.status, + updatedAt: record.updatedAt, + }; + } + + /** + * บันทึกหรืออัพเดต Checkpoint ของ Batch (Upsert) + */ + async saveCheckpoint(dto: SaveCheckpointDto): Promise { + const existing = await this.progressRepo.findOne({ + where: { batchId: dto.batchId }, + }); + + const record = + existing ?? this.progressRepo.create({ batchId: dto.batchId }); + record.lastProcessedIndex = dto.lastProcessedIndex; + record.status = dto.status ?? MigrationProgressStatus.RUNNING; + + const saved = await this.progressRepo.save(record); + this.logger.log( + `Checkpoint saved — batchId=${dto.batchId} index=${dto.lastProcessedIndex}` + ); + return { + batchId: saved.batchId, + lastProcessedIndex: saved.lastProcessedIndex, + status: saved.status, + updatedAt: saved.updatedAt, + }; + } + + /** + * บันทึกรายการเข้า Review Queue (Upsert โดยใช้ idempotencyKey) + */ + async upsertQueueRecord( + dto: MigrationQueueRecordDto + ): Promise<{ publicId: string }> { + const idempotencyKey = + dto.idempotencyKey ?? `${dto.batchId}:${dto.documentNumber}`; + + const existing = await this.reviewRepo.findOne({ + where: { idempotencyKey }, + }); + + const record = existing ?? this.reviewRepo.create({ idempotencyKey }); + + record.batchId = dto.batchId; + record.originalFileName = dto.documentNumber; + record.tempAttachmentId = dto.tempAttachmentId ?? undefined; + record.confidenceScore = dto.confidence ?? undefined; + record.status = + dto.status === 'PENDING_REVIEW' + ? MigrationReviewRecordStatus.PENDING_REVIEW + : MigrationReviewRecordStatus.PENDING; + record.errorReason = dto.reviewReason ?? undefined; + record.extractedMetadata = { + documentNumber: dto.documentNumber, + subject: dto.subject, + originalSubject: dto.originalSubject, + ...(dto.aiResult ?? {}), + }; + + const saved = await this.reviewRepo.save(record); + this.logger.log( + `Queue record upserted — batchId=${dto.batchId} doc=${dto.documentNumber} status=${dto.status}` + ); + return { publicId: saved.publicId }; + } + + /** + * บันทึก Error Log สำหรับเอกสารที่ประมวลผลไม่สำเร็จ + */ + async logError(dto: MigrationErrorLogDto): Promise<{ id: number }> { + const result = await this.dataSource.query<{ insertId: number }[]>( + `INSERT INTO migration_errors (batch_id, document_number, error_type, error_message, created_at) + VALUES (?, ?, ?, ?, NOW())`, + [ + dto.batchId, + dto.documentNumber, + dto.errorType ?? 'UNKNOWN', + dto.errorMessage ?? '', + ] + ); + this.logger.warn( + `Error logged — batchId=${dto.batchId} doc=${dto.documentNumber} type=${dto.errorType}` + ); + return { id: result[0]?.insertId ?? 0 }; + } +} diff --git a/backend/src/modules/ai/ai.controller.ts b/backend/src/modules/ai/ai.controller.ts index e6aa7ce7..07dab9ee 100644 --- a/backend/src/modules/ai/ai.controller.ts +++ b/backend/src/modules/ai/ai.controller.ts @@ -7,6 +7,7 @@ // - 2026-05-21: เพิ่ม GET /ai/admin/health สำหรับดึงสถานะสุขภาพ AI Infrastructure (T028). // - 2026-05-21: เพิ่ม POST /ai/admin/sandbox/extract endpoint สำหรับ Superadmin OCR sandbox (T041 & T042) // - 2026-05-21: แก้ไขข้อห้ามใช้ parseInt โดยการใช้ Number แทนตามกฎ Tier 1 +// - 2026-05-23: เพิ่ม Migration Checkpoint API endpoints แทน MySQL direct access (ADR-023A) // Controller สำหรับ AI Gateway Endpoints (ADR-023) import { @@ -79,6 +80,12 @@ import { AiEnabledGuard } from './guards/ai-enabled.guard'; import { InjectRedis } from '@nestjs-modules/ioredis'; import Redis from 'ioredis'; import { FileStorageService } from '../../common/file-storage/file-storage.service'; +import { AiMigrationCheckpointService } from './ai-migration-checkpoint.service'; +import { + MigrationErrorLogDto, + MigrationQueueRecordDto, + SaveCheckpointDto, +} from './dto/migration-checkpoint.dto'; @ApiTags('AI Gateway') @Controller('ai') @@ -91,6 +98,7 @@ export class AiController { private readonly aiSettingsService: AiSettingsService, private readonly aiToolRegistryService: AiToolRegistryService, private readonly fileStorageService: FileStorageService, + private readonly migrationCheckpointService: AiMigrationCheckpointService, @InjectRedis() private readonly redis: Redis ) {} @@ -682,4 +690,47 @@ export class AiController { user.user_id ); } + + // ─── Migration Checkpoint API (ADR-023A) ────────────────────────────────── + + @Get('migration/checkpoint/:batchId') + @UseGuards(JwtAuthGuard) + @ApiBearerAuth() + @ApiOperation({ summary: 'Migration: ดึง Checkpoint ของ Batch (ADR-023A)' }) + @ApiParam({ + name: 'batchId', + description: 'Batch ID ที่ต้องการดึง Checkpoint', + }) + async getMigrationCheckpoint(@Param('batchId') batchId: string) { + return this.migrationCheckpointService.getCheckpoint(batchId); + } + + @Post('migration/checkpoint') + @UseGuards(JwtAuthGuard) + @ApiBearerAuth() + @HttpCode(HttpStatus.OK) + @ApiOperation({ summary: 'Migration: บันทึก/อัพเดต Checkpoint (ADR-023A)' }) + async saveMigrationCheckpoint(@Body() dto: SaveCheckpointDto) { + return this.migrationCheckpointService.saveCheckpoint(dto); + } + + @Post('migration/queue/record') + @UseGuards(JwtAuthGuard) + @ApiBearerAuth() + @HttpCode(HttpStatus.OK) + @ApiOperation({ + summary: 'Migration: บันทึกรายการเข้า Review Queue (ADR-023A)', + }) + async upsertMigrationQueueRecord(@Body() dto: MigrationQueueRecordDto) { + return this.migrationCheckpointService.upsertQueueRecord(dto); + } + + @Post('migration/errors') + @UseGuards(JwtAuthGuard) + @ApiBearerAuth() + @HttpCode(HttpStatus.OK) + @ApiOperation({ summary: 'Migration: บันทึก Error Log (ADR-023A)' }) + async logMigrationError(@Body() dto: MigrationErrorLogDto) { + return this.migrationCheckpointService.logError(dto); + } } diff --git a/backend/src/modules/ai/ai.module.ts b/backend/src/modules/ai/ai.module.ts index a5b855e5..ff5744f3 100644 --- a/backend/src/modules/ai/ai.module.ts +++ b/backend/src/modules/ai/ai.module.ts @@ -6,6 +6,7 @@ // - 2026-05-19: เพิ่ม AiToolModule (ADR-025 AI Tool Layer). // - 2026-05-21: ลงทะเบียน SystemSetting, AiSettingsService และ AiEnabledGuard สำหรับ ADR-027. // - 2026-05-22: นำเข้าและลงทะเบียน CleanupTempFilesWorker (T016) เพื่อลบไฟล์แนบชั่วคราวหมดอายุ +// - 2026-05-23: ลงทะเบียน MigrationProgress + AiMigrationCheckpointService (ADR-023A) // Module สำหรับ AI Gateway — ลงทะเบียน Services และ Controllers (ADR-023) import { Logger, Module, OnModuleInit } from '@nestjs/common'; @@ -33,7 +34,9 @@ import { EmbeddingService } from './services/embedding.service'; import { MigrationLog } from './entities/migration-log.entity'; import { AiAuditLog } from './entities/ai-audit-log.entity'; import { MigrationReviewRecord } from './entities/migration-review.entity'; +import { MigrationProgress } from './entities/migration-progress.entity'; import { SystemSetting } from './entities/system-setting.entity'; +import { AiMigrationCheckpointService } from './ai-migration-checkpoint.service'; import { AiEnabledGuard } from './guards/ai-enabled.guard'; import { UserModule } from '../user/user.module'; import { MigrationModule } from '../migration/migration.module'; @@ -67,6 +70,7 @@ import { AiAuditLog, AuditLog, MigrationReviewRecord, + MigrationProgress, SystemSetting, Attachment, Project, @@ -129,6 +133,7 @@ import { AiService, AiSettingsService, AiIngestService, + AiMigrationCheckpointService, AiQueueService, AiQdrantService, AiValidationService, @@ -151,6 +156,7 @@ import { AiService, AiSettingsService, AiIngestService, + AiMigrationCheckpointService, AiQueueService, AiQdrantService, AiValidationService, diff --git a/backend/src/modules/ai/dto/migration-checkpoint.dto.ts b/backend/src/modules/ai/dto/migration-checkpoint.dto.ts new file mode 100644 index 00000000..37653e96 --- /dev/null +++ b/backend/src/modules/ai/dto/migration-checkpoint.dto.ts @@ -0,0 +1,97 @@ +// File: src/modules/ai/dto/migration-checkpoint.dto.ts +// Change Log: +// - 2026-05-23: สร้าง DTOs สำหรับ Migration Checkpoint API endpoints (ADR-023A) + +import { + IsEnum, + IsNotEmpty, + IsNumber, + IsObject, + IsOptional, + IsString, + Max, + Min, +} from 'class-validator'; +import { MigrationProgressStatus } from '../entities/migration-progress.entity'; + +/** DTO สำหรับบันทึก/อัพเดต Checkpoint */ +export class SaveCheckpointDto { + @IsString() + @IsNotEmpty() + batchId!: string; + + @IsNumber() + @Min(0) + lastProcessedIndex!: number; + + @IsEnum(MigrationProgressStatus) + @IsOptional() + status?: MigrationProgressStatus; +} + +/** DTO สำหรับบันทึกรายการเข้า Review Queue */ +export class MigrationQueueRecordDto { + @IsString() + @IsNotEmpty() + batchId!: string; + + @IsString() + @IsNotEmpty() + documentNumber!: string; + + @IsString() + @IsOptional() + subject?: string; + + @IsString() + @IsOptional() + originalSubject?: string; + + @IsNumber() + @IsOptional() + tempAttachmentId?: number; + + @IsNumber() + @Min(0) + @Max(1) + @IsOptional() + confidence?: number; + + @IsString() + @IsOptional() + reviewReason?: string; + + @IsEnum(['PENDING', 'PENDING_REVIEW']) + status!: 'PENDING' | 'PENDING_REVIEW'; + + @IsObject() + @IsOptional() + aiResult?: Record; + + @IsString() + @IsOptional() + idempotencyKey?: string; +} + +/** DTO สำหรับบันทึก Error Log */ +export class MigrationErrorLogDto { + @IsString() + @IsNotEmpty() + batchId!: string; + + @IsString() + @IsNotEmpty() + documentNumber!: string; + + @IsString() + @IsOptional() + errorType?: string; + + @IsString() + @IsOptional() + errorMessage?: string; + + @IsString() + @IsOptional() + jobId?: string; +} diff --git a/backend/src/modules/ai/entities/migration-progress.entity.ts b/backend/src/modules/ai/entities/migration-progress.entity.ts new file mode 100644 index 00000000..82108171 --- /dev/null +++ b/backend/src/modules/ai/entities/migration-progress.entity.ts @@ -0,0 +1,32 @@ +// File: src/modules/ai/entities/migration-progress.entity.ts +// Change Log: +// - 2026-05-23: สร้าง entity สำหรับ migration_progress table (Checkpoint management ADR-023A) + +import { Column, Entity, PrimaryColumn, UpdateDateColumn } from 'typeorm'; + +export enum MigrationProgressStatus { + RUNNING = 'RUNNING', + COMPLETED = 'COMPLETED', + FAILED = 'FAILED', +} + +/** ตารางติดตามความคืบหน้า Batch Migration — ใช้สำหรับ Resume กรณี n8n หยุดกลางทาง */ +@Entity('migration_progress') +export class MigrationProgress { + @PrimaryColumn({ name: 'batch_id', type: 'varchar', length: 50 }) + batchId!: string; + + @Column({ name: 'last_processed_index', type: 'int', default: 0 }) + lastProcessedIndex!: number; + + @Column({ + name: 'STATUS', + type: 'enum', + enum: MigrationProgressStatus, + default: MigrationProgressStatus.RUNNING, + }) + status!: MigrationProgressStatus; + + @UpdateDateColumn({ name: 'updated_at' }) + updatedAt!: Date; +} diff --git a/backend/src/modules/ai/entities/migration-review.entity.ts b/backend/src/modules/ai/entities/migration-review.entity.ts index d3152a15..917035b1 100644 --- a/backend/src/modules/ai/entities/migration-review.entity.ts +++ b/backend/src/modules/ai/entities/migration-review.entity.ts @@ -15,6 +15,7 @@ import { UuidBaseEntity } from '../../../common/entities/uuid-base.entity'; export enum MigrationReviewRecordStatus { PENDING = 'PENDING', + PENDING_REVIEW = 'PENDING_REVIEW', IMPORTED = 'IMPORTED', REJECTED = 'REJECTED', } diff --git a/memory/agent-memory.md b/memory/agent-memory.md new file mode 100644 index 00000000..68521cb9 --- /dev/null +++ b/memory/agent-memory.md @@ -0,0 +1,144 @@ + + + +# 🧠 Agent Long-term Project Memory + +> **Project:** NAP-DMS (LCBP3) — Laem Chabang Port Phase 3 Document Management System +> **Version:** 1.9.6 (Last Synced: 2026-05-23) +> **Stack:** NestJS 11 + Next.js 16 + TypeScript + MariaDB 11.8 + Redis + BullMQ + Elasticsearch + Ollama (on-prem AI) + +--- + +## 🧭 1. กฎการรันคำสั่งและการทำงานบนระบบ (OS Rules & Sandbox Constraints) + +> [!IMPORTANT] +> **ระบบรันอยู่บน Windows OS** +> +> - ห้ามใช้คำสั่ง `bash` หรือคำสั่งของ Linux โดยเด็ดขาด +> - คำสั่งทุกประเภทที่จะส่งให้ผู้ใช้รันหรือรันผ่าน Terminal ต้องเป็น **PowerShell** หรือ **CMD** เท่านั้น +> - ห้ามใช้คำสั่ง `cd` ในการสลับ Directory ให้ระบุพารามิเตอร์ `Cwd` ใน Tool ตรง ๆ + +--- + +## 🔴 2. กฎเหล็กระดับ Tier 1 (CI Blocker - ห้ามละเมิดเด็ดขาด) + +### 🆔 2.1 กลยุทธ์ UUID (ADR-019) + +- คีย์หลักในฐานข้อมูล (Internal PK) เป็น `INT AUTO_INCREMENT` ส่วนคีย์สาธารณะใน API/URL (Public API) จะใช้ **UUIDv7** เท่านั้น +- ฟิลด์คีย์สาธารณะชื่อ `publicId: string` จะถูกส่งออกไปใน API โดยตรง +- ❌ **ห้ามใช้** `parseInt()`, `Number()`, หรือเครื่องหมายบวก `+` บน UUID (เช่น `parseInt(projectId)` จะคำนวณผิดพลาดทันที) +- ❌ **ห้ามใช้** fallback ในลักษณะ `id ?? ''` ในฝั่ง Frontend ให้ใช้ `publicId` เท่านั้น + +### 🛡️ 2.2 สิทธิ์การใช้งานและความปลอดภัย (RBAC & Auth) + +- API ทุกตัวที่เป็นการปรับปรุงข้อมูล (Mutation: POST/PUT/PATCH) ต้องใช้ **CASL Guard** ตรวจสอบสิทธิ์ 4 ระดับ (RBAC Matrix) เสมอ +- API สำหรับการแก้ไขข้อมูล (POST/PUT/PATCH) ต้องตรวจสอบและยืนยัน **`Idempotency-Key`** ใน HTTP Header ทุกครั้ง +- การเข้ารหัสรหัสผ่านใช้ `bcrypt` ที่ระดับ 12 salt rounds เสมอ + +### 💾 2.3 การปรับแต่งและเปลี่ยนแปลง Schema (ADR-009) + +- ❌ **ห้ามใช้ TypeORM migrations ในการอัปเดต Schema** บนสภาพแวดล้อม Production +- ให้ใช้วิธี **แก้ไขไฟล์ SQL โครงสร้างหลักตรง ๆ** ใน `specs/03-Data-and-Storage/lcbp3-v1.9.0-schema-02-tables.sql` หรือเพิ่มการทำงานในโฟลเดอร์ `deltas/*.sql` เท่านั้น + +### 🏎️ 2.4 ป้องกัน Race Conditions ในเลขเอกสาร (ADR-002) + +- การจัดสรรและจองเลขที่เอกสารใหม่ (Document Numbering) ต้องใช้ **Redis Redlock** ควบคู่กับ TypeORM `@VersionColumn` เสมอ ห้ามเขียนตัวนับ (Counter) ในฝั่ง Application เดี่ยว ๆ + +### 🤖 2.5 ขอบเขตและการแยกส่วน AI (ADR-023/023A) + +- **Ollama (AI Inference) ต้องทำงานบน Admin Desktop เท่านั้น** ห้ามรันบน Server หรือ Docker ใน Production +- AI ห้ามเชื่อมต่อและเข้าถึง Database หรือ Storage โดยตรง (ต้องผ่าน DMS API เท่านั้น) +- โมเดลที่ใช้: `gemma4:e4b Q8_0` (LLM) และ `nomic-embed-text` (Embeddings) +- การทำงานแบบ Background Job หรือ Inference ที่ใช้เวลานานต้องสั่งงานผ่าน **BullMQ** (คิว `ai-realtime` และ `ai-batch`) +- ข้อมูลผลลัพธ์จาก AI ทั้งหมดต้องผ่านการตรวจสอบความถูกต้องโดยมนุษย์ (Human-in-the-loop) เสมอ + +--- + +## 🏷️ 3. Domain Terminology (คำศัพท์เฉพาะของระบบ DMS) + +ห้ามใช้คำศัพท์ทั่วไป ให้ใช้คำศัพท์ตามสารบัญหลักของโปรเจกต์เสมอ: + +| คำศัพท์ที่ถูกต้อง (✅ Use) | คำศัพท์ที่ห้ามใช้ (❌ Don't Use) | คำอธิบายภาษาไทย | +| :------------------------- | :------------------------------- | :--------------------------------------------- | +| **Correspondence** | Letter, Communication | จดหมาย/เอกสารติดต่อสื่อสาร (ครอบคลุมทุกประเภท) | +| **RFA** | Approval Request | เอกสารขออนุมัติ (Request for Approval) | +| **Transmittal** | Delivery Note, Cover Letter | เอกสารนำส่งแบบและเอกสาร | +| **Circulation** | Distribution, Routing | ใบเวียนเอกสารภายในหน่วยงาน | +| **Shop Drawing** | Construction Drawing | แบบก่อสร้างจริง | +| **Contract Drawing** | Design Drawing, Blueprint | แบบคู่สัญญา | +| **Workflow Engine** | Approval Flow, Process Engine | ระบบควบคุมและเปลี่ยนสถานะเอกสาร | +| **Document Numbering** | Document ID, Auto Number | ระบบออกเลขที่เอกสารอัตโนมัติ | + +--- + +## 🔄 4. สถานะและประวัติการทำงาน (Latest Session Progress) + +### Session 1 — 2026-05-23 (Specs Reorganization) + +- Reorganize โครงสร้างโฟลเดอร์ `specs/` สำเร็จ (`100-Infrastructures`, `200-fullstacks`, `300-others`) +- อัปเดตกฎ `AGENTS.md` และ `GEMINI.md` ให้ตรงกับมาตรฐานใหม่ +- ริเริ่มระบบ `memory/agent-memory.md` + +### Session 2 — 2026-05-23 (N8N Workflow Refactor) ← **ล่าสุด** + +#### Decisions ที่ Lock แล้ว (จาก QuizMe Session) + +| # | Decision | +| ------- | ------------------------------------------------------------------------------------ | +| S1 | **n8n = Migration only** — New Correspondence ใช้ Backend pipeline (BullMQ) | +| S2 | New Correspondence → BullMQ `ai-realtime` (ไม่ผ่าน n8n) | +| S3 | n8n call `POST /api/ai/jobs` (Backend) แทน Ollama direct (ADR-023A) | +| PA | Excel metadata (`docNumber`, `title`, `sender`, etc.) ส่งไปพร้อม AI job เป็น context | +| PB-Tags | Tag suggestion ทาง C — แนะนำ existing + สร้าง tag ใหม่ได้ถ้าไม่มี (`isNew` flag) | +| PB-UX | Editable form pre-filled ด้วย AI suggestions — user approve/edit ก่อน submit | + +#### Endpoint ที่ Verified แล้ว + +- `POST /api/ai/jobs` (`type: migrate-document`) — **พร้อมใช้งานแล้ว** (verified 2026-05-23) +- `GET /api/ai/jobs/:jobId` — polling endpoint พร้อม +- `POST /api/storage/upload` — two-phase upload พร้อม + +#### ไฟล์ที่สร้าง/แก้ไข + +| ไฟล์ | การเปลี่ยนแปลง | +| -------------------------------------------------------------- | -------------------------------------------------------------- | +| `specs/03-Data-and-Storage/CONTEXT-N8N-Refactor.md` | ✅ สร้างใหม่ — context doc สำหรับ implement | +| `specs/03-Data-and-Storage/n8n.workflow.v2.json` | ✅ สร้างใหม่ — ADR-023A compliant workflow | +| `specs/03-Data-and-Storage/03-05-n8n-migration-setup-guide.md` | ✅ อัพเดต v1.9.0 — ลบ Ollama direct, เพิ่มขั้นตอน update token | +| `specs/03-Data-and-Storage/03-06-migration-business-scope.md` | ✅ อัพเดต — Gate #1 blocker → Verified 2026-05-23 | + +#### สาระสำคัญของ n8n.workflow.v2.json + +**Nodes ที่ลบ (ADR-023A violations):** `Ollama AI Analysis`, `Build AI Prompt`, `Extract PDF Text` (Tika), `Check/Update Fallback State`, `Import to Backend`, `Upsert Tags`, `Link Tags` + +**Nodes ใหม่:** `Validate Token` → `Upload PDF to Backend` → `Build AI Job Payload` → `Submit AI Job` → `Poll AI Job Status` + +**Flow สรุป:** + +``` +Form Trigger → Set Config → Health/Token Check → Fetch Master Data +→ File Mount Check → Read Excel → Read Checkpoint +→ [Per Record: File Validate → Upload PDF → Submit AI Job → Poll → Parse/Route] + → Auto/Flagged → migration_review_queue + → Rejected → CSV Log + → Error → CSV + DB Log +→ Save Checkpoint → Delay → Loop +``` + +--- + +## 🎯 5. แผนงานขั้นต่อไป (Next Session Focus) + +### N8N Migration (งานหลักที่เหลือ) + +- [ ] **Import `n8n.workflow.v2.json`** เข้า n8n UI และทดสอบ End-to-End +- [ ] **ตรวจสอบ `ai-realtime` processor** ว่า return `suggestedTags[]` พร้อม `isNew` flag +- [ ] **Frontend Editable Review Form** (Pipeline B) — pre-fill AI suggestions + tag suggestion UI +- [ ] **Dry Run** กับ Excel จริงก่อน Production Migration + +### งานทั่วไป + +- [ ] รักษาความเป็นระเบียบและอัปเดต `memory/agent-memory.md` ทุกครั้งที่ Task สำคัญเสร็จ diff --git a/specs/03-Data-and-Storage/deltas/2026-05-23-alter-migration-review-queue-enum.sql b/specs/03-Data-and-Storage/deltas/2026-05-23-alter-migration-review-queue-enum.sql new file mode 100644 index 00000000..cde75b6f --- /dev/null +++ b/specs/03-Data-and-Storage/deltas/2026-05-23-alter-migration-review-queue-enum.sql @@ -0,0 +1,15 @@ +-- File: specs/03-Data-and-Storage/deltas/2026-05-23-alter-migration-review-queue-enum.sql +-- Change Log: +-- - 2026-05-23: เพิ่ม PENDING_REVIEW เข้า status ENUM ของ migration_review_queue (TypeORM-managed) +-- n8n workflow ไม่ใช้ MySQL direct access แล้ว — ใช้ POST /api/ai/migration/queue/record แทน (ADR-023A) +-- Delta: เพิ่ม PENDING_REVIEW ใน status column ของ migration_review_queue +-- Date: 2026-05-23 +-- Related: ADR-028, ADR-023A +-- รันก่อน deploy backend ที่มี PENDING_REVIEW ใน MigrationReviewRecordStatus enum +ALTER TABLE migration_review_queue +MODIFY COLUMN STATUS ENUM( + 'PENDING', + 'PENDING_REVIEW', + 'IMPORTED', + 'REJECTED' + ) DEFAULT 'PENDING'; diff --git a/specs/03-Data-and-Storage/lcbp3-v1.9.0-migration.sql b/specs/03-Data-and-Storage/lcbp3-v1.9.0-migration.sql index a6c36131..31f591a4 100644 --- a/specs/03-Data-and-Storage/lcbp3-v1.9.0-migration.sql +++ b/specs/03-Data-and-Storage/lcbp3-v1.9.0-migration.sql @@ -39,7 +39,12 @@ CREATE TABLE IF NOT EXISTS migration_review_queue ( extracted_tags JSON COMMENT 'Tag ที่ AI นำเสนอหรือจับคู่ได้', temp_attachment_id INT NULL COMMENT 'ID ของไฟล์ชั่วคราวจาก Two-Phase Storage', review_reason VARCHAR(255), - STATUS ENUM('PENDING', 'APPROVED', 'REJECTED') DEFAULT 'PENDING', + STATUS ENUM( + 'PENDING', + 'PENDING_REVIEW', + 'APPROVED', + 'REJECTED' + ) DEFAULT 'PENDING', reviewed_by VARCHAR(100), reviewed_at TIMESTAMP NULL, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, diff --git a/specs/03-Data-and-Storage/n8n.workflow.v2.json b/specs/03-Data-and-Storage/n8n.workflow.v2.json index ec9b3ae0..0cff9207 100644 --- a/specs/03-Data-and-Storage/n8n.workflow.v2.json +++ b/specs/03-Data-and-Storage/n8n.workflow.v2.json @@ -30,7 +30,7 @@ }, { "parameters": { - "jsCode": "// ============================================\n// CONFIGURATION v2.0 — ADR-023A Compliant\n// n8n เรียกแค่ DMS Backend API เท่านั้น — ห้ามเรียก Ollama โดยตรง\n// ============================================\nconst formData = $('Form Trigger').first()?.json || {};\nconst batchSizeInput = parseInt(String(formData['Batch Size'] || '0'));\nconst excelFileInput = String(formData['Excel File Path'] || '').trim();\n\nconst BATCH_ID = (() => {\n const d = new Date(Date.now() + 7 * 3600000);\n const s = d.toISOString();\n return s.substring(0,10).replace(/-/g,'') + ':' + s.substring(11,16).replace(/:/g,'');\n})();\n\nconst CONFIG = {\n // Backend Settings\n BACKEND_URL: 'https://backend.np-dms.work',\n MIGRATION_TOKEN: 'Bearer YOUR_MIGRATION_TOKEN_HERE', // 🔴 เปลี่ยน\n\n // Batch Settings\n BATCH_SIZE: batchSizeInput > 0 ? batchSizeInput : 10,\n BATCH_ID: BATCH_ID,\n DELAY_MS: 2000,\n\n // AI Job Settings (ADR-023A)\n AI_JOB_POLL_INTERVAL_MS: 5000,\n AI_JOB_TIMEOUT_MS: 120000,\n\n // Confidence Thresholds\n CONFIDENCE_HIGH: 0.85,\n CONFIDENCE_LOW: 0.60,\n\n // Paths (Container paths — ต้องตรงกับ volume mount ใน docker-compose)\n EXCEL_FILE: excelFileInput || '/home/node/.n8n-files/staging_ai/C22024.xlsx',\n SOURCE_PDF_DIR: '/home/node/.n8n-files/staging_ai',\n LOG_PATH: '/home/node/.n8n-files/migration_logs',\n};\n\nreturn [{ json: { config: CONFIG } }];" + "jsCode": "// ============================================\n// CONFIGURATION v2.0 — ADR-023A Compliant\n// n8n เรียกแค่ DMS Backend API เท่านั้น — ห้ามเรียก Ollama โดยตรง\n// ============================================\nconst formData = $('Form Trigger').first()?.json || {};\nconst batchSizeInput = parseInt(String(formData['Batch Size'] || '0'));\nconst excelFileInput = String(formData['Excel File Path'] || '').trim();\n\nconst BATCH_ID = (() => {\n const d = new Date(Date.now() + 7 * 3600000);\n const s = d.toISOString();\n return s.substring(0,10).replace(/-/g,'') + ':' + s.substring(11,16).replace(/:/g,'');\n})();\n\nconst CONFIG = {\n // Backend Settings\n BACKEND_URL: 'https://backend.np-dms.work',\n MIGRATION_TOKEN: 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6Im1pZ3JhdGlvbl9ib3QiLCJzdWIiOjUsInNjb3BlIjoiR2xvYmFsIiwiaWF0IjoxNzc5NTQwNDk4LCJleHAiOjQ5MzUzMDA0OTh9.TZVzG8u4Cyz8hi0cU3eGaeXinKWKHN3HtYnDcS5p_5I', // 🔴 เปลี่ยน\n\n // Batch Settings\n BATCH_SIZE: batchSizeInput > 0 ? batchSizeInput : 10,\n BATCH_ID: BATCH_ID,\n DELAY_MS: 2000,\n\n // AI Job Settings (ADR-023A)\n AI_JOB_POLL_INTERVAL_MS: 5000,\n AI_JOB_TIMEOUT_MS: 120000,\n\n // Confidence Thresholds\n CONFIDENCE_HIGH: 0.85,\n CONFIDENCE_LOW: 0.60,\n\n // Paths (Container paths — ต้องตรงกับ volume mount ใน docker-compose)\n EXCEL_FILE: excelFileInput || '/home/node/.n8n-files/staging_ai/C22024.xlsx',\n SOURCE_PDF_DIR: '/home/node/.n8n-files/staging_ai',\n LOG_PATH: '/home/node/.n8n-files/migration_logs',\n};\n\nreturn [{ json: { config: CONFIG } }];" }, "id": "8f1d3378-cca6-48b6-99db-693e46ac81ef", "name": "Set Configuration", @@ -56,7 +56,7 @@ }, { "parameters": { - "url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/auth/me", + "url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/auth/profile", "sendHeaders": true, "headerParameters": { "parameters": [ @@ -160,28 +160,30 @@ }, { "parameters": { - "operation": "executeQuery", - "query": "SELECT last_processed_index, status FROM migration_progress WHERE batch_id = '{{$('Set Configuration').first().json.config.BATCH_ID}}' LIMIT 1", + "url": "={{ $('Set Configuration').first().json.config.BACKEND_URL + '/api/ai/migration/checkpoint/' + $('Set Configuration').first().json.config.BATCH_ID }}", + "sendHeaders": true, + "headerParameters": { + "parameters": [ + { + "name": "Authorization", + "value": "={{$('Set Configuration').first().json.config.MIGRATION_TOKEN}}" + } + ] + }, "options": {} }, "id": "a83f8598-72fd-4cc8-9d98-1ea3cb3b42df", "name": "Read Checkpoint", - "type": "n8n-nodes-base.mySql", - "typeVersion": 2.4, + "type": "n8n-nodes-base.httpRequest", + "typeVersion": 4.1, "position": [32096, 13504], "alwaysOutputData": true, - "credentials": { - "mySql": { - "id": "CHHfbKhMacNo03V4", - "name": "MySQL account" - } - }, "onError": "continueErrorOutput", - "notes": "อ่านตำแหน่งล่าสุดที่ประมวลผล" + "notes": "GET /api/ai/migration/checkpoint/:batchId — ADR-023A (ไม่ใช้ MySQL โดยตรง)" }, { "parameters": { - "jsCode": "const cpJson = $input.first()?.json || {};\nconst startIndex = cpJson.last_processed_index || 0;\nconst config = $('Set Configuration').first().json.config;\n\nconst allItems = $('Read Excel').all().map(i => i.json);\nconst remaining = allItems.slice(startIndex);\nconst currentBatch = remaining.slice(0, config.BATCH_SIZE);\n\n// Exit condition: ไม่มี records เหลือ\nif (currentBatch.length === 0) {\n return [{ json: { batch_complete: true, total_processed: startIndex } }];\n}\n\nconst normalize = (str) => {\n if (!str) return '';\n return String(str).normalize('NFC').trim();\n};\n\nreturn currentBatch.map((item, i) => {\n const getVal = (possibleKeys) => {\n const exactMatch = possibleKeys.find(k => item[k] !== undefined);\n if (exactMatch) return item[exactMatch];\n const lowerTrimmedKeys = Object.keys(item).map(k => ({ original: k, parsed: k.toLowerCase().trim() }));\n for (const pk of possibleKeys) {\n const found = lowerTrimmedKeys.find(k => k.parsed === pk.toLowerCase().trim());\n if (found) return item[found.original];\n }\n return '';\n };\n\n const docNum = getVal(['document_number', 'correspondence_number', 'Document Number', 'Corr. No.']);\n const excelFileName = getVal(['File name', 'file_name', 'File Name', 'filename']);\n \n if (!excelFileName) {\n throw new Error(`Missing 'File name' column for row ${i + startIndex + 1}, document: ${docNum}`);\n }\n \n return {\n json: {\n batch_complete: false,\n document_number: normalize(docNum),\n title: normalize(getVal(['title', 'Title', 'Subject', 'subject'])),\n legacy_number: normalize(getVal(['legacy_number', 'Legacy Number', 'Response Doc.'])),\n excel_revision: getVal(['revision', 'Revision', 'rev']) || 1,\n original_index: startIndex + i,\n batch_id: config.BATCH_ID,\n file_name: normalize(excelFileName),\n issued_date: normalize(getVal(['issued_date', 'Issued_date', 'Issued Date', 'date', 'Date'])),\n received_date: normalize(getVal(['received_date', 'Received_date', 'Received Date'])),\n sender: normalize(getVal(['sender', 'Sender', 'From', 'from'])),\n receiver: normalize(getVal(['receiver', 'Receiver', 'To', 'to'])),\n }\n };\n});" + "jsCode": "const cpJson = $input.first()?.json || {};\nconst startIndex = cpJson.data?.lastProcessedIndex ?? cpJson.lastProcessedIndex ?? cpJson.last_processed_index ?? 0;\nconst config = $('Set Configuration').first().json.config;\n\nconst allItems = $('Read Excel').all().map(i => i.json);\nconst remaining = allItems.slice(startIndex);\nconst currentBatch = remaining.slice(0, config.BATCH_SIZE);\n\n// Exit condition: ไม่มี records เหลือ\nif (currentBatch.length === 0) {\n return [{ json: { batch_complete: true, total_processed: startIndex } }];\n}\n\nconst normalize = (str) => {\n if (!str) return '';\n return String(str).normalize('NFC').trim();\n};\n\nreturn currentBatch.map((item, i) => {\n const getVal = (possibleKeys) => {\n const exactMatch = possibleKeys.find(k => item[k] !== undefined);\n if (exactMatch) return item[exactMatch];\n const lowerTrimmedKeys = Object.keys(item).map(k => ({ original: k, parsed: k.toLowerCase().trim() }));\n for (const pk of possibleKeys) {\n const found = lowerTrimmedKeys.find(k => k.parsed === pk.toLowerCase().trim());\n if (found) return item[found.original];\n }\n return '';\n };\n\n const docNum = getVal(['document_number', 'correspondence_number', 'Document Number', 'Corr. No.']);\n const excelFileName = getVal(['File name', 'file_name', 'File Name', 'filename']);\n \n if (!excelFileName) {\n throw new Error(`Missing 'File name' column for row ${i + startIndex + 1}, document: ${docNum}`);\n }\n \n return {\n json: {\n batch_complete: false,\n document_number: normalize(docNum),\n title: normalize(getVal(['title', 'Title', 'Subject', 'subject'])),\n legacy_number: normalize(getVal(['legacy_number', 'Legacy Number', 'Response Doc.'])),\n excel_revision: getVal(['revision', 'Revision', 'rev']) || 1,\n original_index: startIndex + i,\n batch_id: config.BATCH_ID,\n file_name: normalize(excelFileName),\n issued_date: normalize(getVal(['issued_date', 'Issued_date', 'Issued Date', 'date', 'Date'])),\n received_date: normalize(getVal(['received_date', 'Received_date', 'Received Date'])),\n sender: normalize(getVal(['sender', 'Sender', 'From', 'from'])),\n receiver: normalize(getVal(['receiver', 'Receiver', 'To', 'to'])),\n }\n };\n});" }, "id": "80845e32-c283-4e9f-af73-6339d675fb38", "name": "Process Batch + Encoding", @@ -218,7 +220,7 @@ { "parameters": { "method": "POST", - "url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/storage/upload", + "url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/files/upload", "sendHeaders": true, "headerParameters": { "parameters": [ @@ -368,41 +370,55 @@ }, { "parameters": { - "operation": "executeQuery", - "query": "INSERT INTO migration_review_queue (document_number, title, original_title, ai_suggested_category, ai_confidence, ai_issues, review_reason, status, created_at) VALUES ('{{$json.document_number}}', '{{$json.ai_result.subject || $json.title}}', '{{$json.title}}', '{{$json.ai_result.suggested_category}}', {{$json.ai_result.confidence}}, '{{JSON.stringify({ job_id: $json.job_id, temp_attachment_id: $json.temp_attachment_id, type_code: $json.ai_result.type_code, subject: $json.ai_result.subject, suggested_tags: $json.ai_result.suggested_tags, key_points: $json.ai_result.key_points || [], issued_date: $json.ai_result.issued_date, received_date: $json.ai_result.received_date })}}', '{{$json.review_reason || \"\"}}', 'PENDING', NOW()) ON DUPLICATE KEY UPDATE status = 'PENDING', review_reason = '{{$json.review_reason || \"\"}}', ai_issues = VALUES(ai_issues), created_at = NOW()", + "method": "POST", + "url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/ai/migration/queue/record", + "sendHeaders": true, + "headerParameters": { + "parameters": [ + { + "name": "Authorization", + "value": "={{$('Set Configuration').first().json.config.MIGRATION_TOKEN}}" + } + ] + }, + "sendBody": true, + "specifyBody": "json", + "jsonBody": "={{ JSON.stringify({ batchId: $json.batch_id, documentNumber: $json.document_number, subject: ($json.ai_result?.subject || $json.title), originalSubject: $json.title, tempAttachmentId: $json.temp_attachment_id, confidence: $json.ai_result?.confidence, reviewReason: $json.review_reason || '', status: 'PENDING', aiResult: $json.ai_result, idempotencyKey: $json.batch_id + ':' + $json.document_number }) }}", "options": {} }, "id": "c1bd4485-e58f-4270-892e-edda34c2e328", "name": "Insert Review Queue (Auto)", - "type": "n8n-nodes-base.mySql", - "typeVersion": 2.4, + "type": "n8n-nodes-base.httpRequest", + "typeVersion": 4.1, "position": [33856, 13312], - "credentials": { - "mySql": { - "id": "CHHfbKhMacNo03V4", - "name": "MySQL account" - } - }, - "notes": "Auto Ready (confidence ≥ 0.85) → migration_review_queue PENDING" + "onError": "continueErrorOutput", + "notes": "POST /api/ai/migration/queue/record (PENDING) — ADR-023A" }, { "parameters": { - "operation": "executeQuery", - "query": "INSERT INTO migration_review_queue (document_number, title, original_title, ai_suggested_category, ai_confidence, ai_issues, review_reason, status, created_at) VALUES ('{{$json.document_number}}', '{{$json.ai_result.subject || $json.title}}', '{{$json.title}}', '{{$json.ai_result.suggested_category}}', {{$json.ai_result.confidence}}, '{{JSON.stringify({ job_id: $json.job_id, temp_attachment_id: $json.temp_attachment_id, type_code: $json.ai_result.type_code, subject: $json.ai_result.subject, suggested_tags: $json.ai_result.suggested_tags, key_points: $json.ai_result.key_points || [], issued_date: $json.ai_result.issued_date, received_date: $json.ai_result.received_date })}}', '{{$json.review_reason}}', 'PENDING_REVIEW', NOW()) ON DUPLICATE KEY UPDATE status = 'PENDING_REVIEW', review_reason = '{{$json.review_reason}}', ai_issues = VALUES(ai_issues), created_at = NOW()", + "method": "POST", + "url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/ai/migration/queue/record", + "sendHeaders": true, + "headerParameters": { + "parameters": [ + { + "name": "Authorization", + "value": "={{$('Set Configuration').first().json.config.MIGRATION_TOKEN}}" + } + ] + }, + "sendBody": true, + "specifyBody": "json", + "jsonBody": "={{ JSON.stringify({ batchId: $json.batch_id, documentNumber: $json.document_number, subject: ($json.ai_result?.subject || $json.title), originalSubject: $json.title, tempAttachmentId: $json.temp_attachment_id, confidence: $json.ai_result?.confidence, reviewReason: $json.review_reason || '', status: 'PENDING_REVIEW', aiResult: $json.ai_result, idempotencyKey: $json.batch_id + ':' + $json.document_number }) }}", "options": {} }, "id": "f1a2b3c4-d5e6-7890-abcd-567890123456", "name": "Insert Review Queue (Flagged)", - "type": "n8n-nodes-base.mySql", - "typeVersion": 2.4, + "type": "n8n-nodes-base.httpRequest", + "typeVersion": 4.1, "position": [33856, 13504], - "credentials": { - "mySql": { - "id": "CHHfbKhMacNo03V4", - "name": "MySQL account" - } - }, - "notes": "Flagged (confidence 0.60–0.84) → migration_review_queue PENDING_REVIEW" + "onError": "continueErrorOutput", + "notes": "POST /api/ai/migration/queue/record (PENDING_REVIEW) — ADR-023A" }, { "parameters": { @@ -428,41 +444,55 @@ }, { "parameters": { - "operation": "executeQuery", - "query": "INSERT INTO migration_errors (batch_id, document_number, error_type, error_message, created_at) VALUES ('{{$('Set Configuration').first().json.config.BATCH_ID}}', '{{$json.document_number}}', '{{$json.error_type || \"UNKNOWN\"}}', '{{$json.error || $json.parse_error}}', NOW())", + "method": "POST", + "url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/ai/migration/errors", + "sendHeaders": true, + "headerParameters": { + "parameters": [ + { + "name": "Authorization", + "value": "={{$('Set Configuration').first().json.config.MIGRATION_TOKEN}}" + } + ] + }, + "sendBody": true, + "specifyBody": "json", + "jsonBody": "={{ JSON.stringify({ batchId: $('Set Configuration').first().json.config.BATCH_ID, documentNumber: $json.document_number, errorType: $json.error_type || 'UNKNOWN', errorMessage: $json.error || $json.parse_error || '', jobId: $json.job_id || '' }) }}", "options": {} }, "id": "0f058ad0-3c09-4c9f-bdcf-503cd58ee395", "name": "Log Error to DB", - "type": "n8n-nodes-base.mySql", - "typeVersion": 2.4, + "type": "n8n-nodes-base.httpRequest", + "typeVersion": 4.1, "position": [34032, 13888], - "credentials": { - "mySql": { - "id": "CHHfbKhMacNo03V4", - "name": "MySQL account" - } - }, - "notes": "บันทึก Error ลง MariaDB" + "onError": "continueErrorOutput", + "notes": "POST /api/ai/migration/errors — ADR-023A" }, { "parameters": { - "operation": "executeQuery", - "query": "INSERT INTO migration_progress (batch_id, last_processed_index, status) VALUES ('{{$('Set Configuration').first().json.config.BATCH_ID}}', {{$json.original_index || 0}}, 'RUNNING') ON DUPLICATE KEY UPDATE last_processed_index = {{$json.original_index || 0}}, updated_at = NOW()", + "method": "POST", + "url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/ai/migration/checkpoint", + "sendHeaders": true, + "headerParameters": { + "parameters": [ + { + "name": "Authorization", + "value": "={{$('Set Configuration').first().json.config.MIGRATION_TOKEN}}" + } + ] + }, + "sendBody": true, + "specifyBody": "json", + "jsonBody": "={{ JSON.stringify({ batchId: $('Set Configuration').first().json.config.BATCH_ID, lastProcessedIndex: $json.original_index || 0, status: 'RUNNING' }) }}", "options": {} }, "id": "bb0e611b-db28-4266-ba40-3b5d534a16f7", "name": "Save Checkpoint", - "type": "n8n-nodes-base.mySql", - "typeVersion": 2.4, + "type": "n8n-nodes-base.httpRequest", + "typeVersion": 4.1, "position": [34032, 13312], - "credentials": { - "mySql": { - "id": "CHHfbKhMacNo03V4", - "name": "MySQL account" - } - }, - "notes": "บันทึก Checkpoint ทุก record ที่ผ่าน Review Queue" + "onError": "continueErrorOutput", + "notes": "POST /api/ai/migration/checkpoint — ADR-023A" }, { "parameters": {