From 2f0d67d8b27e6750e2073c354d0bbac126ce2377 Mon Sep 17 00:00:00 2001 From: admin Date: Tue, 17 Mar 2026 08:37:01 +0700 Subject: [PATCH] 260317:0837 Refactor to n8n --- .gemini/GEMINI.md | 10 +- AGENTS.md | 10 +- CHANGELOG.md | 51 ++++- CLAUDE.md | 10 +- README.md | 91 ++++---- .../dto/create-migration-error.dto.ts | 25 +++ .../migration/dto/enqueue-migration.dto.ts | 13 +- .../entities/migration-error.entity.ts | 2 + .../entities/migration-review-queue.entity.ts | 9 +- .../modules/migration/migration.controller.ts | 70 ++++-- .../modules/migration/migration.service.ts | 157 +++++++++----- .../admin/doc-control/reference/tags/page.tsx | 4 +- n8n-workflow-lcbp3.json | 201 ++++++++++-------- .../lcbp3-v1.8.0-migration.sql | 7 +- .../05-03-frontend-guidelines.md | 8 +- specs/05-Engineering-Guidelines/README.md | 16 +- specs/06-Decision-Records/README.md | 18 +- specs/README.md | 14 +- 18 files changed, 475 insertions(+), 241 deletions(-) create mode 100644 backend/src/modules/migration/dto/create-migration-error.dto.ts diff --git a/.gemini/GEMINI.md b/.gemini/GEMINI.md index 058c59e..263dcbe 100644 --- a/.gemini/GEMINI.md +++ b/.gemini/GEMINI.md @@ -14,12 +14,12 @@ You value **Data Integrity**, **Security**, and **Clean Architecture**. **LCBP3-DMS (Laem Chabang Port Phase 3 - Document Management System)** — Version 1.8.1 (Patch) -### 📊 Project Status: UAT Ready (2026-03-11) +### 📊 Project Status: UAT Ready (2026-03-16) | Area | Status | Notes | | ------------- | ------------------------ | ------------------------------------ | -| Backend | ✅ Production Ready | 18 Modules, ADR-018 AI Isolation | -| Frontend | ✅ 100% Complete | App Router, TanStack Query, Zustand | +| Backend | ✅ Production Ready | NestJS 11, Express v5, 18 Modules | +| Frontend | ✅ 100% Complete | Next.js 16, React 19, proxy.ts | | Database | ✅ Schema v1.8.0 Stable | MariaDB 11.8, No-migration (ADR-009) | | Documentation | ✅ **10/10 Gaps Closed** | Product Vision → Release Policy | | AI Migration | 🔄 Pre-migration Setup | n8n + Ollama (ADR-017/018) | @@ -36,9 +36,9 @@ You value **Data Integrity**, **Security**, and **Clean Architecture**. ## 💻 Tech Stack & Constraints -- **Backend:** NestJS (Modular Architecture), TypeORM, MariaDB 11.8, Redis 7.2 (BullMQ), +- **Backend:** NestJS 11 (Express v5, Modular Architecture), TypeORM, MariaDB 11.8, Redis 7.2 (BullMQ), Elasticsearch 8.11, JWT + Passport, CASL (4-Level RBAC), ClamAV (Virus Scanning), Helmet.js -- **Frontend:** Next.js 14+ (App Router), Tailwind CSS, Shadcn/UI, +- **Frontend:** Next.js 16 (App Router, proxy.ts), Tailwind CSS, Shadcn/UI, TanStack Query (**Server State**), Zustand (**Client State**), React Hook Form + Zod (**Form State**), Axios - **Notifications:** BullMQ Queue → Email / LINE Notify / In-App - **AI/Migration:** Ollama (llama3.2:3b / mistral:7b) on Admin Desktop (RTX 2060 SUPER) + n8n on QNAP diff --git a/AGENTS.md b/AGENTS.md index 6f10fec..5fe046d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,12 +12,12 @@ You value **Data Integrity**, **Security**, and **Clean Architecture**. **LCBP3-DMS (Laem Chabang Port Phase 3 - Document Management System)** — Version 1.8.1 (Patch) -### 📊 Project Status: UAT Ready (2026-03-11) +### 📊 Project Status: UAT Ready (2026-03-16) | Area | Status | Notes | | ------------- | ------------------------ | ------------------------------------ | -| Backend | ✅ Production Ready | 18 Modules, ADR-018 AI Isolation | -| Frontend | ✅ 100% Complete | App Router, TanStack Query, Zustand | +| Backend | ✅ Production Ready | NestJS 11, Express v5, 18 Modules | +| Frontend | ✅ 100% Complete | Next.js 16, React 19, proxy.ts | | Database | ✅ Schema v1.8.0 Stable | MariaDB 11.8, No-migration (ADR-009) | | Documentation | ✅ **10/10 Gaps Closed** | Product Vision → Release Policy | | AI Migration | 🔄 Pre-migration Setup | n8n + Ollama (ADR-017/018) | @@ -34,9 +34,9 @@ You value **Data Integrity**, **Security**, and **Clean Architecture**. ## 💻 Tech Stack & Constraints -- **Backend:** NestJS (Modular Architecture), TypeORM, MariaDB 11.8, Redis 7.2 (BullMQ), +- **Backend:** NestJS 11 (Express v5, Modular Architecture), TypeORM, MariaDB 11.8, Redis 7.2 (BullMQ), Elasticsearch 8.11, JWT + Passport, CASL (4-Level RBAC), ClamAV (Virus Scanning), Helmet.js -- **Frontend:** Next.js 14+ (App Router), Tailwind CSS, Shadcn/UI, +- **Frontend:** Next.js 16 (App Router, proxy.ts), Tailwind CSS, Shadcn/UI, TanStack Query (**Server State**), Zustand (**Client State**), React Hook Form + Zod (**Form State**), Axios - **Notifications:** BullMQ Queue → Email / LINE Notify / In-App - **AI/Migration:** Ollama (llama3.2:3b / mistral:7b) on Admin Desktop (RTX 2060 SUPER) + n8n on QNAP diff --git a/CHANGELOG.md b/CHANGELOG.md index 4e01447..b91387f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,49 +10,95 @@ - Final Security Audit — ตาม `04-06-security-operations.md` - Go-Live: Blue-Green Deploy บน QNAP Container Station +### NestJS 11 + Next.js 16 Migration (2026-03-16) + +#### Backend 🔧 + +- **NestJS 11 Upgrade**: `@nestjs/*` v11, Express v5, removed `@nestjs/mapped-types` +- **RequestWithUser Interface**: Shared typed interface replacing `req: any` across 6 controllers (`auth`, `session`, `file-storage`, `workflow-engine`, `correspondence`, `jwt-refresh`) +- **MasterModule DI Fix**: Imported `UserModule` so `RbacGuard` can resolve `UserService` +- **TypeORM Fix**: Explicit typing for `DocumentNumberFormat` save/create overload resolution +- **Swagger**: Updated API version to 1.8.1 + +#### Frontend 🎨 + +- **Next.js 16 Upgrade**: Next.js 16.0.7, React 19 +- **proxy.ts Rename**: `middleware.ts` → `proxy.ts` (Next.js 16 deprecated `middleware` convention) +- **ADR-019 UUID Fixes — Drawing Admin Pages (5 pages)**: + - `contract/volumes`, `contract/categories`, `contract/sub-categories` + - `shop/main-categories`, `shop/sub-categories` + - Changed `useState` → `useState`, removed `parseInt(uuid)` +- **ADR-019 UUID Fixes — Contract Page**: + - Fixed edit form showing "New Contract" (`contract.uuid` → `contract.id` after `@Expose`) + - Fixed blank Project dropdown (was using contract's UUID instead of project's UUID) + - Fixed delete passing `undefined` uuid + - Updated `Contract` interface to match serialized API response +- **ADR-019 UUID Fixes — Disciplines & RFA Types**: + - `useContracts(projectId=1)` → optional param, fetches all contracts when unspecified + - `useDisciplines(contractId?: number)` → `number | string` +- **ADR-019 UUID Fixes — Tags Page**: + - Fixed Radix Select crash on empty `value=""` → `"__none__"` sentinel + - Fixed Project Scope dropdown showing UUIDs (snake_case → camelCase field names) +- **DTO Updates**: `CreateContractDto`, `SearchContractDto`, `CreateShopMainCategoryDto`, etc. — `projectId: number` → `number | string` +- **Service Updates**: `drawing-master-data.service.ts`, `master-data.service.ts` — accept `string | number` for project/contract IDs + +#### Documentation 📚 + +- **ADR-005**: Updated Backend Stack table (NestJS 11, Express v5) +- **Engineering Guidelines** (`05-01`): Added NestJS 11 specific patterns section +- **README.md**: Tech stack versions, status date, ADR-019 in roadmap +- **AGENTS.md**: Tech stack versions updated + --- ## 1.8.1 Patch (2026-03-11) ### Summary -**Product Owner Documentation Complete** — ปิด 10/10 Documentation Gaps สำหรับ UAT Readiness +**Product Owner Documentation Complete** — ปิด 10/10 Documentation Gaps สำหรับ UAT Readiness ระบบมีเอกสารครบถ้วนสำหรับ Stakeholder Sign-off และ Go-Live Process ### Documentation 📚 — 10/10 Gaps Closed #### Gap 1: Product Vision Statement ✅ `specs/00-Overview/00-03-product-vision.md` + - Elevator Pitch, Problem Statement, Geoffrey Moore Vision Format - Strategic Pillars: Speed / Security / Visibility - Phase Roadmap (Now → 24 เดือน), Guardrails, Success Metrics #### Gap 2: User Stories ✅ `specs/01-Requirements/01-04-user-stories.md` + - 27 User Stories ครอบคลุม 8 Epics - MoSCoW Prioritization per Story - Acceptance Criteria + Definition of Done #### Gap 3: Acceptance Criteria (UAT) ✅ `specs/01-Requirements/01-05-acceptance-criteria.md` + - 35 Acceptance Criteria (All Modules) - UAT Plan: 4 Phases, Sign-off Process - Go-Live Criteria Matrix #### Gap 4: UI/UX Wireframes ✅ `specs/01-Requirements/01-07-ui-wireframes.md` + - Screen Inventory: 26 Screens พร้อม Role + Priority - Navigation Map / Site Map ครบทุก Route - ASCII Wireframes: Login, Dashboard, Correspondence, RFA, Circulation, Admin - Design System Tokens + Interaction Patterns #### Gap 5: Stakeholder Sign-off & Risk ✅ `specs/00-Overview/00-04-stakeholder-signoff-and-risk.md` + - Sign-off Process 4-Step + Digital Sign Matrix - Risk Register: 15 Risks (Impact × Probability Matrix) - Change Control Policy + Emergency Change Process #### Gap 6: KPI Baseline Data ✅ `specs/00-Overview/00-05-kpi-baseline.md` + - 14 KPIs พร้อม Baseline Collection Form - SQL Measurement Queries + Grafana Dashboard Specs - User Satisfaction Survey Template #### Gap 7: Migration Business Scope ✅ `specs/03-Data-and-Storage/03-06-migration-business-scope.md` + - Data Scope: IN/OUT SCOPE (ปี 2564 → Go-Live) - Migration Tiers: Tier 1 (2K Critical) / Tier 2 (10K) / Tier 3 (8K Archive) - Excel Metadata Mapping (11 Columns → Field ใหม่) @@ -62,6 +108,7 @@ - Data Security: AI Isolation (ADR-018) + Token 7 วัน + IP Whitelist #### Gap 8: Release Management Policy ✅ `specs/04-Infrastructure-OPS/04-08-release-management-policy.md` + - SemVer Strategy + Git Flow (main/release/develop/hotfix) - 5 Release Gates: Code Complete → QA → Staging → Approval → Production - Quality Thresholds: TS 0 errors, ≥80% Test Coverage, 0 Critical Vuln @@ -71,11 +118,13 @@ - Release Checklist + Security Pre-release Requirements #### Gap 9: Training Plan ✅ `specs/00-Overview/00-06-training-plan.md` + - Curriculum แบ่งตาม Role (4 Roles) - 4-Phase Training Timeline - Hands-on Lab + Assessment Criteria #### Gap 10: Edge Cases & Business Rules ✅ `specs/01-Requirements/01-06-edge-cases-and-rules.md` + - 37 Edge Cases ครอบคลุมทุก Module - Business Logic Guards + Error Handling Matrix diff --git a/CLAUDE.md b/CLAUDE.md index e9bf9a4..d259ba5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,12 +10,12 @@ You value **Data Integrity**, **Security**, and **Clean Architecture**. **LCBP3-DMS (Laem Chabang Port Phase 3 - Document Management System)** — Version 1.8.1 (Patch) -### 📊 Project Status: UAT Ready (2026-03-11) +### 📊 Project Status: UAT Ready (2026-03-16) | Area | Status | Notes | | ------------- | ------------------------ | ------------------------------------ | -| Backend | ✅ Production Ready | 18 Modules, ADR-018 AI Isolation | -| Frontend | ✅ 100% Complete | App Router, TanStack Query, Zustand | +| Backend | ✅ Production Ready | NestJS 11, Express v5, 18 Modules | +| Frontend | ✅ 100% Complete | Next.js 16, React 19, proxy.ts | | Database | ✅ Schema v1.8.0 Stable | MariaDB 11.8, No-migration (ADR-009) | | Documentation | ✅ **10/10 Gaps Closed** | Product Vision → Release Policy | | AI Migration | 🔄 Pre-migration Setup | n8n + Ollama (ADR-017/018) | @@ -32,9 +32,9 @@ You value **Data Integrity**, **Security**, and **Clean Architecture**. ## 💻 Tech Stack & Constraints -- **Backend:** NestJS (Modular Architecture), TypeORM, MariaDB 11.8, Redis 7.2 (BullMQ), +- **Backend:** NestJS 11 (Express v5, Modular Architecture), TypeORM, MariaDB 11.8, Redis 7.2 (BullMQ), Elasticsearch 8.11, JWT + Passport, CASL (4-Level RBAC), ClamAV (Virus Scanning), Helmet.js -- **Frontend:** Next.js 14+ (App Router), Tailwind CSS, Shadcn/UI, +- **Frontend:** Next.js 16 (App Router, proxy.ts), Tailwind CSS, Shadcn/UI, TanStack Query (**Server State**), Zustand (**Client State**), React Hook Form + Zod (**Form State**), Axios - **Notifications:** BullMQ Queue → Email / LINE Notify / In-App - **AI/Migration:** Ollama (llama3.2:3b / mistral:7b) on Admin Desktop (RTX 2060 SUPER) + n8n on QNAP diff --git a/README.md b/README.md index cb555f8..1d08574 100644 --- a/README.md +++ b/README.md @@ -10,19 +10,19 @@ --- -## 📈 Current Status (As of 2026-03-11) +## 📈 Current Status (As of 2026-03-16) **Version 1.8.1 (Patch) — UAT Ready** -| Area | Status | หมายเหตุ | -|------|--------|--------| -| 🔧 **Backend** | ✅ Production Ready | 18 Modules, ADR-018 AI Isolation | -| 🎨 **Frontend** | ✅ 100% Complete | App Router, TanStack Query, Zustand | -| 💾 **Database** | ✅ Schema v1.8.0 Stable | MariaDB 11.8, No-migration Policy | -| 📘 **Documentation** | ✅ **10/10 Gaps Closed** | Product Vision → Release Policy | -| 🤖 **AI Migration** | 🔄 Pre-migration Setup | n8n + Ollama (ADR-017/018) | -| 🧪 **Testing** | 🔄 UAT Preparation | E2E + Acceptance Criteria ready | -| 🚀 **Deployment** | 📋 Pending Go-Live Gate | Blue-Green on QNAP Container Station | +| Area | Status | หมายเหตุ | +| -------------------- | ------------------------ | ------------------------------------ | +| 🔧 **Backend** | ✅ Production Ready | NestJS 11, Express v5, 18 Modules | +| 🎨 **Frontend** | ✅ 100% Complete | Next.js 16, React 19, TanStack Query | +| 💾 **Database** | ✅ Schema v1.8.0 Stable | MariaDB 11.8, No-migration Policy | +| 📘 **Documentation** | ✅ **10/10 Gaps Closed** | Product Vision → Release Policy | +| 🤖 **AI Migration** | 🔄 Pre-migration Setup | n8n + Ollama (ADR-017/018) | +| 🧪 **Testing** | 🔄 UAT Preparation | E2E + Acceptance Criteria ready | +| 🚀 **Deployment** | 📋 Pending Go-Live Gate | Blue-Green on QNAP Container Station | --- @@ -57,7 +57,7 @@ LCBP3-DMS เป็นระบบบริหารจัดการเอก ```typescript { - "framework": "NestJS (TypeScript, ESM)", + "framework": "NestJS 11 (TypeScript, Express v5)", "database": "MariaDB 11.8", "orm": "TypeORM", "authentication": "JWT + Passport", @@ -75,7 +75,7 @@ LCBP3-DMS เป็นระบบบริหารจัดการเอก ```typescript { - "framework": "Next.js 14+ (App Router)", + "framework": "Next.js 16 (App Router, proxy.ts)", "language": "TypeScript", "styling": "Tailwind CSS", "components": "shadcn/ui", @@ -219,7 +219,7 @@ lcbp3-dms/ │ │ ├── common/ # Shared utilities, guards, decorators │ │ ├── config/ # Configuration module │ │ ├── database/ # Database entities & migrations -│ │ ├── modules/ # Feature modules (17 modules) +│ │ ├── modules/ # Feature modules (18 modules) │ │ │ ├── auth/ # JWT Authentication │ │ │ ├── user/ # User management & RBAC │ │ │ ├── project/ # Project & Contract management @@ -298,6 +298,7 @@ lcbp3-dms/ │ ├── .gemini/ # 🤖 AI agent configuration ├── .agents/ # Agent workflows and tools +├── AGENTS.md # AI agent rules & project context ├── GEMINI.md # AI coding guidelines ├── CONTRIBUTING.md # Contribution guidelines ├── CHANGELOG.md # Version history @@ -310,21 +311,21 @@ lcbp3-dms/ ### เอกสารหลัก (specs/ folder) -| เอกสาร | คำอธิบาย | Gap | ไฟล์หลัก | -|--------|---------|-----|--------| -| **Product Vision** | Vision, Strategic Pillars, Guardrails | Gap 1 ✅ | `00-03-product-vision.md` | -| **User Stories** | 27 Stories, 8 Epics, MoSCoW | Gap 2 ✅ | `01-04-user-stories.md` | -| **Acceptance Criteria** | UAT Criteria, Sign-off Process | Gap 3 ✅ | `01-05-acceptance-criteria.md` | -| **UI/UX Wireframes** | 26 Screens, ASCII Wireframes, Design System | Gap 4 ✅ | `01-07-ui-wireframes.md` | -| **Stakeholder & Risk** | Sign-off, Risk Register, Change Control | Gap 5 ✅ | `00-04-stakeholder-signoff-and-risk.md` | -| **KPI Baseline** | 14 KPIs, SQL Queries, Grafana Specs | Gap 6 ✅ | `00-05-kpi-baseline.md` | -| **Migration Scope** | 20K Docs, 3 Tiers, Go/No-Go Gates | Gap 7 ✅ | `03-06-migration-business-scope.md` | -| **Release Policy** | SemVer, 5 Gates, Hotfix, Rollback | Gap 8 ✅ | `04-08-release-management-policy.md` | -| **Training Plan** | Curriculum per Role, UAT Training | Gap 9 ✅ | `00-06-training-plan.md` | -| **Edge Cases & Rules** | 37 Edge Cases, Business Logic Guards | Gap 10 ✅ | `01-06-edge-cases-and-rules.md` | -| **Schema v1.8.0** | Tables, Views, Indexes (3-file split) | — | `lcbp3-v1.8.0-schema-*.sql` | -| **Data Dictionary** | Field Meanings, Business Rules | — | `03-01-data-dictionary.md` | -| **ADRs (17+1)** | All Architecture Decisions incl. ADR-018 | — | `06-Decision-Records/` | +| เอกสาร | คำอธิบาย | Gap | ไฟล์หลัก | +| ----------------------- | -------------------------------------------- | --------- | --------------------------------------- | +| **Product Vision** | Vision, Strategic Pillars, Guardrails | Gap 1 ✅ | `00-03-product-vision.md` | +| **User Stories** | 27 Stories, 8 Epics, MoSCoW | Gap 2 ✅ | `01-04-user-stories.md` | +| **Acceptance Criteria** | UAT Criteria, Sign-off Process | Gap 3 ✅ | `01-05-acceptance-criteria.md` | +| **UI/UX Wireframes** | 26 Screens, ASCII Wireframes, Design System | Gap 4 ✅ | `01-07-ui-wireframes.md` | +| **Stakeholder & Risk** | Sign-off, Risk Register, Change Control | Gap 5 ✅ | `00-04-stakeholder-signoff-and-risk.md` | +| **KPI Baseline** | 14 KPIs, SQL Queries, Grafana Specs | Gap 6 ✅ | `00-05-kpi-baseline.md` | +| **Migration Scope** | 20K Docs, 3 Tiers, Go/No-Go Gates | Gap 7 ✅ | `03-06-migration-business-scope.md` | +| **Release Policy** | SemVer, 5 Gates, Hotfix, Rollback | Gap 8 ✅ | `04-08-release-management-policy.md` | +| **Training Plan** | Curriculum per Role, UAT Training | Gap 9 ✅ | `00-06-training-plan.md` | +| **Edge Cases & Rules** | 37 Edge Cases, Business Logic Guards | Gap 10 ✅ | `01-06-edge-cases-and-rules.md` | +| **Schema v1.8.0** | Tables, Views, Indexes (3-file split) | — | `lcbp3-v1.8.0-schema-*.sql` | +| **Data Dictionary** | Field Meanings, Business Rules | — | `03-01-data-dictionary.md` | +| **ADRs (17+2)** | All Architecture Decisions incl. ADR-018/019 | — | `06-Decision-Records/` | ### Schema & Seed Data (v1.8.0) @@ -562,22 +563,32 @@ This project is **Internal Use Only** - ลิขสิทธิ์เป็น **10/10 Documentation Gaps Closed:** -| Gap | เอกสาร | สถานะ | -|-----|--------|------| -| 1 | Product Vision Statement | ✅ | -| 2 | User Stories (27 Stories, 8 Epics) | ✅ | -| 3 | Acceptance Criteria & UAT Plan | ✅ | -| 4 | UI/UX Wireframes (26 Screens) | ✅ | -| 5 | Stakeholder Sign-off & Risk Register | ✅ | -| 6 | KPI Baseline Data (14 KPIs) | ✅ | -| 7 | Migration Business Scope (20K Docs) | ✅ | -| 8 | Release Management Policy (SemVer + Gates) | ✅ | -| 9 | Training Plan (per Role, 4 phases) | ✅ | -| 10 | Edge Cases & Business Rules (37 rules) | ✅ | +| Gap | เอกสาร | สถานะ | +| --- | ------------------------------------------ | ----- | +| 1 | Product Vision Statement | ✅ | +| 2 | User Stories (27 Stories, 8 Epics) | ✅ | +| 3 | Acceptance Criteria & UAT Plan | ✅ | +| 4 | UI/UX Wireframes (26 Screens) | ✅ | +| 5 | Stakeholder Sign-off & Risk Register | ✅ | +| 6 | KPI Baseline Data (14 KPIs) | ✅ | +| 7 | Migration Business Scope (20K Docs) | ✅ | +| 8 | Release Management Policy (SemVer + Gates) | ✅ | +| 9 | Training Plan (per Role, 4 phases) | ✅ | +| 10 | Edge Cases & Business Rules (37 rules) | ✅ | - ✅ ADR-018: AI Boundary (Ollama Isolation มี No Direct DB/Storage Access) +- ✅ ADR-019: Hybrid Identifier Strategy (INT PK + UUIDv7 Public API) - ✅ Migration n8n Workflow + AI Isolation Plan +### ✅ NestJS 11 + Next.js 16 Migration (Mar 2026) + +- ✅ Backend upgraded to **NestJS 11** (Express v5, `@nestjs/*` v11) +- ✅ Shared `RequestWithUser` typed interface (replaced `req: any` across 6 controllers) +- ✅ Frontend upgraded to **Next.js 16** (React 19) +- ✅ Renamed `middleware.ts` → `proxy.ts` (Next.js 16 convention) +- ✅ ADR-019 UUID fixes: Drawing admin pages (5), Contracts, Disciplines, Tags, RFA Types +- ✅ Fixed contract edit form (UUID mismatch), disciplines dropdown (hardcoded projectId), tags crash (empty Select value) + ### 🔄 Next: Go-Live Preparation - 🔄 **UAT**: ทำ User Acceptance Testing ตาม `01-05-acceptance-criteria.md` diff --git a/backend/src/modules/migration/dto/create-migration-error.dto.ts b/backend/src/modules/migration/dto/create-migration-error.dto.ts new file mode 100644 index 0000000..7b9dc4c --- /dev/null +++ b/backend/src/modules/migration/dto/create-migration-error.dto.ts @@ -0,0 +1,25 @@ +import { IsString, IsOptional, IsEnum } from 'class-validator'; +import { MigrationErrorType } from '../entities/migration-error.entity'; + +export class CreateMigrationErrorDto { + @IsString() + @IsOptional() + batch_id?: string; + + @IsString() + @IsOptional() + document_number?: string; + + @IsString() + @IsOptional() + @IsEnum(MigrationErrorType) + error_type?: MigrationErrorType; + + @IsString() + @IsOptional() + error_message?: string; + + @IsString() + @IsOptional() + raw_ai_response?: string; +} diff --git a/backend/src/modules/migration/dto/enqueue-migration.dto.ts b/backend/src/modules/migration/dto/enqueue-migration.dto.ts index 945ed64..c10fdd1 100644 --- a/backend/src/modules/migration/dto/enqueue-migration.dto.ts +++ b/backend/src/modules/migration/dto/enqueue-migration.dto.ts @@ -14,11 +14,15 @@ export class EnqueueMigrationDto { @IsString() @IsOptional() - title?: string; + subject?: string; @IsString() @IsOptional() - original_title?: string; + original_subject?: string; + + @IsString() + @IsOptional() + body?: string; @IsString() @IsOptional() @@ -54,7 +58,10 @@ export class EnqueueMigrationDto { @IsArray() @IsOptional() - extracted_tags?: any[]; + extracted_tags?: Record[]; + + @IsOptional() + details?: Record; @IsNumber() @IsOptional() diff --git a/backend/src/modules/migration/entities/migration-error.entity.ts b/backend/src/modules/migration/entities/migration-error.entity.ts index fd5ca0d..b4e0217 100644 --- a/backend/src/modules/migration/entities/migration-error.entity.ts +++ b/backend/src/modules/migration/entities/migration-error.entity.ts @@ -7,6 +7,8 @@ import { export enum MigrationErrorType { FILE_NOT_FOUND = 'FILE_NOT_FOUND', + MISSING_FILENAME = 'MISSING_FILENAME', + FILE_ERROR = 'FILE_ERROR', AI_PARSE_ERROR = 'AI_PARSE_ERROR', API_ERROR = 'API_ERROR', DB_ERROR = 'DB_ERROR', diff --git a/backend/src/modules/migration/entities/migration-review-queue.entity.ts b/backend/src/modules/migration/entities/migration-review-queue.entity.ts index b51b32d..4a1cf79 100644 --- a/backend/src/modules/migration/entities/migration-review-queue.entity.ts +++ b/backend/src/modules/migration/entities/migration-review-queue.entity.ts @@ -20,10 +20,13 @@ export class MigrationReviewQueue { documentNumber!: string; @Column({ type: 'text', nullable: true }) - title?: string; + subject?: string; - @Column({ name: 'original_title', type: 'text', nullable: true }) - originalTitle?: string; + @Column({ name: 'original_subject', type: 'text', nullable: true }) + originalSubject?: string; + + @Column({ type: 'text', nullable: true }) + body?: string; @Column({ name: 'ai_suggested_category', length: 50, nullable: true }) aiSuggestedCategory?: string; diff --git a/backend/src/modules/migration/migration.controller.ts b/backend/src/modules/migration/migration.controller.ts index e141751..2aed7bb 100644 --- a/backend/src/modules/migration/migration.controller.ts +++ b/backend/src/modules/migration/migration.controller.ts @@ -1,11 +1,30 @@ -import { Controller, Post, Body, Headers, UseGuards, Get, Param, Query, Res, ParseIntPipe } from '@nestjs/common'; +import { + Controller, + Post, + Body, + Headers, + UseGuards, + Get, + Param, + Query, + Res, + ParseIntPipe, +} from '@nestjs/common'; import { MigrationService } from './migration.service'; import { ImportCorrespondenceDto } from './dto/import-correspondence.dto'; import { EnqueueMigrationDto } from './dto/enqueue-migration.dto'; import { CommitBatchDto } from './dto/commit-batch.dto'; +import { CreateMigrationErrorDto } from './dto/create-migration-error.dto'; import { JwtAuthGuard } from '../../common/guards/jwt-auth.guard'; import { CurrentUser } from '../../common/decorators/current-user.decorator'; -import { ApiTags, ApiOperation, ApiBearerAuth, ApiHeader, ApiQuery, ApiParam } from '@nestjs/swagger'; +import { + ApiTags, + ApiOperation, + ApiBearerAuth, + ApiHeader, + ApiQuery, + ApiParam, +} from '@nestjs/swagger'; import { MigrationQueueQueryDto } from './dto/migration-queue-query.dto'; import type { Response } from 'express'; @@ -17,10 +36,13 @@ export class MigrationController { @Post('import') @UseGuards(JwtAuthGuard) - @ApiOperation({ summary: 'Import generic legacy correspondence record via n8n integration' }) + @ApiOperation({ + summary: 'Import generic legacy correspondence record via n8n integration', + }) @ApiHeader({ name: 'Idempotency-Key', - description: 'Unique key per document and batch to prevent duplicate inserts', + description: + 'Unique key per document and batch to prevent duplicate inserts', required: true, }) async importCorrespondence( @@ -29,15 +51,22 @@ export class MigrationController { @CurrentUser() user: any ) { const userId = user?.id || user?.userId || 5; - return this.migrationService.importCorrespondence(dto, idempotencyKey, userId); + return this.migrationService.importCorrespondence( + dto, + idempotencyKey, + userId + ); } @Post('commit_batch') @UseGuards(JwtAuthGuard) - @ApiOperation({ summary: 'Batch approve and import migration review queue items' }) + @ApiOperation({ + summary: 'Batch approve and import migration review queue items', + }) @ApiHeader({ name: 'Idempotency-Key', - description: 'Unique key for the entire batch to prevent duplicate execution', + description: + 'Unique key for the entire batch to prevent duplicate execution', required: true, }) async commitBatch( @@ -51,7 +80,9 @@ export class MigrationController { @Post('queue') @UseGuards(JwtAuthGuard) - @ApiOperation({ summary: 'Enqueue a record into the staging migration review queue' }) + @ApiOperation({ + summary: 'Enqueue a record into the staging migration review queue', + }) async enqueueRecord(@Body() dto: EnqueueMigrationDto) { return this.migrationService.enqueueRecord(dto); } @@ -71,6 +102,13 @@ export class MigrationController { return this.migrationService.getQueueItemById(id); } + @Post('errors') + @UseGuards(JwtAuthGuard) + @ApiOperation({ summary: 'Log a migration error from n8n workflow' }) + async createError(@Body() dto: CreateMigrationErrorDto) { + return this.migrationService.createError(dto); + } + @Get('errors') @UseGuards(JwtAuthGuard) @ApiOperation({ summary: 'Get migration errors' }) @@ -89,7 +127,8 @@ export class MigrationController { @ApiParam({ name: 'id', type: Number }) @ApiHeader({ name: 'Idempotency-Key', - description: 'Unique key per document and batch to prevent duplicate inserts', + description: + 'Unique key per document and batch to prevent duplicate inserts', required: true, }) async approveQueueItem( @@ -99,7 +138,12 @@ export class MigrationController { @CurrentUser() user: any ) { const userId = user?.id || user?.userId || 5; - return this.migrationService.approveQueueItem(id, dto, idempotencyKey, userId); + return this.migrationService.approveQueueItem( + id, + dto, + idempotencyKey, + userId + ); } @Post('queue/:id/reject') @@ -118,10 +162,7 @@ export class MigrationController { @UseGuards(JwtAuthGuard) @ApiOperation({ summary: 'Stream a file from staging' }) @ApiQuery({ name: 'path', required: true, type: String }) - async getStagingFile( - @Query('path') filePath: string, - @Res() res: Response - ) { + async getStagingFile(@Query('path') filePath: string, @Res() res: Response) { const stream = this.migrationService.getStagingFileStream(filePath); res.set({ 'Content-Type': 'application/pdf', @@ -130,4 +171,3 @@ export class MigrationController { stream.pipe(res); } } - diff --git a/backend/src/modules/migration/migration.service.ts b/backend/src/modules/migration/migration.service.ts index 59560ee..c85498c 100644 --- a/backend/src/modules/migration/migration.service.ts +++ b/backend/src/modules/migration/migration.service.ts @@ -10,6 +10,7 @@ import { Repository, DataSource } from 'typeorm'; import { ImportCorrespondenceDto } from './dto/import-correspondence.dto'; import { EnqueueMigrationDto } from './dto/enqueue-migration.dto'; import { CommitBatchDto } from './dto/commit-batch.dto'; +import { CreateMigrationErrorDto } from './dto/create-migration-error.dto'; import { ImportTransaction } from './entities/import-transaction.entity'; import { Correspondence } from '../correspondence/entities/correspondence.entity'; import { CorrespondenceRevision } from '../correspondence/entities/correspondence-revision.entity'; @@ -78,12 +79,21 @@ export class MigrationService { } // 2. Fetch Dependencies + // Alias map: n8n AI categories → correspondence_types.type_code + const CATEGORY_ALIAS: Record = { + Correspondence: 'LETTER', + Letter: 'LETTER', + Drawing: 'OTHER', + Report: 'OTHER', + Other: 'OTHER', + }; + const type = await this.correspondenceTypeRepo.findOne({ where: { typeName: dto.category }, }); // If exact name isn't found, try typeCode just in case - const typeId = type + let typeId = type ? type.id : ( await this.correspondenceTypeRepo.findOne({ @@ -91,6 +101,15 @@ export class MigrationService { }) )?.id; + // Third-level fallback: resolve via alias map + if (!typeId && dto.category && CATEGORY_ALIAS[dto.category]) { + typeId = ( + await this.correspondenceTypeRepo.findOne({ + where: { typeCode: CATEGORY_ALIAS[dto.category] }, + }) + )?.id; + } + if (!typeId) { throw new BadRequestException( `Category "${dto.category}" not found in system.` @@ -152,9 +171,9 @@ export class MigrationService { // --- CTI: insert RFA class --- if (isRFA) { // Default RFA type generic mapping - const rfaTypeRes = (await queryRunner.manager.query( + const rfaTypeRes = await queryRunner.manager.query( "SELECT id FROM rfa_types WHERE type_code = 'GEN' LIMIT 1" - )) as Array<{ id: number }>; + ); const rfa = queryRunner.manager.create('Rfa', { id: correspondence.id, rfaTypeId: rfaTypeRes[0]?.id || 1, // fallback to id 1 @@ -190,8 +209,11 @@ export class MigrationService { { isTemporary: false } ); } catch (fileError: unknown) { - const errMsg = fileError instanceof Error ? fileError.message : String(fileError); - this.logger.warn(`Failed to update temp_file [id:${attachmentId}]: ${errMsg}`); + const errMsg = + fileError instanceof Error ? fileError.message : String(fileError); + this.logger.warn( + `Failed to update temp_file [id:${attachmentId}]: ${errMsg}` + ); } } else if (dto.source_file_path) { try { @@ -220,7 +242,8 @@ export class MigrationService { } const parsed = new Date(d); if (isNaN(parsed.getTime())) return undefined; - if (parsed.getFullYear() > 2100 || parsed.getFullYear() < 1900) return undefined; + if (parsed.getFullYear() > 2100 || parsed.getFullYear() < 1900) + return undefined; return parsed; }; @@ -269,9 +292,9 @@ export class MigrationService { // --- CTI: insert RfaRevision --- if (isRFA) { // Map Status code to RFA Equivalent 'APP' (Approved) if exist, or id 3 (typically Approved) - const rfaStatusRes = (await queryRunner.manager.query( + const rfaStatusRes = await queryRunner.manager.query( "SELECT id FROM rfa_status_codes WHERE status_code = 'APP' LIMIT 1" - )) as Array<{ id: number }>; + ); const rfaRev = queryRunner.manager.create('RfaRevision', { id: revision.id, @@ -306,19 +329,19 @@ export class MigrationService { if (!tagName) continue; // Find or create Tag - const tagRes = (await queryRunner.manager.query( + const tagRes = await queryRunner.manager.query( 'SELECT id FROM tags WHERE project_id = ? AND tag_name = ? LIMIT 1', [project.id, tagName] - )) as Array<{ id: number }>; + ); let tagId: number; if (tagRes && tagRes.length > 0) { tagId = tagRes[0].id; } else { - const insertRes = (await queryRunner.manager.query( + const insertRes = await queryRunner.manager.query( "INSERT INTO tags (project_id, tag_name, color_code, created_by) VALUES (?, ?, 'default', ?)", [project.id, tagName, userId] - )) as { insertId: number }; + ); tagId = insertRes.insertId; } @@ -384,7 +407,10 @@ export class MigrationService { // Determine status based on confidence policy in ADR-017 let autoStatus = MigrationReviewStatus.PENDING; - if (dto.is_valid === false || (dto.confidence != null && dto.confidence < 0.60)) { + if ( + dto.is_valid === false || + (dto.confidence != null && dto.confidence < 0.6) + ) { autoStatus = MigrationReviewStatus.REJECTED; } @@ -399,8 +425,9 @@ export class MigrationService { }); } - queueItem.title = dto.title; - queueItem.originalTitle = dto.original_title; + queueItem.subject = dto.subject; + queueItem.originalSubject = dto.original_subject; + queueItem.body = dto.body; queueItem.aiSuggestedCategory = dto.category; queueItem.aiConfidence = dto.confidence; queueItem.aiIssues = dto.ai_issues; @@ -424,7 +451,9 @@ export class MigrationService { await this.reviewQueueRepo.save(queueItem); - this.logger.log(`Enqueued document [${dto.document_number}] to staging queue with status [${autoStatus}]`); + this.logger.log( + `Enqueued document [${dto.document_number}] to staging queue with status [${autoStatus}]` + ); return { message: 'Document enqueued successfully', @@ -441,7 +470,7 @@ export class MigrationService { if (status) { queryBuilder.where('queue.status = :status', { status }); } - + queryBuilder.orderBy('queue.createdAt', 'DESC'); queryBuilder.skip(skip).take(limit); @@ -464,6 +493,21 @@ export class MigrationService { return item; } + async createError(dto: CreateMigrationErrorDto) { + const error = this.errorRepo.create({ + batchId: dto.batch_id, + documentNumber: dto.document_number, + errorType: dto.error_type, + errorMessage: dto.error_message, + rawAiResponse: dto.raw_ai_response, + }); + const saved = await this.errorRepo.save(error); + this.logger.warn( + `Migration error logged [${dto.error_type}] for doc [${dto.document_number}] batch [${dto.batch_id}]` + ); + return { message: 'Error logged', id: saved.id }; + } + async getErrors(page: number = 1, limit: number = 10) { const skip = (page - 1) * limit; @@ -482,14 +526,21 @@ export class MigrationService { }; } - async approveQueueItem(id: number, dto: ImportCorrespondenceDto, idempotencyKey: string, userId: number) { + async approveQueueItem( + id: number, + dto: ImportCorrespondenceDto, + idempotencyKey: string, + userId: number + ) { const queueItem = await this.reviewQueueRepo.findOne({ where: { id } }); if (!queueItem) { throw new BadRequestException(`Queue item ${id} not found`); } if (queueItem.status !== MigrationReviewStatus.PENDING) { - throw new BadRequestException(`Queue item ${id} is already ${queueItem.status}`); + throw new BadRequestException( + `Queue item ${id} is already ${queueItem.status}` + ); } // Attempt the import @@ -504,42 +555,53 @@ export class MigrationService { return result; } - async commitBatch(dto: CommitBatchDto, idempotencyKey: string, userId: number) { + async commitBatch( + dto: CommitBatchDto, + idempotencyKey: string, + userId: number + ) { if (!idempotencyKey) { throw new BadRequestException('Idempotency-Key header is required'); } const results = []; const errors = []; - - // We let each import have its own transaction via approveQueueItem + + // We let each import have its own transaction via approveQueueItem // to avoid one bad record failing the entire batch of valid ones. - + for (const item of dto.items) { - // Create a unique sub-key for each item to avoid idempotency conflicts - // when using a batch idempotency key. - const subKey = `${idempotencyKey}_${item.queueId}`; - - // Force batchId on the item dto - item.dto.batch_id = dto.batchId; - - try { - const result = await this.approveQueueItem(item.queueId, item.dto, subKey, userId); - results.push({ queueId: item.queueId, result }); - } catch (err: unknown) { - const errorMessage = err instanceof Error ? err.message : String(err); - errors.push({ queueId: item.queueId, error: errorMessage }); - this.logger.error(`Batch commit failed for queue ID ${item.queueId}: ${errorMessage}`); - } + // Create a unique sub-key for each item to avoid idempotency conflicts + // when using a batch idempotency key. + const subKey = `${idempotencyKey}_${item.queueId}`; + + // Force batchId on the item dto + item.dto.batch_id = dto.batchId; + + try { + const result = await this.approveQueueItem( + item.queueId, + item.dto, + subKey, + userId + ); + results.push({ queueId: item.queueId, result }); + } catch (err: unknown) { + const errorMessage = err instanceof Error ? err.message : String(err); + errors.push({ queueId: item.queueId, error: errorMessage }); + this.logger.error( + `Batch commit failed for queue ID ${item.queueId}: ${errorMessage}` + ); + } } - + return { - message: 'Batch processing completed', - batchId: dto.batchId, - processed: results.length, - failed: errors.length, - results, - errors + message: 'Batch processing completed', + batchId: dto.batchId, + processed: results.length, + failed: errors.length, + results, + errors, }; } @@ -562,15 +624,14 @@ export class MigrationService { getStagingFileStream(filePath: string) { if (!filePath) { - throw new BadRequestException('File path is required'); + throw new BadRequestException('File path is required'); } const resolvedPath = path.resolve(filePath); if (!existsSync(resolvedPath)) { - throw new BadRequestException('File not found at specified path'); + throw new BadRequestException('File not found at specified path'); } return createReadStream(resolvedPath); } } - diff --git a/frontend/app/(admin)/admin/doc-control/reference/tags/page.tsx b/frontend/app/(admin)/admin/doc-control/reference/tags/page.tsx index be3da5b..cd3dbba 100644 --- a/frontend/app/(admin)/admin/doc-control/reference/tags/page.tsx +++ b/frontend/app/(admin)/admin/doc-control/reference/tags/page.tsx @@ -16,7 +16,7 @@ export default function TagsPage() { const projectOptions = [ { label: "Global (All Projects)", value: "__none__" }, ...(projectsData || []).map((p: Record) => ({ - label: p.project_name || p.project_code || `Project ${p.id}`, + label: p.projectName || p.projectCode || p.project_name || p.project_code || `Project ${p.id}`, value: String(p.id), })), ]; @@ -29,7 +29,7 @@ export default function TagsPage() { const pId = row.original.project_id; if (!pId) return Global; const p = (projectsData || []).find((proj: Record) => proj.id === pId); - return p ? (p.project_name || p.project_code || `Project ${pId}`) as React.ReactNode : pId as React.ReactNode; + return p ? (p.projectName || p.projectCode || p.project_name || p.project_code || `Project ${pId}`) as React.ReactNode : pId as React.ReactNode; }, }, { diff --git a/n8n-workflow-lcbp3.json b/n8n-workflow-lcbp3.json index dd2d9c3..18c952a 100644 --- a/n8n-workflow-lcbp3.json +++ b/n8n-workflow-lcbp3.json @@ -38,23 +38,23 @@ }, "options": {} }, - "id": "4609ab68-f7e4-4800-ad39-19ce32de60d0", + "id": "8ae8102d-0de5-4646-87c9-ed4bb619614d", "name": "Form Trigger", "type": "n8n-nodes-base.formTrigger", "typeVersion": 2.2, - "position": [31024, 13504], - "webhookId": "e164a362-0c6b-4243-a5ad-b325aa943f4f", + "position": [-1360, -27472], + "webhookId": "5cb2ee58-164a-4db4-a107-46cf1a51009f", "notes": "เปิด URL เพื่อเลือก Model ก่อนรัน" }, { "parameters": { "jsCode": "// Read model selected from Form Trigger dropdown\nconst formData = $('Form Trigger').first()?.json || {};\nconst selectedModelLabel = String(formData['Ollama Model (Primary)'] || '');\n\n// Extract just the model ID (before the space in the label)\nconst MODEL_MAP = {\n 'qwen2.5:7b-instruct-q4_K_M (สมดุล - แนะนำ)': 'qwen2.5:7b-instruct-q4_K_M',\n 'scb10x/typhoon2.1-gemma3-4b (เร็ว + ไทยดี)': 'scb10x/typhoon2.1-gemma3-4b',\n 'promptnow/openthaigpt1.5-7b-instruct-q4_k_m (ไทยเฉพาะทาง)': 'promptnow/openthaigpt1.5-7b-instruct-q4_k_m'\n};\nconst selectedModel = MODEL_MAP[selectedModelLabel] || 'scb10x/typhoon2.1-gemma3-4b';\n\nconst batchSizeInput = parseInt(formData['Batch Size'] || '0');\nconst excelFileInput = String(formData['Excel File Path'] || '').trim();\n\nconst CONFIG = {\n // Ollama Settings\n OLLAMA_HOST: 'http://192.168.20.100:11434',\n // Model selected from Form UI\n OLLAMA_MODEL_PRIMARY: selectedModel,\n // Fallback\n OLLAMA_MODEL_FALLBACK: 'mistral:7b-instruct-q4_K_M',\n \n // Backend Settings\n BACKEND_URL: 'https://backend.np-dms.work',\n MIGRATION_TOKEN: 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6Im1pZ3JhdGlvbl9ib3QiLCJzdWIiOjUsInNjb3BlIjoiR2xvYmFsIiwiaWF0IjoxNzcyNzc0MzI5LCJleHAiOjQ5Mjg1MzQzMjl9.TtA8zoHy7G9J5jPgYQPv7yw-9X--B_hl-Nv-c9V4PaA',\n \n // Batch Settings\n BATCH_SIZE: batchSizeInput > 0 ? batchSizeInput : 2,\n BATCH_ID: (() => { const d = new Date(Date.now() + 7 * 3600000); const s = d.toISOString(); return s.substring(0,10).replace(/-/g,'') + ':' + s.substring(11,16).replace(/:/g,''); })(),\n DELAY_MS: 2000,\n \n // Thresholds\n CONFIDENCE_HIGH: 0.85,\n CONFIDENCE_LOW: 0.60,\n MAX_RETRY: 3,\n FALLBACK_THRESHOLD: 5,\n \n // Source Definitions - แก้ไขโฟลเดอร์และไฟล์ทำงานที่นี่\n EXCEL_FILE: excelFileInput || '/home/node/.n8n-files/staging_ai/C22024.xlsx',\n SOURCE_PDF_DIR: '/home/node/.n8n-files/staging_ai/Incoming/08C.2/2567',\n LOG_PATH: '/home/node/.n8n-files/migration_logs',\n PROJECT_ID: 1\n};\n\nreturn { config: CONFIG };" }, - "id": "8f1d3378-cca6-48b6-99db-693e46ac81ef", + "id": "f6d94e21-daa6-4dcc-ba37-e822ded168d6", "name": "Set Configuration", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [31216, 13504], + "position": [-1184, -27584], "notes": "กำหนดค่า Configuration ทั้งหมด - แก้ไขที่นี่ก่อนรัน" }, { @@ -73,11 +73,11 @@ "timeout": 10000 } }, - "id": "6c6679b4-85f3-4c2c-ac8e-4281d6ae61f6", + "id": "9f8bd98a-997d-4471-a80d-d93abe64888f", "name": "Fetch Categories", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4.1, - "position": [31216, 13696], + "position": [-1184, -27392], "notes": "ดึง Categories จาก Backend" }, { @@ -96,11 +96,11 @@ "timeout": 10000 } }, - "id": "98b9159a-f21d-4b33-9524-058a78ccfc93", + "id": "c40d67a5-de33-4898-8d2d-cf77bd89fa20", "name": "Fetch Tags", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4.1, - "position": [31392, 13696], + "position": [-1008, -27392], "notes": "ดึง Tags ที่มีอยู่แล้วจาก Backend" }, { @@ -110,23 +110,23 @@ "timeout": 5000 } }, - "id": "60e81de6-e9b2-4bff-afcc-bef9d5b959b5", + "id": "02ef2241-b2b9-436e-98b2-7bbad7da67e6", "name": "Check Backend Health", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4.1, - "position": [31392, 13504], + "position": [-816, -27584], "onError": "continueErrorOutput", "notes": "ตรวจสอบ Backend พร้อมใช้งาน" }, { "parameters": { - "jsCode": "const fs = require('fs');\nconst config = $('Set Configuration').first().json.config;\n\n// Check file mount and inputs\ntry {\n if (!fs.existsSync(config.EXCEL_FILE)) {\n throw new Error(`Excel file not found at: ${config.EXCEL_FILE}`);\n }\n if (!fs.existsSync(config.SOURCE_PDF_DIR)) {\n throw new Error(`PDF Source directory not found at: ${config.SOURCE_PDF_DIR}`);\n }\n \n const files = fs.readdirSync(config.SOURCE_PDF_DIR);\n \n // Check write permission to log path\n if (!fs.existsSync(config.LOG_PATH)) {\n fs.mkdirSync(config.LOG_PATH, { recursive: true });\n }\n fs.writeFileSync(`${config.LOG_PATH}/.preflight_ok`, new Date().toISOString());\n \n // Grab categories out of the previous node (Fetch Categories) if available\n let categories = ['Correspondence','RFA','Drawing','Transmittal','Report','Other'];\n try {\n const upstreamData = $('Fetch Categories').first()?.json?.data;\n if (upstreamData && Array.isArray(upstreamData)) {\n categories = upstreamData.map(c => c.name || c.type || c); \n }\n } catch(e) {}\n \n // Grab existing tags from Fetch Tags node\n let existingTags = [];\n try {\n const tagData = $('Fetch Tags').first()?.json?.data || [];\n existingTags = Array.isArray(tagData) ? tagData.map(t => t.tag_name || t.name || '').filter(Boolean) : [];\n } catch(e) {}\n \n return [{ json: { \n preflight_ok: true, \n pdf_count_in_source: files.length,\n excel_target: config.EXCEL_FILE,\n system_categories: categories,\n existing_tags: existingTags,\n timestamp: new Date().toISOString()\n }}];\n} catch (err) {\n throw new Error(`Pre-flight check failed: ${err.message}`);\n}" + "jsCode": "const fs = require('fs');\nconst config = $('Set Configuration').first().json.config;\n\n// Check file mount and inputs\ntry {\n if (!fs.existsSync(config.EXCEL_FILE)) {\n throw new Error(`Excel file not found at: ${config.EXCEL_FILE}`);\n }\n if (!fs.existsSync(config.SOURCE_PDF_DIR)) {\n throw new Error(`PDF Source directory not found at: ${config.SOURCE_PDF_DIR}`);\n }\n \n const files = fs.readdirSync(config.SOURCE_PDF_DIR);\n \n // Check write permission to log path\n if (!fs.existsSync(config.LOG_PATH)) {\n fs.mkdirSync(config.LOG_PATH, { recursive: true });\n }\n fs.writeFileSync(`${config.LOG_PATH}/.preflight_ok`, new Date().toISOString());\n \n // Grab categories out of the previous node (Fetch Categories) if available\n // API returns raw array — each item becomes a separate n8n item\n let categories = ['Correspondence','RFA','Drawing','Transmittal','Report','Other'];\n try {\n const upstreamItems = $('Fetch Categories').all().map(i => i.json);\n if (upstreamItems && upstreamItems.length > 0) {\n categories = upstreamItems.map(c => c.typeName || c.typeCode || c); \n }\n } catch(e) {}\n \n // Grab existing tags from Fetch Tags node\n // API returns raw array — each item becomes a separate n8n item\n let existingTags = [];\n try {\n const tagItems = $('Fetch Tags').all().map(i => i.json);\n existingTags = Array.isArray(tagItems) ? tagItems.map(t => t.tag_name || t.name || '').filter(Boolean) : [];\n } catch(e) {}\n \n return [{ json: { \n preflight_ok: true, \n pdf_count_in_source: files.length,\n excel_target: config.EXCEL_FILE,\n system_categories: categories,\n existing_tags: existingTags,\n timestamp: new Date().toISOString()\n }}];\n} catch (err) {\n throw new Error(`Pre-flight check failed: ${err.message}`);\n}" }, - "id": "910b13e2-994a-4fb6-bca1-637e1628c586", + "id": "2204f397-a2bf-4eaa-88b6-d0242d06fad7", "name": "File Mount Check", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [31216, 13904], + "position": [-816, -27392], "notes": "ตรวจสอบ File System มีไฟล์ Excel และ Folder ตามตั้งค่า" }, { @@ -135,11 +135,11 @@ "query": "SELECT last_processed_index, status FROM migration_progress WHERE batch_id = '{{$('Set Configuration').first().json.config.BATCH_ID}}' LIMIT 1", "options": {} }, - "id": "a83f8598-72fd-4cc8-9d98-1ea3cb3b42df", + "id": "a1630364-d44b-4ef4-bc51-f0f75adf20f9", "name": "Read Checkpoint", "type": "n8n-nodes-base.mySql", "typeVersion": 2.4, - "position": [31632, 13744], + "position": [-768, -27168], "alwaysOutputData": true, "credentials": { "mySql": { @@ -155,45 +155,45 @@ "fileSelector": "={{ $json.excel_target }}", "options": {} }, - "id": "063bcef1-791a-4923-a659-8b9a0ba3e336", + "id": "71e94a39-e01d-46c1-8f35-c36da02211dc", "name": "Read Excel Binary", "type": "n8n-nodes-base.readWriteFile", "typeVersion": 1, - "position": [31392, 13904], + "position": [-1168, -27168], "notes": "ดึงไฟล์ Excel ขึ้นมาไว้ในหน่วยความจำ" }, { "parameters": { "options": {} }, - "id": "e07efdde-b9b1-402a-ba01-44175982749b", + "id": "023aaed1-8c33-480c-ab43-8241702ac17d", "name": "Read Excel", "type": "n8n-nodes-base.spreadsheetFile", "typeVersion": 2, - "position": [31392, 14112], + "position": [-976, -27168], "notes": "แปลงข้อมูล Excel เป็น JSON Data" }, { "parameters": { - "jsCode": "const cpJson = $input.first()?.json || {};\nconst startIndex = cpJson.last_processed_index || 0;\nconst config = $('Set Configuration').first().json.config;\n\nconst allItems = $('Read Excel').all().map(i => i.json);\nconst remaining = allItems.slice(startIndex);\nconst currentBatch = remaining.slice(0, config.BATCH_SIZE);\n\n// Encoding Normalization\nconst normalize = (str) => {\n if (!str) return '';\n return String(str).normalize('NFC').trim();\n};\n\nreturn currentBatch.map((item, i) => {\n const getVal = (possibleKeys) => {\n const exactMatch = possibleKeys.find(k => item[k] !== undefined);\n if (exactMatch) return item[exactMatch];\n const lowerTrimmedKeys = Object.keys(item).map(k => ({ original: k, parsed: k.toLowerCase().trim() }));\n for (const pk of possibleKeys) {\n const found = lowerTrimmedKeys.find(k => k.parsed === pk.toLowerCase().trim());\n if (found) return item[found.original];\n }\n return '';\n };\n\n const docNum = getVal(['document_number', 'correspondence_number', 'Document Number', 'Corr. No.']);\n const excelFileName = getVal(['File name', 'file_name', 'File Name', 'filename']);\n \n if (!excelFileName) {\n throw new Error(`Missing 'File name' column for row ${i + startIndex + 1}, document: ${docNum}`);\n }\n \n return {\n json: {\n document_number: normalize(docNum),\n title: normalize(getVal(['title', 'Title', 'Subject', 'subject'])),\n legacy_number: normalize(getVal(['legacy_number', 'Legacy Number', 'Response Doc.'])),\n excel_revision: getVal(['revision', 'Revision', 'rev']) || 1,\n original_index: startIndex + i,\n batch_id: config.BATCH_ID,\n file_name: normalize(excelFileName),\n issued_date: normalize(getVal(['issued_date', 'Issued_date', 'Issued Date', 'date', 'Date', 'document_date'])),\n received_date: normalize(getVal(['received_date', 'Received_date', 'Received Date'])),\n correspondence_type: getVal(['correspondence_type', 'type', 'Type', 'Category']),\n sender: normalize(getVal(['sender', 'Sender', 'From', 'from'])),\n receiver: normalize(getVal(['receiver', 'Receiver', 'To', 'to'])),\n project_code: normalize(getVal(['project', 'Project', 'project_code']))\n }\n };\n});" + "jsCode": "const cpJson = $input.first()?.json || {};\nconst startIndex = cpJson.last_processed_index || 0;\nconst config = $('Set Configuration').first().json.config;\n\nconst allItems = $('Read Excel').all().map(i => i.json);\nconst remaining = allItems.slice(startIndex);\nconst currentBatch = remaining.slice(0, config.BATCH_SIZE);\n\n// Encoding Normalization\nconst normalize = (str) => {\n if (!str) return '';\n return String(str).normalize('NFC').trim();\n};\n\nreturn currentBatch.map((item, i) => {\n const getVal = (possibleKeys) => {\n const exactMatch = possibleKeys.find(k => item[k] !== undefined);\n if (exactMatch) return item[exactMatch];\n const lowerTrimmedKeys = Object.keys(item).map(k => ({ original: k, parsed: k.toLowerCase().trim() }));\n for (const pk of possibleKeys) {\n const found = lowerTrimmedKeys.find(k => k.parsed === pk.toLowerCase().trim());\n if (found) return item[found.original];\n }\n return '';\n };\n\n const docNum = getVal(['document_number', 'correspondence_number', 'Document Number', 'Corr. No.']);\n const excelFileName = getVal(['File name', 'file_name', 'File Name', 'filename']);\n \n if (!excelFileName) {\n throw new Error(`Missing 'File name' column for row ${i + startIndex + 1}, document: ${docNum}`);\n }\n \n return {\n json: {\n document_number: normalize(docNum),\n subject: normalize(getVal(['Subject', 'subject', 'Title', 'title'])),\n legacy_number: normalize(getVal(['legacy_number', 'Legacy Number', 'Response Doc.'])),\n excel_revision: getVal(['revision', 'Revision', 'rev']) || 1,\n original_index: startIndex + i,\n batch_id: config.BATCH_ID,\n file_name: normalize(excelFileName),\n issued_date: normalize(getVal(['issued_date', 'Issued_date', 'Issued Date', 'date', 'Date', 'document_date'])),\n received_date: normalize(getVal(['received_date', 'Received_date', 'Received Date'])),\n correspondence_type: getVal(['correspondence_type', 'type', 'Type', 'Category']),\n sender: normalize(getVal(['sender', 'Sender', 'From', 'from'])),\n receiver: normalize(getVal(['receiver', 'Receiver', 'To', 'to'])),\n project_code: normalize(getVal(['project', 'Project', 'project_code']))\n }\n };\n});" }, - "id": "80845e32-c283-4e9f-af73-6339d675fb38", + "id": "0b6f8817-fe02-4482-b707-64c58692d77b", "name": "Process Batch + Encoding", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [31808, 13488], + "position": [-560, -27584], "alwaysOutputData": true, "notes": "ตัด Batch + Normalize UTF-8" }, { "parameters": { - "jsCode": "const fs = require('fs');\nconst path = require('path');\nconst config = $('Set Configuration').first().json.config;\n\nconst items = $input.all();\nif (!items || items.length === 0) return [];\n\nconst validated = [];\nconst errors = [];\n\nfor (const item of items) {\n const fileName = item.json?.file_name;\n if (!fileName) {\n errors.push({\n ...item,\n json: { ...item.json, file_valid: false, error: 'file_name is missing', error_type: 'MISSING_FILENAME', file_exists: false }\n });\n continue;\n }\n \n let safeName = path.basename(String(fileName)).normalize('NFC');\n if (!safeName.toLowerCase().endsWith('.pdf')) {\n safeName += '.pdf';\n }\n const filePath = path.resolve(config.SOURCE_PDF_DIR, safeName);\n \n if (!filePath.startsWith(path.resolve(config.SOURCE_PDF_DIR))) {\n errors.push({\n ...item,\n json: { ...item.json, file_valid: false, error: 'Path traversal detected', error_type: 'SECURITY', file_exists: false }\n });\n continue;\n }\n \n try {\n if (fs.existsSync(filePath)) {\n const stats = fs.statSync(filePath);\n validated.push({\n ...item,\n json: { ...item.json, file_valid: true, file_exists: true, file_size: stats.size, file_path: filePath }\n });\n } else {\n errors.push({\n ...item,\n json: { ...item.json, file_valid: false, error: `File not found: ${safeName}`, error_type: 'FILE_NOT_FOUND', file_exists: false }\n });\n }\n } catch (err) {\n errors.push({\n ...item,\n json: { ...item.json, file_valid: false, error: err.message, error_type: 'FILE_ERROR', file_exists: false }\n });\n }\n}\n\nreturn [...validated, ...errors];" + "jsCode": "const fs = require('fs');\nconst path = require('path');\nconst config = $('Set Configuration').first().json.config;\n\nconst items = $input.all();\nif (!items || items.length === 0) return [];\n\nconst validated = [];\nconst errors = [];\n\nfor (const item of items) {\n const fileName = item.json?.file_name;\n if (!fileName) {\n errors.push({\n ...item,\n json: { ...item.json, file_valid: false, error: 'file_name is missing', error_type: 'FILE_NOT_FOUND', file_exists: false }\n });\n continue;\n }\n \n let safeName = path.basename(String(fileName)).normalize('NFC');\n if (!safeName.toLowerCase().endsWith('.pdf')) {\n safeName += '.pdf';\n }\n const filePath = path.resolve(config.SOURCE_PDF_DIR, safeName);\n \n if (!filePath.startsWith(path.resolve(config.SOURCE_PDF_DIR))) {\n errors.push({\n ...item,\n json: { ...item.json, file_valid: false, error: 'Path traversal detected', error_type: 'SECURITY', file_exists: false }\n });\n continue;\n }\n \n try {\n if (fs.existsSync(filePath)) {\n const stats = fs.statSync(filePath);\n validated.push({\n ...item,\n json: { ...item.json, file_valid: true, file_exists: true, file_size: stats.size, file_path: filePath }\n });\n } else {\n errors.push({\n ...item,\n json: { ...item.json, file_valid: false, error: `File not found: ${safeName}`, error_type: 'FILE_NOT_FOUND', file_exists: false }\n });\n }\n } catch (err) {\n errors.push({\n ...item,\n json: { ...item.json, file_valid: false, error: err.message, error_type: 'UNKNOWN', file_exists: false }\n });\n }\n}\n\n// Log errors inline to CSV (single-output node — errors don't flow downstream)\nif (errors.length > 0) {\n const csvPath = `${config.LOG_PATH}/error_log.csv`;\n const header = 'timestamp,document_number,error_type,error_message\\n';\n const esc = (s) => `\"${String(s || '').replace(/\"/g, '\"\"')}\"`;\n if (!fs.existsSync(config.LOG_PATH)) fs.mkdirSync(config.LOG_PATH, { recursive: true });\n if (!fs.existsSync(csvPath)) fs.writeFileSync(csvPath, header, 'utf8');\n for (const e of errors) {\n const line = [new Date().toISOString(), esc(e.json.document_number), esc(e.json.error_type), esc(e.json.error)].join(',') + '\\n';\n fs.appendFileSync(csvPath, line, 'utf8');\n }\n}\n\nreturn validated;" }, - "id": "2183d687-4708-4d77-a0a9-13ccf29baf69", + "id": "254e8f42-e32a-486f-a5d8-c77bfcb5ee44", "name": "File Validator", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [31984, 13488], + "position": [-384, -27584], "notes": "ตรวจสอบไฟล์ PDF ตัวชี้ใน Directory จาก Config" }, { @@ -202,11 +202,11 @@ "query": "SELECT is_fallback_active, recent_error_count FROM migration_fallback_state WHERE batch_id = '{{$('Set Configuration').first().json.config.BATCH_ID}}' LIMIT 1", "options": {} }, - "id": "8b0c61d0-96e4-468a-991f-a40e534e167a", + "id": "c86229c0-493d-4138-a450-db65e0bc1d5d", "name": "Check Fallback State", "type": "n8n-nodes-base.mySql", "typeVersion": 2.4, - "position": [31792, 13888], + "position": [-560, -27184], "alwaysOutputData": true, "credentials": { "mySql": { @@ -219,13 +219,13 @@ }, { "parameters": { - "jsCode": "const config = $('Set Configuration').first().json.config;\nconst fallbackState = $('Check Fallback State').first()?.json || { is_fallback_active: false, recent_error_count: 0 };\nconst isFallback = fallbackState.is_fallback_active || false;\nconst model = isFallback ? config.OLLAMA_MODEL_FALLBACK : config.OLLAMA_MODEL_PRIMARY;\n\nconst dbContext = $('Fetch DB Context').all().map(i => i.json);\nconst dbProjects = dbContext.filter(d => d.type === 'projects').map(d => ({id: d.id, code: d.text1, name: d.text2}));\nconst dbDisciplines = dbContext.filter(d => d.type === 'disciplines').map(d => ({id: d.id, th: d.text1, en: d.text2}));\nconst dbOrgs = dbContext.filter(d => d.type === 'organizations').map(d => ({id: d.id, name: d.text1, code: d.text2}));\nconst dbTags = dbContext.filter(d => d.type === 'tags').map(d => ({id: d.id, name: d.text1}));\nconst dbCorrTypes = dbContext.filter(d => d.type === 'correspondence_types').map(d => ({id: d.id, code: d.text1, name: d.text2}));\n\nlet systemCategories = ['Correspondence','RFA','Drawing','Transmittal','Report','Other'];\ntry { systemCategories = $('File Mount Check').first().json.system_categories || systemCategories; } catch (e) {}\n\nconst pdfItems = $('Extract PDF Text').all();\nconst metaItems = $('File Validator').all();\n\nreturn pdfItems.map((pdfItem, i) => {\n const item = metaItems[i] || pdfItem;\n\n const docNum = String(item.json.document_number || '');\n const title = String(item.json.title || '');\n const legacyNum = String(item.json.legacy_number || '');\n const issuedDate = String(item.json.issued_date || '');\n const receivedDate = String(item.json.received_date || '');\n const corrType = String(item.json.correspondence_type || '');\n const senderCode = String(item.json.sender || '');\n const receiverCode = String(item.json.receiver || '');\n\n const prompt = `Validate and summarize this document. Respond in JSON.\nDocument Number: ${docNum}\nOriginal Title: ${title}\nExtracted Text: ${(pdfItem.json.response || pdfItem.json.data || '').substring(0, 4000)}\n\nExisting Projects: ${JSON.stringify(dbProjects)}\nExisting Disciplines: ${JSON.stringify(dbDisciplines)}\nExisting Orgs: ${JSON.stringify(dbOrgs)}\nExisting Categories: ${JSON.stringify(systemCategories)}\nExisting Tags: ${JSON.stringify(dbTags)}\n\nAnalyze the content to provide:\n1. Validation of Subject/Dates with PDF text.\n2. A 4-5 sentence summary.\n3. Suggest tags. Select from Existing Tags if applicable.\n\nRespond ONLY with this exact JSON structure:\n{\n \"is_valid\": true,\n \"confidence\": 0.9,\n \"category\": \"Correspondence\",\n \"subject\": \"...\",\n \"summary\": \"...\",\n \"discipline_id\": 64,\n \"tags\": [{\"tag_name\": \"...\", \"description\": \"...\"}],\n \"key_points\": [\"...\"],\n \"document_date\": \"YYYY-MM-DD\",\n \"issued_date\": \"YYYY-MM-DD\",\n \"received_date\": \"YYYY-MM-DD\"\n}`;\n\n return {\n json: {\n ...item.json,\n temp_attachment_id: pdfItem.json.temp_attachment_id,\n ollama_payload: {\n model: model,\n prompt: prompt,\n stream: false,\n format: \"json\",\n options: { temperature: 0.2, num_ctx: 8192 }\n },\n system_categories: systemCategories,\n pre_mapped: {\n project_id: dbProjects.find(p => docNum.includes(p.code))?.id || config.PROJECT_ID,\n sender_id: dbOrgs.find(o => senderCode.includes(o.code) || senderCode.includes(o.name))?.id,\n receiver_id: dbOrgs.find(o => receiverCode.includes(o.code) || receiverCode.includes(o.name))?.id\n }\n }\n };\n});" + "jsCode": "const config = $('Set Configuration').first().json.config;\nconst fallbackState = $('Check Fallback State').first()?.json || { is_fallback_active: false, recent_error_count: 0 };\nconst isFallback = fallbackState.is_fallback_active || false;\nconst model = isFallback ? config.OLLAMA_MODEL_FALLBACK : config.OLLAMA_MODEL_PRIMARY;\n\nconst dbContext = $('Fetch DB Context').all().map(i => i.json);\nconst dbProjects = dbContext.filter(d => d.type === 'projects').map(d => ({id: d.id, code: d.text1, name: d.text2}));\nconst dbDisciplines = dbContext.filter(d => d.type === 'disciplines').map(d => ({id: d.id, th: d.text1, en: d.text2}));\nconst dbOrgs = dbContext.filter(d => d.type === 'organizations').map(d => ({id: d.id, name: d.text1, code: d.text2}));\nconst dbTags = dbContext.filter(d => d.type === 'tags').map(d => ({id: d.id, name: d.text1, description: d.text2 || ''}));\nconst dbCorrTypes = dbContext.filter(d => d.type === 'correspondence_types').map(d => ({id: d.id, code: d.text1, name: d.text2}));\n\nlet systemCategories = ['Correspondence','RFA','Drawing','Transmittal','Report','Other'];\ntry { systemCategories = $('File Mount Check').first().json.system_categories || systemCategories; } catch (e) {}\n\nconst pdfItems = $('Extract PDF Text').all();\nconst metaItems = $('File Validator').all();\n\nreturn pdfItems.map((pdfItem, i) => {\n const item = metaItems[i] || pdfItem;\n\n const docNum = String(item.json.document_number || '');\n const subject = String(item.json.subject || '');\n const projectCode = String(item.json.project_code || '');\n const legacyNum = String(item.json.legacy_number || '');\n const issuedDate = String(item.json.issued_date || '');\n const receivedDate = String(item.json.received_date || '');\n const corrType = String(item.json.correspondence_type || '');\n const senderCode = String(item.json.sender || '');\n const receiverCode = String(item.json.receiver || '');\n\n const prompt = `Validate and summarize this document. Respond in JSON.\nDocument Number: ${docNum}\nOriginal Subject: ${subject}\nExtracted Text: ${(pdfItem.json.response || pdfItem.json.data || '').substring(0, 4000)}\n\nExisting Projects: ${JSON.stringify(dbProjects)}\nExisting Disciplines: ${JSON.stringify(dbDisciplines)}\nExisting Orgs: ${JSON.stringify(dbOrgs)}\nExisting Categories: ${JSON.stringify(systemCategories)}\nExisting Tags: ${JSON.stringify(dbTags)}\n\nAnalyze the content to provide:\n1. Validate the Subject and Dates against PDF text.\n2. Write a detailed summary (4-5 sentences) for the body field.\n3. Suggest 1-5 tags. Prefer Existing Tags when applicable. Each tag MUST have tag_name and description.\n\nRespond ONLY with this exact JSON structure:\n{\n \"is_valid\": true,\n \"confidence\": 0.9,\n \"category\": \"Correspondence\",\n \"subject\": \"Verified or corrected subject line\",\n \"body\": \"Detailed 4-5 sentence summary of the document content for archival.\",\n \"discipline_id\": 64,\n \"tags\": [{\"tag_name\": \"TagName\", \"description\": \"Why this tag applies\"}],\n \"key_points\": [\"...\"],\n \"document_date\": \"YYYY-MM-DD\",\n \"issued_date\": \"YYYY-MM-DD\",\n \"received_date\": \"YYYY-MM-DD\"\n}`;\n\n return {\n json: {\n ...item.json,\n ollama_payload: {\n model: model,\n prompt: prompt,\n stream: false,\n format: \"json\",\n options: { temperature: 0.2, num_ctx: 8192 }\n },\n system_categories: systemCategories,\n pre_mapped: {\n project_id: (projectCode && dbProjects.find(p => p.code === projectCode)?.id) || dbProjects.find(p => docNum.includes(p.code))?.id || config.PROJECT_ID,\n sender_id: dbOrgs.find(o => senderCode.includes(o.code) || senderCode.includes(o.name))?.id,\n receiver_id: dbOrgs.find(o => receiverCode.includes(o.code) || receiverCode.includes(o.name))?.id\n }\n }\n };\n});" }, - "id": "2ba75d42-1de3-4846-a1a3-39d580e7d764", + "id": "81346da7-a5b0-4a5f-9dc5-34bfc61ebac2", "name": "Build AI Prompt", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [32144, 13872], + "position": [-192, -27200], "notes": "สร้าง Prompt โดยใช้ Categories จาก System" }, { @@ -239,22 +239,22 @@ "timeout": 120000 } }, - "id": "3e8b33cb-8f8f-4d2e-b4cb-9d68cc54d96e", + "id": "8634f965-41f4-485e-9c01-22640b42d8cd", "name": "Ollama AI Analysis", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4.1, - "position": [31792, 14096], + "position": [-560, -26992], "notes": "เรียก Ollama วิเคราะห์เอกสาร" }, { "parameters": { - "jsCode": "const ollamaItems = $input.all();\nconst originalItems = $('Build AI Prompt').all();\nconst results = [];\n\nconst CATEGORY_TO_TYPE_CODE = {\n 'Correspondence': 'LETTER',\n 'RFA': 'RFA',\n 'Transmittal': 'TRANSMITTAL',\n 'Drawing': 'OTHER',\n 'Report': 'OTHER',\n 'Other': 'OTHER',\n};\n\nfor (let i = 0; i < ollamaItems.length; i++) {\n const ollamaItem = ollamaItems[i];\n const originalItem = originalItems[i];\n if (!originalItem) continue;\n const baseJson = originalItem.json;\n\n try {\n let raw = ollamaItem.json.response || '';\n raw = raw.replace(/`{3}json/gi, '').replace(/`{3}/g, '').trim();\n if (!raw) throw new Error('Empty response from AI');\n\n const result = JSON.parse(raw);\n const systemCategories = baseJson.system_categories || [];\n let finalCategory = result.category;\n if (!systemCategories.includes(finalCategory)) {\n finalCategory = String(baseJson.document_number || '').includes('-RFA-') ? 'RFA' : 'Correspondence';\n }\n\n const typeCode = CATEGORY_TO_TYPE_CODE[finalCategory] || 'LETTER';\n const preMapped = baseJson.pre_mapped || {};\n\n results.push({\n json: {\n ...baseJson,\n ai_result: {\n suggested_category: finalCategory,\n type_code: typeCode,\n confidence: result.confidence || 0.8,\n project_id: preMapped.project_id || null,\n discipline_id: result.discipline_id || 64,\n sender_id: preMapped.sender_id || null,\n receiver_id: preMapped.receiver_id || null,\n subject: result.subject || baseJson.title,\n issued_date: result.issued_date || baseJson.issued_date,\n received_date: result.received_date || baseJson.received_date,\n summary: result.summary || '',\n key_points: result.key_points || [],\n tags: result.tags || [],\n is_valid: result.is_valid !== false\n }\n }\n });\n } catch (err) {\n results.push({\n json: {\n ...baseJson,\n parse_error: err.message,\n raw_ai_response: ollamaItem.json.response\n }\n });\n }\n}\n\nreturn results;" + "jsCode": "const ollamaItems = $input.all();\nconst originalItems = $('Build AI Prompt').all();\nconst results = [];\n\nconst CATEGORY_TO_TYPE_CODE = {\n 'Correspondence': 'LETTER',\n 'RFA': 'RFA',\n 'Transmittal': 'TRANSMITTAL',\n 'Drawing': 'OTHER',\n 'Report': 'OTHER',\n 'Other': 'OTHER',\n};\n\nfor (let i = 0; i < ollamaItems.length; i++) {\n const ollamaItem = ollamaItems[i];\n const originalItem = originalItems[i];\n if (!originalItem) continue;\n const baseJson = originalItem.json;\n\n try {\n let raw = ollamaItem.json.response || '';\n raw = raw.replace(/`{3}json/gi, '').replace(/`{3}/g, '').trim();\n if (!raw) throw new Error('Empty response from AI');\n\n const result = JSON.parse(raw);\n const systemCategories = baseJson.system_categories || [];\n let finalCategory = result.category;\n if (!systemCategories.includes(finalCategory)) {\n finalCategory = String(baseJson.document_number || '').includes('-RFA-') ? 'RFA' : 'Correspondence';\n }\n\n const typeCode = CATEGORY_TO_TYPE_CODE[finalCategory] || 'LETTER';\n const preMapped = baseJson.pre_mapped || {};\n\n results.push({\n json: {\n ...baseJson,\n ai_result: {\n suggested_category: finalCategory,\n type_code: typeCode,\n confidence: result.confidence || 0.8,\n project_id: preMapped.project_id || null,\n discipline_id: result.discipline_id || 64,\n sender_id: preMapped.sender_id || null,\n receiver_id: preMapped.receiver_id || null,\n subject: result.subject || baseJson.subject || '',\n body: result.body || result.summary || '',\n issued_date: result.issued_date || baseJson.issued_date,\n received_date: result.received_date || baseJson.received_date,\n summary: result.summary || result.body || '',\n key_points: result.key_points || [],\n tags: (result.tags || []).map(t => (typeof t === 'string' ? { tag_name: t, description: '' } : { tag_name: t.tag_name || t.name || '', description: t.description || '' })).filter(t => t.tag_name),\n is_valid: result.is_valid !== false\n }\n }\n });\n } catch (err) {\n results.push({\n json: {\n ...baseJson,\n parse_error: err.message,\n raw_ai_response: ollamaItem.json.response\n }\n });\n }\n}\n\nreturn results;" }, - "id": "6716162f-1129-4552-a05f-a08ac115fe10", + "id": "2154cfc3-e4d1-499f-b918-923e78f442e9", "name": "Parse & Validate AI Response", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [32000, 14096], + "position": [-352, -26992], "notes": "Parse JSON + Validate Schema + Enum Check" }, { @@ -263,11 +263,11 @@ "query": "INSERT INTO migration_fallback_state (batch_id, recent_error_count, is_fallback_active) VALUES ('{{$('Set Configuration').first().json.config.BATCH_ID}}', 1, FALSE) ON DUPLICATE KEY UPDATE recent_error_count = recent_error_count + 1, is_fallback_active = CASE WHEN recent_error_count + 1 >= {{$('Set Configuration').first().json.config.FALLBACK_THRESHOLD}} THEN TRUE ELSE is_fallback_active END, updated_at = NOW()", "options": {} }, - "id": "b2ac9722-f917-42fd-81e9-c77e84b84104", + "id": "a1adab22-8382-4336-ba74-15a4721d51f4", "name": "Update Fallback State", "type": "n8n-nodes-base.mySql", "typeVersion": 2.4, - "position": [32464, 13472], + "position": [0, -27488], "credentials": { "mySql": { "id": "CHHfbKhMacNo03V4", @@ -280,64 +280,73 @@ "parameters": { "jsCode": "const config = $('Set Configuration').first().json.config;\nconst items = $('Parse & Validate AI Response').all();\n\nconst results = [];\n\nfor (const item of items) {\n const data = item.json;\n let resultItem = { json: { ...data } };\n \n if (data.parse_error || !data.ai_result) {\n resultItem.json.route_index = 3;\n results.push(resultItem);\n continue;\n }\n \n const ai = data.ai_result;\n \n if (ai.confidence >= config.CONFIDENCE_HIGH) {\n resultItem.json.route_index = 0;\n resultItem.json.staging_status = 'PENDING';\n resultItem.json.staging_remarks = 'Ready for auto-ingest (High Confidence)';\n } else if (ai.confidence >= config.CONFIDENCE_LOW) {\n resultItem.json.route_index = 1;\n resultItem.json.staging_status = 'PENDING';\n resultItem.json.staging_remarks = 'Flagged for human review (Medium Confidence)';\n } else {\n resultItem.json.route_index = 2;\n resultItem.json.staging_status = 'REJECTED';\n resultItem.json.staging_remarks = ai.is_valid === false ? 'AI marked invalid' : `Rejected for human review (Low Confidence: ${ai.confidence.toFixed(2)})`;\n }\n results.push(resultItem);\n}\n\nreturn results;" }, - "id": "ccaaee30-ead6-46c0-954a-eb8b98620cb3", + "id": "34288a94-82da-4642-88b1-b0929f921eeb", "name": "Confidence Router", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [32160, 14096], + "position": [-192, -26992], "notes": "แยกตาม Confidence: Auto(≥0.85) / Review(≥0.60) / Reject(<0.60)" }, { "parameters": { "jsCode": "const fs = require('fs');\nconst item = $input.first();\nconst config = $('Set Configuration').first().json.config;\n\nconst csvPath = `${config.LOG_PATH}/reject_log.csv`;\nconst header = 'timestamp,document_number,title,reject_reason,ai_confidence,key_points\\n';\nconst esc = (s) => `\"${String(s || '').replace(/\"/g, '\"\"')}\"`;\n\nif (!fs.existsSync(config.LOG_PATH)) {\n fs.mkdirSync(config.LOG_PATH, { recursive: true });\n}\n\nif (!fs.existsSync(csvPath)) {\n fs.writeFileSync(csvPath, header, 'utf8');\n}\n\nconst line = [\n new Date().toISOString(),\n esc(item.json.document_number),\n esc(item.json.title),\n esc(item.json.staging_remarks),\n item.json.ai_result?.confidence ?? 'N/A',\n esc(JSON.stringify(item.json.ai_result?.key_points || []))\n].join(',') + '\\n';\n\nfs.appendFileSync(csvPath, line, 'utf8');\n\nreturn [$input.first()];" }, - "id": "0bb3530f-02d5-44d0-ad94-c94d97d91b6a", + "id": "e12e7219-0f80-4414-8d4a-aa9363ec1ee9", "name": "Log Reject to CSV", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [32624, 14032], + "position": [304, -27216], "notes": "บันทึกรายการที่ถูกปฏิเสธลง CSV" }, { "parameters": { "jsCode": "const fs = require('fs');\nconst items = $input.all();\nconst config = $('Set Configuration').first().json.config;\n\nconst csvPath = `${config.LOG_PATH}/error_log.csv`;\nconst header = 'timestamp,document_number,error_type,error_message,raw_ai_response\\n';\nconst esc = (s) => `\"${String(s || '').replace(/\"/g, '\"\"')}\"`;\n\nif (!fs.existsSync(config.LOG_PATH)) {\n fs.mkdirSync(config.LOG_PATH, { recursive: true });\n}\n\nif (!fs.existsSync(csvPath)) {\n fs.writeFileSync(csvPath, header, 'utf8');\n}\n\nfor (const item of items) {\n const line = [\n new Date().toISOString(),\n esc(item.json.document_number),\n esc(item.json.error_type || 'UNKNOWN'),\n esc(item.json.error || item.json.parse_error),\n esc(item.json.raw_ai_response || '')\n ].join(',') + '\\n';\n \n fs.appendFileSync(csvPath, line, 'utf8');\n}\n\nreturn items;" }, - "id": "8250dd88-ca81-45aa-93d8-480c9bcd6b14", + "id": "a2526431-176a-4246-a9ad-3b2dbfec574a", "name": "Log Error to CSV", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [32448, 14128], + "position": [144, -27120], "notes": "บันทึก Error ลง CSV (จาก File Validator)" }, { "parameters": { - "operation": "executeQuery", - "query": "INSERT INTO migration_errors (batch_id, document_number, error_type, error_message, raw_ai_response, created_at) VALUES ('{{$('Set Configuration').first().json.config.BATCH_ID}}', '{{$json.document_number}}', '{{$json.error_type || \"UNKNOWN\"}}', '{{$json.error || $json.parse_error}}', '{{$json.raw_ai_response || \"\"}}', NOW())", - "options": {} - }, - "id": "0f058ad0-3c09-4c9f-bdcf-503cd58ee395", - "name": "Log Error to DB", - "type": "n8n-nodes-base.mySql", - "typeVersion": 2.4, - "position": [32752, 14128], - "credentials": { - "mySql": { - "id": "CHHfbKhMacNo03V4", - "name": "MySQL account" + "method": "POST", + "url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/migration/errors", + "sendHeaders": true, + "headerParameters": { + "parameters": [ + { + "name": "Authorization", + "value": "={{$('Set Configuration').first().json.config.MIGRATION_TOKEN}}" + } + ] + }, + "sendBody": true, + "specifyBody": "json", + "jsonBody": "={{ JSON.stringify({ batch_id: $('Set Configuration').first().json.config.BATCH_ID, document_number: $json.document_number || '', error_type: $json.error_type || 'UNKNOWN', error_message: $json.error || $json.parse_error || '', raw_ai_response: $json.raw_ai_response || '' }) }}", + "options": { + "timeout": 10000 } }, - "notes": "บันทึก Error ลง MariaDB" + "id": "7d620e96-a067-430e-af53-06c4492c11e4", + "name": "Log Error to DB", + "type": "n8n-nodes-base.httpRequest", + "typeVersion": 4.1, + "position": [464, -27104], + "onError": "continueErrorOutput", + "notes": "บันทึก Error ผ่าน Backend API (ป้องกัน SQL Injection)" }, { "parameters": { "amount": "={{$('Set Configuration').first().json.config.DELAY_MS / 1000}}", "unit": "seconds" }, - "id": "07c1c5d5-5ffc-4e3d-ab3e-4b62ad079388", + "id": "5909f64b-67a5-40e7-b7b0-3e031376e5ad", "name": "Delay", "type": "n8n-nodes-base.wait", "typeVersion": 1, - "position": [33104, 14080], + "position": [704, -27152], "webhookId": "38e97a99-4dcc-4b63-977a-a02945a1c369", "notes": "หน่วงเวลาระหว่าง Batches" }, @@ -445,22 +454,22 @@ }, "options": {} }, - "id": "65f0bb6c-496a-4409-8b88-3132866cf9a4", + "id": "0558f317-fc7a-460a-bcfd-60ea541fc2b9", "name": "Route by Confidence", "type": "n8n-nodes-base.switch", "typeVersion": 3.2, - "position": [32336, 13744] + "position": [-16, -27312] }, { "parameters": { "fileSelector": "={{ $json.file_path }}", "options": {} }, - "id": "4fd3133e-39e1-4860-95c7-3e87ee43ed51", + "id": "319a9a0b-2837-4f1c-8e5a-58a0e95dfed5", "name": "Read PDF File", "type": "n8n-nodes-base.readWriteFile", "typeVersion": 1, - "position": [31824, 13680], + "position": [-224, -27584], "onError": "continueErrorOutput" }, { @@ -481,8 +490,8 @@ "bodyParameters": { "parameters": [ { - "name": "file", "parameterType": "formBinaryData", + "name": "file", "inputDataFieldName": "data" } ] @@ -491,23 +500,23 @@ "timeout": 60000 } }, - "id": "452d3a33-4f51-404c-8f4b-7a3d3a3a3a3a", + "id": "6db11a13-cd5f-4e85-9263-085590e0b07f", "name": "Upload to Backend", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4.1, - "position": [31968, 13680], + "position": [-544, -27376], "notes": "Upload PDF to Backend Temp Storage" }, { "parameters": { - "jsCode": "const uploadRes = $input.first()?.json || {};\nconst binaryData = $('Read PDF File').first().binary.data;\n\nreturn {\n json: {\n ...$input.first().json,\n temp_attachment_id: uploadRes.id\n },\n binary: {\n data: binaryData\n }\n};" + "jsCode": "const item = $input.first();\nconst binaryData = $('Read PDF File').first().binary.data;\n\nreturn {\n json: { ...item.json },\n binary: { data: binaryData }\n};" }, - "id": "b3e3e3e3-e3e3-4e3e-ae3e-3e3e3e3e3e3e", - "name": "Process Upload", + "id": "eeea6259-6bb0-42cc-8812-d6dd22e8fa1c", + "name": "Restore Binary", "type": "n8n-nodes-base.code", "typeVersion": 2, - "position": [32112, 13680], - "notes": "Capture ID and Restore Binary for Tika" + "position": [-400, -27376], + "notes": "Re-attach PDF binary จาก Read PDF File เพื่อส่ง Upload (หลัง AI ตรวจแล้ว)" }, { "parameters": { @@ -546,11 +555,11 @@ "timeout": 600000 } }, - "id": "2d3868e0-ed56-4921-8d68-bf7b69a64546", + "id": "0b34e1a9-9d83-4c8a-9d59-2d9be4b23123", "name": "Extract PDF Text", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4.2, - "position": [32256, 13664], + "position": [-240, -27376], "onError": "continueErrorOutput" }, { @@ -559,11 +568,11 @@ "query": "SELECT 'projects' as type, id, project_code as text1, project_name as text2 FROM projects\nUNION ALL\nSELECT 'disciplines' as type, id, code_name_th as text1, code_name_en as text2 FROM disciplines\nUNION ALL\nSELECT 'organizations' as type, id, organization_name as text1, organization_code as text2 FROM organizations\nUNION ALL\nSELECT 'tags' as type, id, tag_name as text1, description as text2 FROM tags\nUNION ALL\nSELECT 'correspondence_types' as type, id, type_code as text1, type_name as text2 FROM correspondence_types", "options": {} }, - "id": "2e31dc54-3d57-4c88-9d35-1aba0132cdf9", + "id": "5f785556-e8f5-40ae-8fd6-786c46ed7090", "name": "Fetch DB Context", "type": "n8n-nodes-base.mySql", "typeVersion": 2.4, - "position": [32000, 13872], + "position": [-336, -27200], "alwaysOutputData": true, "credentials": { "mySql": { @@ -575,13 +584,13 @@ }, { "parameters": { - "jsCode": "const items = $input.all();\nconst config = $('Set Configuration').first().json.config;\n\nreturn items.map(itemWrapper => {\n const item = itemWrapper.json;\n const ai = item.ai_result || {};\n\n return {\n json: {\n ...item,\n enqueue_payload: {\n document_number: String(item.document_number || ''),\n title: String(ai.subject || item.title || ''),\n original_title: String(item.title || ''),\n category: ai.suggested_category || 'Correspondence',\n ai_summary: ai.summary || '',\n project_id: Number(ai.project_id || config.PROJECT_ID),\n sender_org_id: ai.sender_id || null,\n receiver_org_id: ai.receiver_id || null,\n issued_date: ai.issued_date || item.issued_date || '',\n received_date: ai.received_date || item.received_date || '',\n remarks: item.staging_remarks || '',\n extracted_tags: ai.tags || [],\n temp_attachment_id: item.temp_attachment_id || null,\n is_valid: ai.is_valid !== false,\n confidence: ai.confidence || 0.0,\n ai_issues: ai.key_points || []\n }\n }\n };\n});" + "jsCode": "const items = $input.all();\nconst config = $('Set Configuration').first().json.config;\n\nreturn items.map(itemWrapper => {\n const item = itemWrapper.json;\n const ai = item.ai_result || {};\n\n return {\n json: {\n ...item,\n enqueue_payload: {\n document_number: String(item.document_number || ''),\n subject: String(ai.subject || item.subject || ''),\n original_subject: String(item.subject || ''),\n category: ai.suggested_category || 'Correspondence',\n body: String(ai.body || ai.summary || ''),\n ai_summary: ai.summary || ai.body || '',\n project_id: Number(ai.project_id || config.PROJECT_ID),\n sender_org_id: ai.sender_id || null,\n receiver_org_id: ai.receiver_id || null,\n issued_date: ai.issued_date || item.issued_date || '',\n received_date: ai.received_date || item.received_date || '',\n remarks: item.staging_remarks || '',\n extracted_tags: ai.tags || [],\n details: { tags: ai.tags || [] },\n temp_attachment_id: $('Upload to Backend').first()?.json?.id || item.temp_attachment_id || null,\n is_valid: ai.is_valid !== false,\n confidence: ai.confidence || 0.0,\n ai_issues: ai.key_points || []\n }\n }\n };\n});" }, - "id": "57421305-8c7e-4fa4-a339-1144902cae22", + "id": "6bc2f3a0-9094-4bfd-a0a0-ba9a8effb53a", "name": "Build Enqueue Payload", "typeVersion": 2, "type": "n8n-nodes-base.code", - "position": [32544, 13664], + "position": [192, -27408], "notes": "สร้าง payload สำหรับ Enqueue Migration" }, { @@ -604,11 +613,11 @@ "timeout": 30000 } }, - "id": "f5f5f5f5-f5f5-4f5f-af5f-f5f5f5f5f5f5", + "id": "69152618-eed4-4b2b-a34f-e03cb630649a", "name": "Enqueue to Review Queue", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4.1, - "position": [32736, 13664], + "position": [368, -27408], "notes": "ส่งข้อมูลเข้า Staging Queue" }, { @@ -617,11 +626,11 @@ "query": "INSERT INTO migration_progress (batch_id, last_processed_index, status) VALUES ('{{$('Set Configuration').first().json.config.BATCH_ID}}', {{$json.original_index || 0}}, 'RUNNING') ON DUPLICATE KEY UPDATE last_processed_index = {{$json.original_index || 0}}, updated_at = NOW()", "options": {} }, - "id": "bb0e611b-db28-4266-ba40-3b5d534a16f7", + "id": "a7926600-01e6-49d3-9f29-ef7bab1c79c0", "name": "Save Checkpoint", "type": "n8n-nodes-base.mySql", "typeVersion": 2.4, - "position": [32928, 13856], + "position": [560, -27312], "credentials": { "mySql": { "id": "CHHfbKhMacNo03V4", @@ -631,6 +640,7 @@ "notes": "บันทึกความคืบหน้าลง Database" } ], + "pinData": {}, "connections": { "Form Trigger": { "main": [ @@ -764,7 +774,7 @@ "main": [ [ { - "node": "Upload to Backend", + "node": "Extract PDF Text", "type": "main", "index": 0 } @@ -775,18 +785,18 @@ "main": [ [ { - "node": "Process Upload", + "node": "Build Enqueue Payload", "type": "main", "index": 0 } ] ] }, - "Process Upload": { + "Restore Binary": { "main": [ [ { - "node": "Extract PDF Text", + "node": "Upload to Backend", "type": "main", "index": 0 } @@ -874,14 +884,14 @@ "main": [ [ { - "node": "Build Enqueue Payload", + "node": "Restore Binary", "type": "main", "index": 0 } ], [ { - "node": "Build Enqueue Payload", + "node": "Restore Binary", "type": "main", "index": 0 } @@ -980,7 +990,16 @@ ] } }, + "active": false, "settings": { - "executionOrder": "v1" - } + "executionOrder": "v1", + "binaryMode": "separate", + "availableInMCP": false + }, + "versionId": "1a305c6e-35fe-43cb-b5fa-bda279a36500", + "meta": { + "instanceId": "9e70e47c1eaf3bac72f497ddfbde0983f840f7d0f059537f7e37dd70de18ecb7" + }, + "id": "8Z6xwWVQ3TUflnSY", + "tags": [] } diff --git a/specs/03-Data-and-Storage/lcbp3-v1.8.0-migration.sql b/specs/03-Data-and-Storage/lcbp3-v1.8.0-migration.sql index a8d1fad..7688c64 100644 --- a/specs/03-Data-and-Storage/lcbp3-v1.8.0-migration.sql +++ b/specs/03-Data-and-Storage/lcbp3-v1.8.0-migration.sql @@ -23,8 +23,9 @@ CREATE TABLE IF NOT EXISTS migration_progress ( CREATE TABLE IF NOT EXISTS migration_review_queue ( id INT AUTO_INCREMENT PRIMARY KEY, document_number VARCHAR(100) NOT NULL, - title TEXT, - original_title TEXT, + subject TEXT COMMENT 'หัวข้อเรื่อง (ตรงกับ correspondence_revisions.subject)', + original_subject TEXT COMMENT 'หัวข้อเดิมจาก Excel (ก่อน AI แก้ไข)', + body TEXT NULL COMMENT 'เนื้อความสรุปจาก AI (เตรียมนำเข้า correspondence_revisions.body)', project_id INT NULL COMMENT 'Project ID จาก Lookups', sender_organization_id INT NULL COMMENT 'Sender ID จาก Lookups', receiver_organization_id INT NULL COMMENT 'Receiver ID จาก Lookups', @@ -54,6 +55,8 @@ CREATE TABLE IF NOT EXISTS migration_errors ( document_number VARCHAR(100), error_type ENUM( 'FILE_NOT_FOUND', + 'MISSING_FILENAME', + 'FILE_ERROR', 'AI_PARSE_ERROR', 'API_ERROR', 'DB_ERROR', diff --git a/specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md b/specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md index 048bb21..bee0cfe 100644 --- a/specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md +++ b/specs/05-Engineering-Guidelines/05-03-frontend-guidelines.md @@ -1,8 +1,8 @@ # Frontend Development Guidelines -**สำหรับ:** NAP-DMS LCBP3 Frontend (Next.js + TypeScript) -**เวอร์ชัน:** 1.5.0 -**อัปเดต:** 2025-12-01 +**สำหรับ:** NAP-DMS LCBP3 Frontend (Next.js 16 + TypeScript) +**เวอร์ชัน:** 1.8.1 +**อัปเดต:** 2026-03-16 --- @@ -45,7 +45,7 @@ frontend/ ├── public/ # Static assets ├── styles/ # Global styles ├── types/ # TypeScript types & DTOs -└── middleware.ts # Next.js Middleware +└── proxy.ts # Next.js Proxy (renamed from middleware.ts in Next.js 16) ``` --- diff --git a/specs/05-Engineering-Guidelines/README.md b/specs/05-Engineering-Guidelines/README.md index d45f54f..8a0d02e 100644 --- a/specs/05-Engineering-Guidelines/README.md +++ b/specs/05-Engineering-Guidelines/README.md @@ -10,9 +10,9 @@ | Attribute | Value | | ------------------ | -------------------------------- | -| **Version** | 1.8.0 | +| **Version** | 1.8.1 | | **Status** | Active | -| **Last Updated** | 2026-02-24 | +| **Last Updated** | 2026-03-16 | | **Owner** | Nattanin Peancharoen | | **Classification** | Internal Technical Documentation | @@ -50,7 +50,7 @@ ## 📖 คู่มือการพัฒนา (Implementation Guides) ### 1. [FullStack JS Guidelines](./05-01-fullstack-js-guidelines.md) -**แนวทางการพัฒนาภาพรวมทั้งระบบ (v1.8.0)** +**แนวทางการพัฒนาภาพรวมทั้งระบบ (v1.8.1 — includes NestJS 11 Patterns)** - โครงสร้างโปรเจกต์ (Monorepo-like focus) - Naming Conventions & Code Style - Secrets & Environment Management @@ -58,7 +58,7 @@ - Double-Lock Mechanism for Numbering ### 2. [Backend Guidelines](./05-02-backend-guidelines.md) -**แนวทางการพัฒนา NestJS Backend** +**แนวทางการพัฒนา NestJS 11 Backend** - Modular Architecture Detail - DTO Validation & Transformer - TypeORM Best Practices & Optimistic Locking @@ -66,7 +66,7 @@ - BullMQ for Background Jobs ### 3. [Frontend Guidelines](./05-03-frontend-guidelines.md) -**แนวทางการพัฒนา Next.js Frontend** +**แนวทางการพัฒนา Next.js 16 Frontend** - App Router Patterns - Shadcn/UI & Tailwind Styling - TanStack Query for Data Fetching @@ -97,8 +97,8 @@ | Layer | Primary Technology | Secondary/Supporting | | ------------ | ------------------ | -------------------- | -| **Backend** | NestJS (Node.js) | TypeORM, BullMQ | -| **Frontend** | Next.js 14+ | Shadcn/UI, Tailwind | +| **Backend** | NestJS 11 (Express v5) | TypeORM, BullMQ | +| **Frontend** | Next.js 16 (React 19) | Shadcn/UI, Tailwind | | **Database** | MariaDB 11.8 | Redis 7 (Cache/Lock) | | **Search** | Elasticsearch | - | | **Testing** | Jest, Vitest | Playwright | @@ -115,7 +115,7 @@
-**LCBP3-DMS Implementation Specification v1.8.0** +**LCBP3-DMS Implementation Specification v1.8.1** [FullStack](./05-01-fullstack-js-guidelines.md) • [Backend](./05-02-backend-guidelines.md) • [Frontend](./05-03-frontend-guidelines.md) • [Testing](./05-04-testing-strategy.md) diff --git a/specs/06-Decision-Records/README.md b/specs/06-Decision-Records/README.md index 0460b10..cfcbf78 100644 --- a/specs/06-Decision-Records/README.md +++ b/specs/06-Decision-Records/README.md @@ -1,7 +1,7 @@ # Architecture Decision Records (ADRs) -**Version:** 1.8.0 -**Last Updated:** 2026-02-24 +**Version:** 1.8.1 +**Last Updated:** 2026-03-16 **Project:** LCBP3-DMS (Laem Chabang Port Phase 3 - Document Management System) --- @@ -43,7 +43,7 @@ Architecture Decision Records (ADRs) เป็นเอกสารที่บ | ADR | Title | Status | Date | Summary | | --------------------------------------------------- | ------------------------------------ | -------------------- | ---------- | ------------------------------------------------------------ | -| [ADR-005](./ADR-005-technology-stack.md) | Technology Stack Selection | ✅ Accepted | 2026-02-24 | Full Stack TypeScript: NestJS + Next.js + MariaDB + Redis | +| [ADR-005](./ADR-005-technology-stack.md) | Technology Stack Selection | ✅ Accepted | 2026-02-24 | Full Stack TypeScript: NestJS 11 + Next.js 16 + MariaDB + Redis | | [ADR-006](./ADR-006-redis-caching-strategy.md) | Redis Usage & Caching Strategy | ✅ Accepted | 2026-02-24 | Redis สำหรับ Distributed Lock, Cache, Queue, และ Rate Limiting | | [ADR-009](./ADR-009-database-migration-strategy.md) | Database Migration & Deployment | ✅ Accepted (Pending) | 2026-02-24 | TypeORM Migrations พร้อม Blue-Green Deployment | | [ADR-015](./ADR-015-deployment-infrastructure.md) | Deployment & Infrastructure Strategy | ✅ Accepted | 2026-02-24 | Docker Compose with Blue-Green Deployment on QNAP | @@ -69,6 +69,12 @@ Architecture Decision Records (ADRs) เป็นเอกสารที่บ | [ADR-013](./ADR-013-form-handling-validation.md) | Form Handling & Validation | ✅ Accepted | 2026-02-24 | React Hook Form + Zod for Type-Safe Forms | | [ADR-014](./ADR-014-state-management.md) | State Management Strategy | ✅ Accepted | 2026-02-24 | Zustand for Client State + Server Components | +### Data & Identity + +| ADR | Title | Status | Date | Summary | +| ------------------------------------------------------------ | ---------------------------- | ---------- | ---------- | -------------------------------------------------------- | +| [ADR-019](./ADR-019-hybrid-identifier-strategy.md) | Hybrid Identifier Strategy | ✅ Accepted | 2026-03-11 | INT PK (internal) + UUIDv7 (public API) on 14 tables | + --- ## 🔍 ADR Categories @@ -91,7 +97,7 @@ Architecture Decision Records (ADRs) เป็นเอกสารที่บ ### 4. Infrastructure & Performance -- **ADR-005:** Technology Stack - TypeScript ecosystem +- **ADR-005:** Technology Stack - TypeScript ecosystem (NestJS 11, Next.js 16) - **ADR-006:** Redis - Caching และ Distributed coordination - **ADR-015:** Deployment - Docker Compose with Blue-Green Deployment @@ -110,6 +116,10 @@ Architecture Decision Records (ADRs) เป็นเอกสารที่บ - **ADR-013:** Form Handling - React Hook Form + Zod Validation - **ADR-014:** State Management - Zustand + Server Components +### 8. Data & Identity + +- **ADR-019:** Hybrid Identifier Strategy - INT PK (internal) + UUIDv7 (public API) บน 14 tables + --- ## 📖 How to Read ADRs diff --git a/specs/README.md b/specs/README.md index bd6321a..40b6a67 100644 --- a/specs/README.md +++ b/specs/README.md @@ -1,7 +1,7 @@ # 📚 LCBP3-DMS Specifications Directory **Version:** 1.8.1 (Patch) -**Last Updated:** 2026-03-11 +**Last Updated:** 2026-03-16 **Project:** LCBP3-DMS (Laem Chabang Port Phase 3 - Document Management System) **Status:** ✅ UAT Ready — 10/10 Documentation Gaps Closed @@ -68,16 +68,18 @@ specs/ │ └── README.md # Infrastructure & Operations Guide │ ├── 05-Engineering-Guidelines/ # มาตรฐานการพัฒนาและการเขียนโค้ด -│ ├── 05-01-fullstack-js-guidelines.md # JS/TS Guidelines รวมๆ +│ ├── 05-01-fullstack-js-guidelines.md # JS/TS Guidelines + NestJS 11 Patterns │ ├── 05-02-backend-guidelines.md # NestJS Backend, Error Handling │ ├── 05-03-frontend-guidelines.md # UI/UX, React Hook Form, State Strategy │ ├── 05-04-testing-strategy.md # Unit/E2E Testing ยุทธศาสตร์ │ ├── 05-05-git-cheatsheet.md # การใช้ Git สำหรับทีมงาน +│ ├── 05-07-hybrid-uuid-implementation-plan.md # ADR-019 Implementation Guide │ └── README.md # ภาพรวมเป้าหมายงาน Engineering │ -├── 06-Decision-Records/ # Architecture Decision Records (17 + 1 Patch) +├── 06-Decision-Records/ # Architecture Decision Records (17 + Patch + ADR-019) │ ├── ADR-001 to ADR-017... # ไฟล์อธิบายสถาปัตยกรรม (ADR) │ ├── ADR-018-ai-boundary.md # ★ Patch 1.8.1: AI/Ollama Isolation Policy +│ ├── ADR-019-hybrid-identifier-strategy.md # ★ Hybrid ID: INT PK + UUIDv7 Public API │ └── README.md # รายชื่อ ADR ทั้งหมดพร้อมสถานะและวันที่ │ └── 99-archives/ # ประวัติการทำงานและ Tasks เก่า @@ -122,6 +124,7 @@ specs/ | **UAT Criteria** | `01-Requirements/01-05-acceptance-criteria.md` | ตรวจความสมบูรณ์ Feature | | **ADR-009** | `06-Decision-Records/ADR-009-db-strategy.md` | Schema Change Process | | **ADR-018** | `06-Decision-Records/ADR-018-ai-boundary.md` | AI/Ollama Integration Rules | +| **ADR-019** | `06-Decision-Records/ADR-019-hybrid-identifier-strategy.md` | Hybrid ID Strategy (INT + UUIDv7) | --- @@ -145,13 +148,13 @@ specs/ --- -## 🏛️ ADR Reference (All 17 + 1 Patch) +## 🏛️ ADR Reference (All 17 + Patch + ADR-019) | ADR | Topic | Key Decision | |-----|-------|-------------| | ADR-001 | Workflow Engine | Unified state machine for document workflows | | ADR-002 | Doc Numbering | Redis Redlock + DB optimistic locking | -| ADR-005 | Technology Stack | NestJS + Next.js + MariaDB + Redis | +| ADR-005 | Technology Stack | NestJS 11 + Next.js 16 + MariaDB + Redis | | ADR-006 | Redis Caching | Cache strategy and invalidation patterns | | ADR-008 | Email Notification | BullMQ queue-based email/LINE/in-app | | ADR-009 | DB Strategy | No TypeORM migrations — modify schema SQL directly | @@ -164,5 +167,6 @@ specs/ | ADR-016 | Security | JWT + CASL RBAC + Helmet.js + ClamAV | | ADR-017 | Ollama Migration | Local AI + n8n for legacy data import | | ADR-018 ★ | AI Boundary (Patch 1.8.1) | AI isolation — no direct DB/storage access | +| ADR-019 ★ | Hybrid Identifier Strategy | INT PK (internal) + UUIDv7 (public API) | > **Priority:** `06-Decision-Records` > `05-Engineering-Guidelines` > others