690403:2205 Modify AI (Add Gemma4 & PaddleOCR
This commit is contained in:
@@ -2,15 +2,17 @@
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-02-26
|
||||
**Version:** 1.8.0
|
||||
**Decision Makers:** Development Team, DevOps Engineer
|
||||
**Version:** 1.8.3 (Aligned with ADR-020)
|
||||
**Decision Makers:** Development Team, DevOps Engineer, AI Integration Lead
|
||||
**Related Documents:**
|
||||
|
||||
- [ADR-020: AI Intelligence Integration Architecture](./ADR-020-ai-intelligence-integration.md) — Overall AI Architecture & RFA-First Strategy
|
||||
- [Legacy Data Migration Plan](../03-Data-and-Storage/03-04-legacy-data-migration.md)
|
||||
- [n8n Migration Setup Guide](../03-Data-and-Storage/03-05-n8n-migration-setup-guide.md)
|
||||
- [ADR-019: Hybrid Identifier Strategy](./ADR-019-hybrid-identifier-strategy.md) — UUID Strategy สำหรับ DB Lookup
|
||||
- [Software Architecture](../02-Architecture/02-02-software-architecture.md)
|
||||
- [Data Dictionary](../03-Data-and-Storage/03-01-data-dictionary.md)
|
||||
> **Note:** ADR-017 is clarified and hardened by ADR-018 regarding AI physical isolation. Category Enum system-driven, Idempotency Contract, Duplicate Handling Clarification, Storage Enforcement, Audit Log Enhancement, Review Queue Integration, Revision Drift Protection, Execution Time, Encoding Normalization, Security Hardening, Orchestrator on QNAP, AI Physical Isolation (Desktop Desk-5439).
|
||||
> **Note:** ADR-017 is clarified and hardened by ADR-018 regarding AI physical isolation. Now part of unified ADR-020 architecture with RFA-First approach, Gemma 4 model, and comprehensive Human-in-the-Loop validation.
|
||||
|
||||
---
|
||||
|
||||
@@ -92,10 +94,10 @@
|
||||
| Component | รายละเอียด |
|
||||
| ---------------------- | ------------------------------------------------------------------------------------------------------------ |
|
||||
| Migration Orchestrator | n8n (Docker บน QNAP NAS) |
|
||||
| AI Model Primary | Ollama `llama3.2:3b` (Validation, Summarization, Tagging) |
|
||||
| AI Model Primary | Ollama `gemma4:9b` (9.6 GB, Gemma 4 9B) — Validation, Summarization, Tagging |
|
||||
| AI Model Fallback | Ollama `mistral:7b-instruct-q4_K_M` |
|
||||
| Hardware | QNAP NAS (Orchestrator) + Desktop Desk-5439 (AI Processing, RTX 2060 SUPER 8GB) |
|
||||
| DB Lookup (n8n) | n8n ทำการ Query `project_id`, `organization_id` และดึง `Tags` จาก DB ให้ AI |
|
||||
| DB Lookup (n8n) | n8n ทำการ Query `project_uuid`, `organization_uuid` และดึง `Tags` จาก DB ให้ AI (ADR-019) |
|
||||
| Data Ingestion | 1. Staging ลง `migration_review_queue` -> 2. กดยืนยันผ่าน Frontend Management UI -> 3. Final Commit ผ่าน API |
|
||||
| Concurrency (n8n) | Sequential — Batch Size 50-100 ป้องกัน DB Connection Overload |
|
||||
| Checkpoint | MariaDB `migration_progress` และการใช้ `ON DUPLICATE KEY UPDATE` ใน Staging |
|
||||
@@ -387,3 +389,12 @@ IF excel_revision != current_db_revision + 1
|
||||
---
|
||||
|
||||
_สำหรับขั้นตอนปฏิบัติงานแบบละเอียด ดูที่ `03-04-legacy-data-migration.md` และ `03-05-n8n-migration-setup-guide.md`_
|
||||
|
||||
---
|
||||
|
||||
## Document History
|
||||
|
||||
| Version | Date | Author | Changes |
|
||||
| ------- | ---------- | ----------- | -------------------------------------------------------------------- |
|
||||
| 1.8.0 | 2026-02-26 | DevOps Team | Initial ADR — Ollama + n8n Migration Architecture |
|
||||
| 1.8.2 | 2026-04-03 | Tech Lead | **Updated** — Aligned with ADR-019 (UUID Strategy), changed AI Model to `gemma4:9b` (9.6 GB) |
|
||||
|
||||
@@ -0,0 +1,317 @@
|
||||
# ADR-017B: Smart Legacy Document Digitization (AI-Powered Use Case Extension)
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-03-27
|
||||
**Version:** 1.8.2 (Aligned with ADR-020)
|
||||
**Decision Makers:** Development Team, AI Integration Lead
|
||||
**Related Documents:**
|
||||
|
||||
- [ADR-020: AI Intelligence Integration Architecture](./ADR-020-ai-intelligence-integration.md) — Overall AI Architecture & RFA-First Strategy
|
||||
- [ADR-017: Ollama Data Migration Architecture](./ADR-017-ollama-data-migration.md)
|
||||
- [ADR-018: AI Boundary Policy](./ADR-018-ai-boundary.md) — AI Physical Isolation (No Direct DB/Storage Access)
|
||||
- [n8n Migration Setup Guide](../03-Data-and-Storage/03-05-n8n-migration-setup-guide.md)
|
||||
- [Legacy Data Migration Plan](../03-Data-and-Storage/03-04-legacy-data-migration.md)
|
||||
- [Migration Business Scope](../03-Data-and-Storage/03-06-migration-business-scope.md)
|
||||
- [Glossary](../00-Overview/00-02-glossary.md)
|
||||
|
||||
> **Note:** ADR-017B เป็น Use Case Extension ของ ADR-017 ที่ขยายขอบเขตการใช้งาน Ollama จาก Migration Batch สู่ระบบจัดหมวดหมู่เอกสารอัจฉริยะ (Smart Categorization) พร้อมควบคุมตาม ADR-018 (AI Isolation Policy) และเป็นส่วนหนึ่งของ ADR-020 (Unified AI Architecture).
|
||||
|
||||
---
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
### ปัญหาที่ต้องการแก้ไข
|
||||
|
||||
โครงการ LCBP3 มีเอกสารเก่าจำนวนมาก (กว่า 20,000 ฉบับ) ที่ต้องนำเข้าสู่ระบบ DMS ใหม่ การจัดหมวดหมู่และสกัด Metadata ด้วยมือมีปัญหาหลัก 3 ประการ:
|
||||
|
||||
1. **Manual Labor สูง:** ต้องใช้เวลานานในการอ่านและพิมพ์ข้อมูล Metadata (Data Entry)
|
||||
2. **Human Error:** ความผิดพลาดจากการพิมพ์ (Typo) โดยเฉพาะเลขที่สัญญาและชื่อเฉพาะ
|
||||
3. **Searchability:** ไฟล์ PDF ที่เป็นแค่ภาพสแกน (Scanned) ไม่สามารถค้นหาเนื้อหาได้
|
||||
|
||||
### ข้อจำกัดที่สำคัญ
|
||||
|
||||
- **Data Privacy:** เอกสารก่อสร้างท่าเรือเป็นความลับ ห้ามส่งข้อมูลขึ้น Cloud AI Provider
|
||||
- **Cost Constraint:** การใช้ Cloud AI (~$0.01–0.03 ต่อ Record) อาจสูงถึง $600 สำหรับ 20,000 records
|
||||
- **Hardware Limitation:** ต้องรันบนเครื่องที่มีอยู่ (i7-9700K / 32GB RAM / RTX 2060 Super 8GB)
|
||||
|
||||
---
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
- **Security First (ADR-018):** AI ต้องรันบน Admin Desktop (Desk-5439) เท่านั้น — ห้ามรันบน QNAP/Production Server
|
||||
- **Privacy Guaranteed:** ประมวลผลภายในเครือข่ายองค์กร (On-Premise) เท่านั้น
|
||||
- **Cost Effectiveness:** Zero Cost สำหรับ AI Inference (ไม่มีค่า Pay-per-use)
|
||||
- **Data Integrity:** ข้อมูลที่ AI สกัดต้องผ่าน Human Verification ก่อน Commit ลงระบบจริง
|
||||
- **AI Isolation (ADR-018):** AI ห้ามเข้าถึง Database/Storage โดยตรง — ต้องสื่อสารผ่าน DMS API เท่านั้น
|
||||
- **Recoverability:** รองรับ Checkpoint/Resume และ Rollback ได้สมบูรณ์
|
||||
|
||||
---
|
||||
|
||||
## Considered Options
|
||||
|
||||
### Option 1: Manual Data Entry (No AI)
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ไม่ต้องลงทุน Hardware เพิ่ม
|
||||
- ความแม่นยำ 100% (ถ้าพิมพ์ถูก)
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ ใช้เวลานานมาก (อาจเป็นสัปดาห์หรือเดือน)
|
||||
- ❌ Human Error สูง (Typo, Inconsistency)
|
||||
- ❌ ไม่สามารถทำให้ Scanned PDF ค้นหาได้
|
||||
|
||||
### Option 2: Cloud AI Service (OpenAI, Google Vision, Azure AI)
|
||||
|
||||
**Pros:**
|
||||
|
||||
- AI ฉลาดสูง แม่นยำมาก
|
||||
- ไม่ต้องดูแล Infrastructure
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ **ผิดนโยบาย Data Privacy** — เอกสารก่อสร้างท่าเรือเป็นความลับ
|
||||
- ❌ **ค่าใช้จ่ายสูง** (~$600 สำหรับ 20,000 records)
|
||||
- ❌ Dependency กับ External Service
|
||||
|
||||
### Option 3: Local AI (Ollama + Gemma 4 9B) + n8n + Human Verification ⭐ (Selected)
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ✅ **Privacy Guaranteed** — รันภายในเครือข่ายองค์กร
|
||||
- ✅ **Zero Cost** — ไม่มีค่า Pay-per-use
|
||||
- ✅ **AI Isolation (ADR-018)** — AI รันบน Desktop แยกต่างหาก ไม่เข้าถึง DB โดยตรง
|
||||
- ✅ **Human-in-the-Loop** — Admin ตรวจสอบก่อน Commit
|
||||
- ✅ **Recoverability** — รองรับ Checkpoint/Resume
|
||||
- ✅ **Clean Architecture** — แยก Migration Logic ออกจาก Core Application
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ ต้องเปิด Desktop ทิ้งไว้ดูแล GPU Temperature
|
||||
- ❌ Model ขนาดเล็กอาจแม่นยำน้อยกว่า Cloud AI → ต้องมี Human Review Queue
|
||||
- ❌ ใช้เวลา Migration ~16.6 ชั่วโมง (~3–4 คืน)
|
||||
|
||||
---
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
**Chosen Option:** Option 3 — Local AI (Ollama + Gemma 4 9B) + n8n + Human Verification
|
||||
|
||||
**Rationale:**
|
||||
|
||||
ประยุกต์ใช้ Hardware ที่มีอยู่ (i7-9700K + RTX 2060 Super) โดยไม่ขัดหลัก Privacy และ Security ของโครงการ n8n ช่วยลด Risk ที่จะกระทบ Core Backend และรองรับ Checkpoint/Resume ได้ดีกว่าการเขียน Script เอง นอกจากนี้ยังสอดคล้องกับ **ADR-018 (AI Boundary)** ที่กำหนดให้ AI ต้องรันบน Admin Desktop แยกต่างหาก และไม่สามารถเข้าถึง Database/Storage โดยตรงได้
|
||||
|
||||
---
|
||||
|
||||
## Implementation Architecture
|
||||
|
||||
### System Components
|
||||
|
||||
| Component | รายละเอียด | ที่ตั้ง |
|
||||
|-----------|-----------|---------|
|
||||
| **Migration Orchestrator** | n8n (Docker บน QNAP NAS) | QNAP NAS |
|
||||
| **AI Processing Engine** | Ollama + Gemma 4 9B (9.6 GB) | Admin Desktop (Desk-5439) |
|
||||
| **OCR Engine** | Tesseract หรือ Google Vision (On-prem) | Same as AI |
|
||||
| **Verification UI** | Next.js Frontend — Review Mode | Web Browser |
|
||||
| **Database** | MariaDB — Staging + Production | QNAP NAS |
|
||||
| **File Storage** | Two-Phase Storage (Temp → Permanent) | NAS Mounts |
|
||||
|
||||
### Workflow (Main Success Scenario)
|
||||
|
||||
```
|
||||
[1. Ingestion] → Batch Upload (PDF/Scan)
|
||||
↓
|
||||
[2. Pre-processing] → File Validation (NestJS)
|
||||
↓
|
||||
[3. OCR Layer] → Raw Text Extraction (Tesseract)
|
||||
↓
|
||||
[4. AI Analysis] → Gemma 4 9B via Ollama (Desk-5439)
|
||||
↓
|
||||
[5. Staging] → migration_review_queue (MariaDB)
|
||||
↓
|
||||
[6. Verification] → Admin Review UI (Next.js)
|
||||
↓
|
||||
[7. Commit] → POST /api/migration/commit_batch
|
||||
↓
|
||||
[8. Finalization] → Permanent Storage (QNAP)
|
||||
```
|
||||
|
||||
### AI Processing Flow (ADR-018 Compliant)
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Scanned PDF │────▶│ OCR Engine │────▶│ Raw Text │
|
||||
│ (Staging) │ │ (Tesseract) │ │ (Text) │
|
||||
└─────────────────┘ └─────────────────┘ └────────┬────────┘
|
||||
│
|
||||
┌──────────────────────────┘
|
||||
▼
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ DMS Backend │◀────│ Ollama API │◀────│ AI Prompt │
|
||||
│ (Validation) │ │ (Desk-5439) │ │ (Gemma 4 9B) │
|
||||
└────────┬────────┘ └─────────────────┘ └─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ JSON Metadata │────▶ migration_review_queue
|
||||
│ (Validated) │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
> ⚠️ **ADR-018 Enforcement:** AI (Ollama) อยู่บน Desktop Desk-5439 แยกต่างหาก — ไม่มี Direct DB Access หรือ File System Access โดยตรง ทุกการสื่อสารผ่าน DMS Backend API เท่านั้น
|
||||
|
||||
---
|
||||
|
||||
## AI Prompt Specification (Prompt Engineering)
|
||||
|
||||
### Document Analysis Prompt
|
||||
|
||||
```markdown
|
||||
บทบาท: คุณคือผู้ช่วยจัดการเอกสารวิศวกรรมโยธาประจำโครงการท่าเรือแหลมฉบัง เฟส 3
|
||||
ภารกิจ: อ่านข้อความดิบจากการสแกน และสรุปข้อมูลออกมาในรูปแบบ JSON เท่านั้น
|
||||
|
||||
ข้อความดิบ: [RAW_TEXT_FROM_OCR]
|
||||
|
||||
รูปแบบ JSON ที่ต้องการ:
|
||||
{
|
||||
"document_type": "รายงาน/สัญญา/จดหมายโต้ตอบ",
|
||||
"contract_number": "เลขที่สัญญา (ถ้ามี)",
|
||||
"contractor_name": "ชื่อบริษัทผู้รับเหมา",
|
||||
"work_phase": "Phase ของงาน (เช่น งานถมทะเล, งานไฟฟ้า)",
|
||||
"summary": "สรุปเนื้อหาสำคัญ 1-2 ประโยค",
|
||||
"key_dates": ["วันที่ในเอกสาร", "วันกำหนดส่งงาน"]
|
||||
}
|
||||
```
|
||||
|
||||
### Extracted Metadata Schema
|
||||
|
||||
| Field | Type | คำอธิบาย |
|
||||
|-------|------|---------|
|
||||
| `document_type` | enum | Correspondence / RFA / Drawing / Minutes / Contract |
|
||||
| `contract_number` | string | เลขที่สัญญา (เช่น LCBP3-C1) |
|
||||
| `contractor_name` | string | ชื่อบริษัทผู้รับเหมา |
|
||||
| `work_phase` | string | Phase งาน (จาก Master Data) |
|
||||
| `work_zone` | string | Zone F, Basin 3, etc. |
|
||||
| `priority` | enum | ด่วนที่สุด / ด่วน / ปกติ |
|
||||
| `summary` | string | สรุปเนื้อหา 4-5 ประโยค |
|
||||
| `confidence` | float | 0.0–1.0 (ความมั่นใจของ AI) |
|
||||
| `suggested_tags` | array | Tags ที่แนะนำ (is_new: true/false) |
|
||||
|
||||
> ⚠️ **Patch Note:** `document_type` ต้องตรงกับ System Enum จาก `GET /api/meta/categories` เท่านั้น — ห้าม hardcode Category List ใน Prompt (ดู ADR-017)
|
||||
|
||||
---
|
||||
|
||||
## Security & Compliance (ADR-018)
|
||||
|
||||
### AI Isolation Requirements
|
||||
|
||||
| Rule | Implementation |
|
||||
|------|----------------|
|
||||
| **Physical Isolation** | Ollama รันบน Admin Desktop (Desk-5439) เท่านั้น |
|
||||
| **No Direct DB Access** | AI ต้องสื่อสารผ่าน DMS API → Backend → Database |
|
||||
| **No Direct Storage Access** | File operations ผ่าน StorageService เท่านั้น |
|
||||
| **Validation Layer** | Backend ตรวจสอบ AI Output ก่อน Write ทุกครั้ง |
|
||||
| **Audit Logging** | ทุก AI Request/Response บันทึกใน Audit Log |
|
||||
|
||||
### Two-Phase Storage (Storage Governance)
|
||||
|
||||
**ข้อห้าม:**
|
||||
|
||||
```bash
|
||||
❌ mv /data/dms/staging_ai/TCC-COR-0001.pdf /final/path/...
|
||||
```
|
||||
|
||||
**ข้อบังคับ (ADR-017):**
|
||||
|
||||
```
|
||||
Phase 1: Temp Upload (โดย n8n)
|
||||
✅ POST /api/storage/upload
|
||||
→ ได้ผลลัพธ์เป็น attachment_id (เช่น 1024)
|
||||
→ ไฟล์จะถูกระบุเป็น is_temporary = TRUE
|
||||
|
||||
Phase 2: Final Commit (โดย Admin ผ่าน Frontend)
|
||||
✅ POST /api/migration/commit_batch
|
||||
body: { queue_ids: [1, 2, 3] }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Confidence Threshold Policy
|
||||
|
||||
ข้อมูลทุกชุดจาก AI จะถูกส่งเข้า `migration_review_queue` โดยจัดสถานะตาม Confidence:
|
||||
|
||||
| Confidence Level | สถานะ | การดำเนินการ |
|
||||
|-----------------|-------|--------------|
|
||||
| `≥ 0.85` และ `is_valid = true` | PENDING | พร้อมให้ Admin Batch Import |
|
||||
| `0.60–0.84` | PENDING | ไฮไลต์ให้ Admin ตรวจสอบก่อน |
|
||||
| `< 0.60` หรือ `is_valid = false` | REJECTED | รอ Admin แก้ไข Manual |
|
||||
| AI Parse Error | ERROR_LOG | Trigger Fallback Logic |
|
||||
|
||||
---
|
||||
|
||||
## Performance Estimation
|
||||
|
||||
| Parameter | ค่า |
|
||||
|-----------|-----|
|
||||
| Delay ระหว่าง Request | 2 วินาที |
|
||||
| Inference Time (avg) | ~1 วินาที |
|
||||
| เวลาต่อ Record | ~3 วินาที |
|
||||
| จำนวน Record | 20,000 |
|
||||
| **เวลารวม** | ~60,000 วินาที (~16.6 ชั่วโมง) |
|
||||
| **จำนวนคืนที่ต้องใช้** | **~3–4 คืน** (รัน 22:00–06:00) |
|
||||
|
||||
> **Recommendation:** สำหรับเครื่อง i7-9700K / 32GB RAM / RTX 2060 Super 8GB แนะนำให้ใช้ Queue (BullMQ) ประมวลผลไฟล์แบบ Sequential (ทีละไฟล์) เพื่อให้ Gemma 4 9B (9.6 GB) ทำงานได้เสถียรที่สุดใน VRAM 8GB
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive Consequences
|
||||
|
||||
1. ✅ **Privacy Guaranteed** — ไม่มีข้อมูลออกนอกเครือข่ายองค์กร
|
||||
2. ✅ **Zero AI Cost** — ไม่มีค่าใช้จ่าย Pay-per-use
|
||||
3. ✅ **Security Compliant (ADR-018)** — AI Isolation ชัดเจน
|
||||
4. ✅ **Human-in-the-Loop** — ความถูกต้อง 100% ก่อน Commit
|
||||
5. ✅ **Recoverability** — รองรับ Checkpoint/Resume และ Rollback
|
||||
6. ✅ **Searchable Legacy** — Scanned PDF กลายเป็น Searchable Metadata
|
||||
|
||||
### Negative Consequences
|
||||
|
||||
1. ❌ **Operational Overhead** — ต้องดูแล GPU Temperature และ Desktop Uptime
|
||||
2. ❌ **Accuracy Trade-off** — Model ขนาดเล็กอาจแม่นยำน้อยกว่า Cloud AI
|
||||
3. ❌ **Time Investment** — ต้องใช้เวลา ~3–4 คืนในการ Migration
|
||||
4. ❌ **Hardware Dependency** — ต้องพึ่งพา Desktop Desk-5439
|
||||
|
||||
### Mitigation Strategies
|
||||
|
||||
- **Human Review Queue** — บังคับตรวจสอบทุก record ก่อน Commit
|
||||
- **Confidence Threshold** — แบ่งระดับตามความมั่นใจของ AI
|
||||
- **Fallback Model** — Auto-switch ไป mistral:7b-instruct เมื่อ Error ≥ Threshold
|
||||
- **Monitoring** — ตรวจสอบ GPU Temperature และ Progress ตลอดเวลา
|
||||
|
||||
---
|
||||
|
||||
## Related Documents
|
||||
|
||||
- [ADR-017: Ollama Data Migration Architecture](./ADR-017-ollama-data-migration.md) — Architecture หลักสำหรับ Migration
|
||||
- [ADR-018: AI Boundary Policy](./ADR-018-ai-boundary.md) — Security Isolation สำหรับ AI
|
||||
- [03-05-n8n-migration-setup-guide.md](../03-Data-and-Storage/03-05-n8n-migration-setup-guide.md) — คู่มือติดตั้ง n8n Workflow
|
||||
- [03-04-legacy-data-migration.md](../03-Data-and-Storage/03-04-legacy-data-migration.md) — แผน Migration แบบละเอียด
|
||||
- [03-06-migration-business-scope.md](../03-Data-and-Storage/03-06-migration-business-scope.md) — Go/No-Go Gates และ Business Scope
|
||||
- [00-02-glossary.md](../00-Overview/00-02-glossary.md) — คำศัพท์และตัวย่อในระบบ DMS
|
||||
|
||||
---
|
||||
|
||||
## Document History
|
||||
|
||||
| Version | Date | Author | Changes |
|
||||
|---------|------------|------------|---------|
|
||||
| 1.0.0 | 2026-02-26 | Nattanin | Initial Use Case Specification |
|
||||
| 1.8.1 | 2026-03-27 | Tech Lead | **Refactored to ADR format** — Aligned with ADR-017, ADR-018, and Project Specs |
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-03-27
|
||||
**Status:** Accepted
|
||||
**Next Review:** 2026-06-01 (Quarterly review)
|
||||
@@ -0,0 +1,406 @@
|
||||
# ADR-018: AI Boundary Policy (AI Isolation)
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-03-27
|
||||
**Version:** 1.8.2 (Aligned with ADR-020)
|
||||
**Decision Makers:** Security Team, System Architect, AI Integration Lead
|
||||
**Related Documents:**
|
||||
|
||||
- [ADR-020: AI Intelligence Integration Architecture](./ADR-020-ai-intelligence-integration.md) — Overall AI Architecture & RFA-First Strategy
|
||||
- [ADR-017: Ollama Data Migration Architecture](./ADR-017-ollama-data-migration.md)
|
||||
- [ADR-017B: Smart Legacy Document Digitization](./ADR-017B-ollama.md)
|
||||
- [ADR-016: Security & Authentication](./ADR-016-security-authentication.md)
|
||||
- [ADR-019: Hybrid Identifier Strategy](./ADR-019-hybrid-identifier-strategy.md)
|
||||
- [n8n Migration Setup Guide](../03-Data-and-Storage/03-05-n8n-migration-setup-guide.md)
|
||||
- [RAG Architecture](../03-Data-and-Storage/03-07-OpenRAG.md)
|
||||
|
||||
> **หมายเหตุ:** ADR-018 เป็น Security Policy หลักที่ควบคุมการทำงานของ AI Components ทั้งหมดในระบบ LCBP3-DMS ทุก Use Case ที่ใช้ AI (Migration, RAG, Smart Categorization) ต้องปฏิบัติตาม Policy นี้ และเป็นส่วนหนึ่งของ ADR-020 (Unified AI Architecture).
|
||||
|
||||
---
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
### ปัญหาที่ต้องการแก้ไข
|
||||
|
||||
การนำ AI (Ollama, OpenRAG, หรือ LLM อื่นๆ) เข้ามาใช้งานในระบบ DMS ที่มีเอกสารสำคัญและข้อมูล Confidential ของโครงการท่าเรือ Laem Chabang Phase 3 มีความเสี่ยงด้าน Security หลัก 4 ประการ:
|
||||
|
||||
1. **Data Exposure Risk:** หาก AI มีสิทธิ์เข้าถึง Database โดยตรง อาจมีการรั่วไหลของข้อมูลทางการค้า / การก่อสร้าง
|
||||
2. **Unauthorized Data Modification:** AI อาจทำการแก้ไขข้อมูลโดยไม่มีการตรวจสอบจาก Human
|
||||
3. **Privilege Escalation:** หาก AI ถูก compromise อาจใช้สิทธิ์ Database Access เพื่อโจมตีระบบอื่น
|
||||
4. **Compliance Violation:** ไม่สอดคล้องกับมาตรฐาน ISO 27001 และ PDPA สำหรับข้อมูลส่วนบุคคล
|
||||
|
||||
### ข้อจำกัดด้าน Infrastructure
|
||||
|
||||
- **QNAP NAS:** เป็น Production Server ที่ไม่ควรรัน AI Workload (Resource contention + Security boundary)
|
||||
- **Admin Desktop (Desk-5439):** เครื่องสำหรับ Admin มี GPU (RTX 2060 Super 8GB) เหมาะสำหรับ AI Inference
|
||||
- **Network Segmentation:** ต้องแยก Zone ระหว่าง AI Processing (Untrusted) กับ Database (Trusted)
|
||||
|
||||
---
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
- **Zero Trust Architecture:** AI ถือเป็น Untrusted Component เสมอ ไม่ว่าเป็น On-Premise หรือไม่
|
||||
- **Defense in Depth:** หลายชั้นของการควบคุม (Physical → Network → API → Data)
|
||||
- **Auditability:** ทุกการสื่อสารกับ AI ต้อง Log ได้
|
||||
- **Human-in-the-Loop:** ข้อมูลจาก AI ต้องผ่าน Human Validation ก่อน Commit ลง Database
|
||||
- **Minimal Privilege:** AI ได้รับสิทธิ์น้อยที่สุด (Principle of Least Privilege)
|
||||
|
||||
---
|
||||
|
||||
## Considered Options
|
||||
|
||||
### Option 1: AI รันบน QNAP NAS (Same Host กับ Database)
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ✅ ติดตั้งง่าย ไม่ต้องดูแลหลายเครื่อง
|
||||
- ✅ Network Latency ต่ำ (localhost)
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ **Security Risk สูง:** AI มี Direct Access ถึง Database หากถูก compromise
|
||||
- ❌ **Resource Contention:** AI Inference กิน RAM/CPU สูง กระทบ Production Services
|
||||
- ❌ **No Isolation:** ไม่มี Security Boundary ระหว่าง AI กับ Core Application
|
||||
|
||||
### Option 2: AI บน Cloud AI Provider (OpenAI, Google, Azure)
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ✅ AI ฉลาดสูง แม่นยำมาก
|
||||
- ✅ ไม่ต้องดูแล Hardware
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ **ผิดนโยบาย Data Privacy:** เอกสารก่อสร้างท่าเรือเป็นความลับ ห้ามส่งข้อมูลขึ้น Cloud
|
||||
- ❌ **Cost สูง:** Pay-per-use ไม่เหมาะกับงานประมวลผลจำนวนมาก
|
||||
- ❌ **No Control:** ไม่สามารถควบคุม Data Retention หรือ Audit ได้
|
||||
|
||||
### Option 3: Physical Isolation + API-only Communication ⭐ (Selected)
|
||||
|
||||
**Pros:**
|
||||
|
||||
- ✅ **Security Boundary ชัดเจน:** AI รันบน Desktop แยกต่างหาก ไม่เข้าถึง DB โดยตรง
|
||||
- ✅ **Zero Trust:** AI ถือเป็น Untrusted Component สื่อสารผ่าน API เท่านั้น
|
||||
- ✅ **Audit Trail:** ทุก Request/Response ผ่าน Backend ซึ่งมี Audit Log ครบถ้วน
|
||||
- ✅ **Human-in-the-Loop:** Backend ตรวจสอบข้อมูลก่อน Write ลง Database
|
||||
- ✅ **Resource Isolation:** AI Workload ไม่กระทบ Production Services บน QNAP
|
||||
- ✅ **Compliance:** สอดคล้องกับ ISO 27001 และ PDPA
|
||||
|
||||
**Cons:**
|
||||
|
||||
- ❌ ต้องดูแลเครื่อง Desktop เพิ่ม (GPU Temperature, Uptime)
|
||||
- ❌ Network Latency เพิ่มขึ้นเล็กน้อย (LAN traffic)
|
||||
- ❌ ต้องออกแบบ API Contract ให้รัดกุม
|
||||
|
||||
---
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
**Chosen Option:** Option 3 — Physical Isolation + API-only Communication
|
||||
|
||||
**Rationale:**
|
||||
|
||||
การแยก AI ไปรันบน Admin Desktop (Desk-5439) และบังคับให้สื่อสารผ่าน DMS Backend API เท่านั้น เป็นแนวทางที่ Balance ระหว่าง Security, Privacy, และ Operational Feasibility ดีที่สุด ทำให้ AI ถูกมองว่าเป็น **Untrusted External Component** เสมอ แม้จะรันในเครือข่ายเดียวกัน
|
||||
|
||||
---
|
||||
|
||||
## AI Isolation Architecture
|
||||
|
||||
### Infrastructure Layout
|
||||
|
||||
| Component | Host | Zone | Network Access | Database Access |
|
||||
|-----------|------|------|----------------|-----------------|
|
||||
| **Ollama / OpenRAG** | Admin Desktop (Desk-5439) | Untrusted (AI Zone) | LAN only (QNAP NAS mount) | ❌ **ไม่มี** |
|
||||
| **DMS Backend** | QNAP NAS (Docker) | Trusted (App Zone) | LAN + Frontend | ✅ Full Access |
|
||||
| **MariaDB** | QNAP NAS | Trusted (DB Zone) | Localhost only | — |
|
||||
| **n8n** | QNAP NAS (Docker) | Trusted (Orchestrator) | LAN + DB | ✅ Via API only |
|
||||
|
||||
### Communication Flow
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Untrusted Zone (AI Zone) │
|
||||
│ Admin Desktop (Desk-5439) — RTX 2060 Super 8GB │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ Ollama (LLM) │ │ OpenRAG │ │
|
||||
│ │ Port: 11434 │ │ (Docling) │ │
|
||||
│ └────────┬────────┘ └────────┬────────┘ │
|
||||
└───────────┼─────────────────────┼───────────────────────────────┘
|
||||
│ │
|
||||
│ HTTP API │ Write JSON
|
||||
│ │
|
||||
┌───────────┼─────────────────────┼───────────────────────────────┐
|
||||
│ ▼ ▼ │
|
||||
│ Trusted Zone (App Zone) │
|
||||
│ QNAP NAS (Docker) │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌────────────┐ │
|
||||
│ │ DMS Backend │◀───│ n8n │ │ MariaDB │ │
|
||||
│ │ (NestJS) │ │ (Poll JSON) │ │ (Auth DB) │ │
|
||||
│ │ Port: 3001 │ └─────────────────┘ └────────────┘ │
|
||||
│ └────────┬────────┘ │
|
||||
│ │ │
|
||||
│ │ Validation + Audit Log │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────┐ │
|
||||
│ │ Database │ │
|
||||
│ │ (MariaDB) │ │
|
||||
│ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
> ⚠️ **ข้อห้าม:** Ollama/OpenRAG **ห้าม** อยู่บน QNAP NAS และ **ห้าม** มี Database Connection String
|
||||
|
||||
---
|
||||
|
||||
## Security Rules (Non-Negotiable)
|
||||
|
||||
### Rule 1: Physical Isolation
|
||||
|
||||
| ข้อกำหนด | รายละเอียด |
|
||||
|----------|------------|
|
||||
| **AI Host** | Admin Desktop (Desk-5439) เท่านั้น |
|
||||
| **Forbidden Hosts** | QNAP NAS, Production Servers, Cloud VM |
|
||||
| **Hardware** | i7-9700K / 32GB RAM / RTX 2060 Super 8GB |
|
||||
| **Network** | LAN (192.168.x.x) — No Public IP |
|
||||
|
||||
### Rule 2: No Direct Database Access
|
||||
|
||||
```typescript
|
||||
// ❌ FORBIDDEN — AI ห้ามเชื่อมต่อ Database โดยตรง
|
||||
const connection = await mysql.createConnection({
|
||||
host: '192.168.1.100',
|
||||
user: 'ai_service', // NEVER!
|
||||
password: '***',
|
||||
database: 'lcbp3_dms'
|
||||
});
|
||||
|
||||
// ✅ CORRECT — AI สื่อสารผ่าน DMS Backend API เท่านั้น
|
||||
const response = await fetch('http://192.168.1.100:3001/api/ai/analyze', {
|
||||
method: 'POST',
|
||||
headers: { 'Authorization': 'Bearer ' + ai_token },
|
||||
body: JSON.stringify({ text: extractedText })
|
||||
});
|
||||
```
|
||||
|
||||
### Rule 3: No Direct Storage Access
|
||||
|
||||
```bash
|
||||
# ❌ FORBIDDEN — AI ห้ามเข้าถึง File System โดยตรง
|
||||
mv /data/dms/uploads/TCC-COR-0001.pdf /final/path/
|
||||
cp /staging_ai/*.pdf /processed/
|
||||
|
||||
# ✅ CORRECT — ใช้ StorageService ผ่าน API เท่านั้น
|
||||
POST /api/storage/upload
|
||||
POST /api/migration/commit_batch
|
||||
```
|
||||
|
||||
### Rule 4: Validation Layer
|
||||
|
||||
```typescript
|
||||
// Backend ตรวจสอบ AI Output ทุกครั้งก่อน Write
|
||||
@Injectable()
|
||||
export class AiValidationService {
|
||||
validateAiOutput(output: AiOutputDto): ValidationResult {
|
||||
// 1. Schema Validation (Zod/class-validator)
|
||||
const schemaCheck = this.validateSchema(output);
|
||||
|
||||
// 2. Confidence Threshold (≥ 0.85 auto-approve, 0.60–0.84 review, < 0.60 reject)
|
||||
const confidenceCheck = this.checkConfidence(output.confidence);
|
||||
|
||||
// 3. Enum Enforcement (Category must be from System Enum)
|
||||
const enumCheck = this.validateCategoryEnum(output.suggested_category);
|
||||
|
||||
// 4. Audit Log Recording
|
||||
this.auditLog.record({
|
||||
action: 'AI_VALIDATION',
|
||||
source: 'AI_SERVICE',
|
||||
confidence: output.confidence,
|
||||
result: schemaCheck && confidenceCheck && enumCheck
|
||||
});
|
||||
|
||||
return { isValid: schemaCheck && confidenceCheck && enumCheck };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Rule 5: Audit Logging
|
||||
|
||||
| Event | Log Level | Fields |
|
||||
|-------|-----------|--------|
|
||||
| AI Request | INFO | `timestamp`, `source_ip`, `model`, `prompt_hash` |
|
||||
| AI Response | INFO | `timestamp`, `confidence`, `processing_time`, `response_hash` |
|
||||
| Validation Pass | INFO | `record_id`, `confidence`, `validator` |
|
||||
| Validation Fail | WARN | `record_id`, `reason`, `raw_response` |
|
||||
| Unauthorized Access | ERROR | `source_ip`, `attempted_action`, `blocked_by` |
|
||||
|
||||
---
|
||||
|
||||
## AI Communication Contract
|
||||
|
||||
### API Endpoint Design
|
||||
|
||||
```typescript
|
||||
// AI เรียก Backend (ผ่าน n8n หรือตรง)
|
||||
POST /api/ai/analyze-document
|
||||
Headers:
|
||||
- Authorization: Bearer {ai_service_token}
|
||||
- Idempotency-Key: {document_hash}
|
||||
- X-AI-Source: ollama | openrag
|
||||
Body:
|
||||
{
|
||||
"extracted_text": "ข้อความจาก OCR...",
|
||||
"document_type_hint": "pdf",
|
||||
"source_file": "TCC-COR-2024-001.pdf"
|
||||
}
|
||||
|
||||
Response:
|
||||
{
|
||||
"is_valid": true,
|
||||
"confidence": 0.92,
|
||||
"suggested_category": "Correspondence",
|
||||
"extracted_metadata": { ... },
|
||||
"audit_log_id": "0195..."
|
||||
}
|
||||
```
|
||||
|
||||
### Authentication for AI Services
|
||||
|
||||
| Service | Auth Method | Token Lifetime | Scope |
|
||||
|---------|-------------|----------------|-------|
|
||||
| **Ollama** | mTLS / IP Whitelist | Session-based | `ai:invoke` |
|
||||
| **n8n → Backend** | JWT (Service Account) | 1 hour | `migration:write`, `ai:read` |
|
||||
| **OpenRAG** | File-based (Shared NAS) | N/A | Write to `rag-output/` only |
|
||||
|
||||
---
|
||||
|
||||
## Data Flow Compliance
|
||||
|
||||
### Flow 1: Migration (ADR-017)
|
||||
|
||||
```
|
||||
[Scanned PDF] → [OCR on Desktop] → [Ollama AI] → [JSON Output]
|
||||
│
|
||||
▼
|
||||
[DMS Backend API] → [Validation Layer] → [Audit Log]
|
||||
│
|
||||
▼
|
||||
[Staging Table: migration_review_queue]
|
||||
│
|
||||
▼
|
||||
[Human Review] → [Commit via Frontend] → [Permanent DB + Storage]
|
||||
```
|
||||
|
||||
### Flow 2: RAG (OpenRAG)
|
||||
|
||||
```
|
||||
[PDF Folder] → [OpenRAG on Desktop] → [JSON to rag-output/]
|
||||
│
|
||||
▼
|
||||
[n8n Poll JSON] → [DMS Backend API] → [Validation + Audit]
|
||||
│
|
||||
▼
|
||||
[Elasticsearch Index + MariaDB Metadata]
|
||||
```
|
||||
|
||||
### Flow 3: Smart Categorization (ADR-017B)
|
||||
|
||||
```
|
||||
[User Upload PDF] → [Temporary Storage]
|
||||
│
|
||||
▼
|
||||
[Queue Job] → [Ollama AI via API]
|
||||
│
|
||||
▼
|
||||
[Validation Layer] → [Suggestion to User]
|
||||
│
|
||||
▼
|
||||
[User Confirm] → [Final Category Assignment]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Compliance Matrix
|
||||
|
||||
| Requirement | Implementation | Evidence |
|
||||
|-------------|----------------|----------|
|
||||
| **ISO 27001 A.9.4.1** | JWT + mTLS for AI Auth | Token logs in `audit_logs` |
|
||||
| **ISO 27001 A.12.3.1** | IP Whitelist for AI Host | `192.168.x.x` only |
|
||||
| **PDPA Data Minimization** | AI ไม่เก็บข้อมูลระยะยาว | Temporary processing only |
|
||||
| **PDPA Security** | Physical Isolation + Encryption | TLS 1.3 for all API calls |
|
||||
| **OWASP BOLA** | UUID for all identifiers | ADR-019 Compliance |
|
||||
| **Zero Trust** | API-only communication | No direct DB/Storage access |
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive Consequences
|
||||
|
||||
1. ✅ **Security Hardened:** AI treated as untrusted component — all outputs validated
|
||||
2. ✅ **Audit Trail Complete:** Every AI interaction logged with hash + timestamp
|
||||
3. ✅ **Compliance Ready:** ISO 27001 + PDPA requirements met
|
||||
4. ✅ **Operational Safety:** AI failures don't compromise Production Database
|
||||
5. ✅ **Scalability:** Can add more AI services without security redesign
|
||||
|
||||
### Negative Consequences
|
||||
|
||||
1. ❌ **Complexity:** Need to maintain separate AI host + API contracts
|
||||
2. ❌ **Latency:** Network round-trip between AI and Backend (LAN only, acceptable)
|
||||
3. ❌ **Monitoring Overhead:** Need to monitor both QNAP and Desktop systems
|
||||
4. ❌ **Token Management:** Service accounts for AI need rotation policy
|
||||
|
||||
### Mitigation Strategies
|
||||
|
||||
- **Health Check:** Ollama `/api/tags` + Backend `/health` monitoring every 60 seconds
|
||||
- **Auto-Failover:** Switch to fallback model (mistral:7b) if primary model fails
|
||||
- **Token Rotation:** Service account JWT rotated every 7 days
|
||||
- **Network Redundancy:** อุปกรณ์สำรองสำหรับ Admin Desktop (เตรียมสำรอง)
|
||||
|
||||
---
|
||||
|
||||
## Security Checklist (Pre-Deployment)
|
||||
|
||||
### 🔴 Critical (Must Pass)
|
||||
|
||||
| Check | Command/Method | Expected Result |
|
||||
|-------|---------------|-----------------|
|
||||
| AI Host Isolation | `ping 192.168.1.100` from AI Host | Success (LAN only) |
|
||||
| No DB Access from AI | `mysql -h qnap_ip -u root` from Desktop | **Connection Refused** |
|
||||
| API Auth Required | `curl http://qnap:3001/api/ai/analyze` | 401 Unauthorized |
|
||||
| Valid Token Works | `curl -H "Authorization: Bearer {valid}" ...` | 200 OK |
|
||||
| Audit Log Written | `SELECT * FROM audit_logs WHERE source='AI_SERVICE'` | Records found |
|
||||
|
||||
### 🟡 Important (Should Pass)
|
||||
|
||||
| Check | Method | Expected Result |
|
||||
|-------|--------|-----------------|
|
||||
| TLS Enabled | `curl -v https://...` | TLS 1.3 handshake |
|
||||
| IP Whitelist Active | Try from unauthorized IP | Blocked by Firewall |
|
||||
| Token Expiration | Use expired JWT | 401 Token Expired |
|
||||
| Idempotency Key | Replay same request | 200 OK (no duplicate write) |
|
||||
|
||||
---
|
||||
|
||||
## Related Documents
|
||||
|
||||
- [ADR-017: Ollama Data Migration Architecture](./ADR-017-ollama-data-migration.md) — Migration implementation following ADR-018
|
||||
- [ADR-017B: Smart Legacy Document Digitization](./ADR-017B-ollama.md) — Smart categorization use case
|
||||
- [ADR-016: Security & Authentication](./ADR-016-security-authentication.md) — General security strategy
|
||||
- [ADR-019: Hybrid Identifier Strategy](./ADR-019-hybrid-identifier-strategy.md) — UUID strategy for API security
|
||||
- [03-07-OpenRAG.md](../03-Data-and-Storage/03-07-OpenRAG.md) — RAG architecture under ADR-018
|
||||
- [03-05-n8n-migration-setup-guide.md](../03-Data-and-Storage/03-05-n8n-migration-setup-guide.md) — n8n setup with AI isolation
|
||||
|
||||
---
|
||||
|
||||
## Document History
|
||||
|
||||
| Version | Date | Author | Changes |
|
||||
| ------- | ---------- | ------------ | -------------------------------------------------------- |
|
||||
| 1.8.1 | 2026-03-27 | Security Lead| Initial ADR — AI Boundary Policy (Physical Isolation) |
|
||||
| 1.8.2 | 2026-04-03 | Tech Lead | Updated — Aligned AI Model spec with ADR-017/017B |
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-04-03
|
||||
**Status:** Accepted
|
||||
**Next Review:** 2026-06-01 (Quarterly security review)
|
||||
@@ -0,0 +1,515 @@
|
||||
# ADR-020: AI Intelligence Integration Architecture
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-04-03
|
||||
**Version:** 1.8.5
|
||||
**Decision Makers:** Development Team, AI Integration Lead, System Architect
|
||||
**Related Documents:**
|
||||
|
||||
- [ADR-017: Ollama Data Migration Architecture](./ADR-017-ollama-data-migration.md)
|
||||
- [ADR-017B: Smart Legacy Document Digitization](./ADR-017B-ollama.md)
|
||||
- [ADR-018: AI Boundary Policy](./ADR-018-ai-boundary.md) — AI Physical Isolation
|
||||
- [ADR-019: Hybrid Identifier Strategy](./ADR-019-hybrid-identifier-strategy.md) — UUID Strategy
|
||||
- [n8n Migration Setup Guide](../03-Data-and-Storage/03-05-n8n-migration-setup-guide.md)
|
||||
|
||||
> **หมายุ:** ADR-020 กำหนดสถาปัตยกรรมการผสานรวม AI Intelligence เข้ากับระบบ NAP-DMS แบบครบวงจร โดยใช้แนวทาง "RFA-First" เพื่อให้ครอบคลุมทั้งการนำเข้าเอกสารเก่า (Legacy Migration) และการสร้างเอกสารใหม่ (New Ingestion)
|
||||
|
||||
---
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
### ปัญหาที่ต้องการแก้ไข
|
||||
|
||||
ระบบ NAP-DMS v1.8.5 ต้องการเพิ่มประสิทธิภาพการทำงานกับเอกสารวิศวกรรมโดยใช้ AI Intelligence ใน 2 สถานการณ์หลัก:
|
||||
|
||||
1. **Legacy Document Migration:** มีเอกสาร PDF เก่าจำนวนมากที่ต้องนำเข้าระบบ พร้อมตรวจสอบความถูกต้องระหว่าง Metadata ใน Excel กับเนื้อหาใน PDF
|
||||
2. **New Document Ingestion:** ผู้ใช้งานอัปโหลดเอกสารใหม่และต้องการความช่วยเหลือจาก AI ในการสกัดข้อมูลอัตโนมัติ
|
||||
|
||||
### ข้อจำกัดและข้อกำหนด
|
||||
|
||||
- **Security (ADR-018):** AI ต้องรันบน Admin Desktop (Desk-5439) แยกส่วนกับระบบหลัก
|
||||
- **Data Privacy:** ห้ามส่งข้อมูลขึ้น Cloud Provider ต้องประมวลผลภายในองค์กรเท่านั้น
|
||||
- **Human-in-the-Loop:** ข้อมูลที่ AI สกัดต้องผ่านการตรวจสอบโดยมนุษย์เสมอ
|
||||
- **Thai Language Support:** ต้องรองรับเอกสารภาษาไทยและวิศวกรรม
|
||||
|
||||
---
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
- **RFA-First Approach:** เริ่มจากเอกสาร RFA (Request for Approval) ที่มีความซับซ้อนสูง
|
||||
- **Unified Architecture:** ใช้ Pipeline และ Component ร่วมกันทั้ง 2 รูปแบบการทำงาน
|
||||
- **Data Integrity:** รักษาความถูกต้องของข้อมูลเป็นสำคัญสูงสุด
|
||||
- **User Experience:** จัดหมวดหมู่ระหว่าง Batch Throughput กับ Real-time UX
|
||||
- **Cost Efficiency:** ใช้ Ollama แบบ On-Premise เพื่อลดต้นทุน
|
||||
- **Maintainability:** แยก Logic ของ AI ออกจาก Core Application
|
||||
|
||||
---
|
||||
|
||||
## Considered Options
|
||||
|
||||
### Option 1: Separate AI Systems per Use Case
|
||||
|
||||
**Pros:**
|
||||
- ✅ เชี่ยวชาญเฉพาะทาง (Specialized)
|
||||
- ✅ แยก Failure Domain
|
||||
|
||||
**Cons:**
|
||||
- ❌ Code Dupification สูง
|
||||
- ❌ บำรุงรักษายาก (Multiple systems)
|
||||
- ❌ Inconsistent AI Behavior
|
||||
|
||||
### Option 2: Unified AI Pipeline with Different Frontends ⭐ (Selected)
|
||||
|
||||
**Pros:**
|
||||
- ✅ **Single Source of Truth:** Pipeline กลางเดียว
|
||||
- ✅ **Reusable Components:** DocumentReviewForm ใช้ร่วมกันได้
|
||||
- ✅ **Consistent Quality:** Prompt และ Model เดียวกัน
|
||||
- ✅ **Easier Maintenance:** แก้ไขที่เดียว ใช้ได้ทั้งหมด
|
||||
- ✅ **Cost Effective:** ใช้ Ollama รุ่นเดียว (Gemma 4)
|
||||
|
||||
**Cons:**
|
||||
- ❌ ต้องออกแบบให้รองรับทั้ง Batch และ Real-time
|
||||
- ❌ Complex Component Design
|
||||
|
||||
---
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
**Chosen Option:** Option 2 — Unified AI Pipeline with Different Frontends
|
||||
|
||||
**Rationale:**
|
||||
|
||||
การสร้าง Pipeline กลางเดียวสำหรับ AI และใช้ Component ร่วมกันทาง Frontend จะช่วยลดความซับซ้อนในการบำรุงรักษา และรับประกันความสม่ำเสมอของคุณภาพ AI ทั้งในการนำเข้าเอกสารเก่าและใหม่
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
### Core Technology Stack
|
||||
|
||||
| Component | Technology | Host | Purpose |
|
||||
|-----------|------------|------|---------|
|
||||
| **AI Engine** | Ollama + Gemma 4 | Admin Desktop (Desk-5439) | LLM Inference |
|
||||
| **OCR Engine** | PaddleOCR | Admin Desktop (Desk-5439) | Thai/English Text Extraction |
|
||||
| **Orchestrator** | n8n | QNAP NAS (Docker) | Workflow Management |
|
||||
| **AI Gateway** | NestJS AiModule | QNAP NAS (Docker) | API Gateway & Validation |
|
||||
|
||||
### Data Flow Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ AI Processing Flow │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐ │
|
||||
│ │ Input │───▶│ n8n │───▶│ AI Services │ │
|
||||
│ │ (PDF/Excel)│ │ Workflow │ │ (OCR+LLM) │ │
|
||||
│ └─────────────┘ └─────────────┘ └────────┬────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ DMS Backend API │ │
|
||||
│ │ ┌─────────────┐ ┌─────────────┐ ┌────────────┐ │ │
|
||||
│ │ │AiService │ │Validation │ │Audit Log │ │ │
|
||||
│ │ │Gateway │◀───│Layer │◀───│Service │ │ │
|
||||
│ │ └─────────────┘ └─────────────┘ └────────────┘ │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ ▼ │
|
||||
│ ┌─────────────────────────────────────────────────────────┐ │
|
||||
│ │ Frontend Layer │ │
|
||||
│ │ ┌─────────────────────┐ ┌─────────────────────┐ │ │
|
||||
│ │ │ Migration Dashboard │ │ Document Review │ │ │
|
||||
│ │ │ (Admin) │ │ Form (User) │ │ │
|
||||
│ │ └─────────────────────┘ └─────────────────────┘ │ │
|
||||
│ └─────────────────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Modules
|
||||
|
||||
### Backend Components
|
||||
|
||||
#### 1. AiModule & AiService
|
||||
|
||||
```typescript
|
||||
@Injectable()
|
||||
export class AiService {
|
||||
// Single entry point for all AI operations
|
||||
async extractMetadata(fileId: string): Promise<AiExtractionResult> {
|
||||
// 1. Send to n8n workflow
|
||||
// 2. Wait for OCR + LLM processing
|
||||
// 3. Validate results
|
||||
// 4. Return structured data
|
||||
}
|
||||
|
||||
async validateExtraction(result: AiExtractionResult): Promise<ValidationResult> {
|
||||
// Confidence scoring, enum validation, audit logging
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 2. Migration Entity
|
||||
|
||||
```sql
|
||||
CREATE TABLE migration_logs (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
publicId BINARY(16) DEFAULT (UUID_TO_BIN(UUID(), 1)),
|
||||
source_file VARCHAR(255) NOT NULL,
|
||||
source_metadata JSON, -- Data from Excel
|
||||
ai_extracted JSON, -- Data from AI
|
||||
confidence_score DECIMAL(3,2),
|
||||
status ENUM('PENDING_REVIEW', 'APPROVED', 'REJECTED'),
|
||||
reviewed_by INT,
|
||||
reviewed_at TIMESTAMP NULL,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
||||
);
|
||||
```
|
||||
|
||||
#### 3. API Endpoints
|
||||
|
||||
| Endpoint | Purpose | Access |
|
||||
|----------|---------|--------|
|
||||
| `POST /api/ai/extract` | Real-time extraction | Authenticated Users |
|
||||
| `POST /api/migration/batch` | Batch migration | Admin Only |
|
||||
| `GET /api/migration/queue` | Review queue | Admin Only |
|
||||
| `POST /api/migration/commit` | Commit approved items | Admin Only |
|
||||
|
||||
### Frontend Components
|
||||
|
||||
#### 1. DocumentReviewForm (Reusable Component)
|
||||
|
||||
```typescript
|
||||
interface DocumentReviewFormProps {
|
||||
// Source: Migration Table or AI API Response
|
||||
sourceData: MigrationItem | AiExtractionResult;
|
||||
// Mode: 'migration' | 'new'
|
||||
mode: 'migration' | 'new';
|
||||
onSubmit: (validatedData: ValidatedDocument) => void;
|
||||
}
|
||||
|
||||
// Features:
|
||||
// - Highlight AI-suggested fields
|
||||
// - Show confidence scores
|
||||
// - Allow human correction
|
||||
// - Track feedback for AI improvement
|
||||
```
|
||||
|
||||
#### 2. Migration Dashboard (Admin)
|
||||
|
||||
```typescript
|
||||
// Features:
|
||||
// - Filter by confidence level
|
||||
// - Bulk approve/reject
|
||||
// - Compare source vs AI data
|
||||
// - Export review reports
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflow Specifications
|
||||
|
||||
### Workflow 1: Legacy Migration (Batch Processing)
|
||||
|
||||
```
|
||||
Input: Excel Metadata + PDF Files
|
||||
│
|
||||
▼
|
||||
n8n Workflow:
|
||||
1. Read Excel row
|
||||
2. Send PDF to PaddleOCR
|
||||
3. Extract Thai/English text
|
||||
4. Send text + metadata to Gemma 4
|
||||
5. AI validates consistency
|
||||
6. Generate confidence score
|
||||
7. Store in migration_logs (PENDING_REVIEW)
|
||||
│
|
||||
▼
|
||||
Output: Migration Dashboard for Admin Review
|
||||
│
|
||||
▼
|
||||
Action: Admin approves → Commit to permanent storage
|
||||
```
|
||||
|
||||
### Workflow 2: New Ingestion (Real-time Processing)
|
||||
|
||||
```
|
||||
Input: User uploads PDF in RFA creation form
|
||||
│
|
||||
▼
|
||||
n8n Workflow (Real-time):
|
||||
1. OCR extraction (PaddleOCR)
|
||||
2. AI analysis (Gemma 4)
|
||||
3. Return suggestions to frontend
|
||||
│
|
||||
▼
|
||||
Output: Form auto-fill with AI suggestions
|
||||
│
|
||||
▼
|
||||
Action: User reviews/edits → Saves to database
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## AI Model Configuration
|
||||
|
||||
### Gemma 4 Prompt Strategy
|
||||
|
||||
```prompt
|
||||
You are an AI assistant for Laem Chabang Phase 3 construction project document analysis.
|
||||
|
||||
TASK: Extract and validate metadata from engineering documents.
|
||||
|
||||
RULES:
|
||||
1. Extract: Subject, Date, Discipline, Drawing Reference, Contract Number
|
||||
2. Validate: Check consistency between provided metadata and document content
|
||||
3. Confidence: Rate accuracy (0-100%) for each extracted field
|
||||
4. Language: Support Thai and English engineering terms
|
||||
5. Format: Return structured JSON
|
||||
|
||||
OUTPUT FORMAT:
|
||||
{
|
||||
"extracted_metadata": {
|
||||
"subject": "...",
|
||||
"date": "YYYY-MM-DD",
|
||||
"discipline": "Civil|Mechanical|Electrical|Architectural",
|
||||
"drawing_reference": "...",
|
||||
"contract_number": "..."
|
||||
},
|
||||
"validation": {
|
||||
"is_consistent": true|false,
|
||||
"discrepancies": ["..."],
|
||||
"confidence_score": 0.95
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Confidence Scoring Strategy
|
||||
|
||||
| Score Range | Action |
|
||||
|-------------|--------|
|
||||
| **95-100%** | Auto-approve (migration only) |
|
||||
| **85-94%** | Low priority review |
|
||||
| **60-84%** | High priority review |
|
||||
| **< 60%** | Reject / Requires manual entry |
|
||||
|
||||
---
|
||||
|
||||
## Security & Compliance
|
||||
|
||||
### AI Boundary Enforcement (ADR-018)
|
||||
|
||||
| Rule | Implementation |
|
||||
|------|----------------|
|
||||
| **Physical Isolation** | AI runs on Admin Desktop only |
|
||||
| **No Direct DB Access** | All communication via DMS API |
|
||||
| **API Authentication** | JWT tokens with `ai:invoke` scope |
|
||||
| **Audit Logging** | Every AI interaction logged |
|
||||
| **Human Validation** | No auto-commit without review |
|
||||
|
||||
### Data Privacy Measures
|
||||
|
||||
- **Local Processing Only:** No data leaves corporate network
|
||||
- **Temporary Storage:** AI processes data in memory only
|
||||
- **Encryption:** All API calls use TLS 1.3
|
||||
- **Data Retention:** AI logs retained for 90 days only
|
||||
|
||||
---
|
||||
|
||||
## Implementation Roadmap
|
||||
|
||||
### Phase 1: Pipeline Infrastructure (Task BE-AI-01)
|
||||
|
||||
**Week 1-2: AI Pipeline Foundation**
|
||||
1. **Docker Environment Setup** on Admin Desktop (Desk-5439)
|
||||
- n8n service with Basic Authentication
|
||||
- Ollama with Gemma 4 model (GPU optimized)
|
||||
- PaddleOCR service with Thai language support
|
||||
2. **n8n Workflow Development**
|
||||
- Webhook trigger for DMS integration
|
||||
- OCR → AI → JSON processing pipeline
|
||||
- Error handling and retry logic
|
||||
3. **Prompt Engineering**
|
||||
- Thai engineering document templates
|
||||
- JSON schema validation
|
||||
- Confidence scoring implementation
|
||||
4. **Integration Testing**
|
||||
- End-to-end pipeline validation
|
||||
- Security boundary verification
|
||||
- Performance benchmarking
|
||||
|
||||
### Phase 2: Backend AI Gateway (Task BE-AI-02)
|
||||
|
||||
**Week 3-4: NestJS Integration Layer**
|
||||
1. **Database Schema Design** (SQL First)
|
||||
- `migration_logs` table with UUIDv7 primary keys
|
||||
- `ai_audit_logs` for performance tracking
|
||||
- Data dictionary updates
|
||||
2. **AI Module Architecture**
|
||||
- `AiService` with n8n webhook integration
|
||||
- `MigrationService` for business logic
|
||||
- Validation layer with confidence thresholds
|
||||
3. **API Endpoints & Security**
|
||||
- Admin migration endpoints with CASL guards
|
||||
- Real-time extraction endpoint (`/api/ai/extract`)
|
||||
- Idempotency and rate limiting implementation
|
||||
4. **Configuration Management**
|
||||
- Service account authentication
|
||||
- Environment variables for AI endpoints
|
||||
- Monitoring and logging setup
|
||||
|
||||
### Phase 3: Frontend Human-in-the-Loop (Task FE-AI-03)
|
||||
|
||||
**Week 5-6: User Experience & Validation**
|
||||
1. **Reusable AI Components**
|
||||
- `AiSuggestionField` with confidence indicators
|
||||
- `DocumentComparisonView` with PDF sidebar
|
||||
- Client-side validation with Zod + React Hook Form
|
||||
2. **Admin Migration Dashboard**
|
||||
- Paginated table with filtering/sorting
|
||||
- Bulk actions for high-confidence items
|
||||
- Error logging and retry mechanisms
|
||||
3. **Real-time Ingestion Integration**
|
||||
- RFA creation flow enhancement
|
||||
- AI processing state indicators
|
||||
- Auto-fill with user override capability
|
||||
4. **Human-AI Feedback Loop**
|
||||
- User correction tracking
|
||||
- Performance analytics dashboard
|
||||
- Accuracy metrics and reporting
|
||||
|
||||
### Phase 4: Testing & Deployment
|
||||
|
||||
**Week 7-8: Production Readiness**
|
||||
1. **Comprehensive Testing**
|
||||
- Thai/English document validation
|
||||
- Confidence scoring accuracy verification
|
||||
- Load testing and performance optimization
|
||||
2. **Security Audit**
|
||||
- ADR-018 boundary verification
|
||||
- Penetration testing of AI endpoints
|
||||
- Data privacy compliance check
|
||||
3. **User Training & Documentation**
|
||||
- Admin workflow training
|
||||
- User guide for AI-assisted document creation
|
||||
- Troubleshooting and support procedures
|
||||
4. **Production Deployment**
|
||||
- Blue-green deployment strategy
|
||||
- Monitoring and alerting setup
|
||||
- Rollback procedures and contingency plans
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics
|
||||
|
||||
### Technical Performance Metrics
|
||||
|
||||
| Metric | Target | Measurement Method |
|
||||
|--------|--------|-------------------|
|
||||
| **Thai OCR Accuracy** | >90% | Character-by-character comparison with ground truth |
|
||||
| **AI JSON Validity** | 100% | Automated validation of all AI responses |
|
||||
| **Processing Time** | <15s/document | End-to-end timing from upload to suggestion |
|
||||
| **GPU Memory Usage** | <6GB per doc | Resource monitoring on Admin Desktop |
|
||||
| **System Uptime** | >99% | Service availability monitoring |
|
||||
|
||||
### Business Impact Metrics
|
||||
|
||||
| Metric | Target | Measurement Method |
|
||||
|--------|--------|-------------------|
|
||||
| **Data Entry Time Reduction** | 70% | Time comparison manual vs AI-assisted workflows |
|
||||
| **AI Accuracy Rate** | 85%+ | Human verification of AI extractions |
|
||||
| **Migration Throughput** | 1000 docs/day | Batch processing capacity with admin review |
|
||||
| **User Correction Rate** | <15% | Percentage of AI suggestions modified by users |
|
||||
| **Admin Productivity** | 3x improvement | Documents processed per admin hour |
|
||||
|
||||
### User Experience Metrics
|
||||
|
||||
| Metric | Target | Measurement Method |
|
||||
|--------|--------|-------------------|
|
||||
| **User Satisfaction** | 4.0/5.0 | Post-deployment user survey |
|
||||
| **Task Completion Rate** | >95% | Successful document creation rate |
|
||||
| **Learning Curve** | <30 min | Time to proficiency for new users |
|
||||
| **Error Rate** | <2% | Failed AI extractions requiring manual intervention |
|
||||
|
||||
### Security & Compliance Metrics
|
||||
|
||||
| Metric | Target | Measurement Method |
|
||||
|--------|--------|-------------------|
|
||||
| **Security Incidents** | 0 | Audit log monitoring and breach detection |
|
||||
| **Data Privacy Compliance** | 100% | Adherence to ADR-018 and PDPA requirements |
|
||||
| **Audit Trail Completeness** | 100% | All AI interactions logged and traceable |
|
||||
| **API Response Times** | <200ms | DMS API performance under load |
|
||||
|
||||
---
|
||||
|
||||
## Risk Assessment & Mitigation
|
||||
|
||||
### 🔴 High-Risk Items
|
||||
|
||||
| Risk | Impact | Probability | Mitigation Strategy |
|
||||
|------|--------|-------------|-------------------|
|
||||
| **AI Accuracy on Thai Documents** | High | Medium | Custom prompt engineering + Extensive testing with Thai engineering docs |
|
||||
| **Admin Desktop Hardware Failure** | High | Low | Backup desktop ready + Cloud AI fallback plan (emergency only) |
|
||||
| **Data Privacy Violations** | Critical | Low | Strict ADR-018 enforcement + Regular security audits |
|
||||
| **Performance Bottlenecks** | Medium | Medium | Queue system + GPU monitoring + Load balancing |
|
||||
|
||||
### 🟡 Medium-Risk Items
|
||||
|
||||
| Risk | Impact | Probability | Mitigation Strategy |
|
||||
|------|--------|-------------|-------------------|
|
||||
| **User Adoption Resistance** | Medium | Medium | Comprehensive training + UI/UX optimization + Early user involvement |
|
||||
| **Thai OCR Quality Issues** | Medium | Medium | Multiple OCR engines + Manual correction workflow |
|
||||
| **Integration Complexity** | Medium | Low | Phased deployment + Extensive testing + Rollback procedures |
|
||||
| **Cost Overruns** | Medium | Low | On-premise AI eliminates per-use costs | Resource monitoring |
|
||||
|
||||
### 🟢 Low-Risk Items
|
||||
|
||||
| Risk | Impact | Probability | Mitigation Strategy |
|
||||
|------|--------|-------------|-------------------|
|
||||
| **Technology Stack Changes** | Low | Low | Containerized deployment + Version pinning |
|
||||
| **Vendor Dependency** | Low | Low | Open-source stack + Multiple model options |
|
||||
| **Regulatory Changes** | Medium | Low | Flexible architecture + Compliance monitoring |
|
||||
|
||||
---
|
||||
|
||||
## Related Documents & Tasks
|
||||
|
||||
### Architecture Decision Records
|
||||
- **[ADR-017: Ollama Data Migration](./ADR-017-ollama-data-migration.md)** — Foundation migration architecture
|
||||
- **[ADR-017B: Smart Categorization](./ADR-017B-ollama.md)** — AI categorization use cases
|
||||
- **[ADR-018: AI Boundary Policy](./ADR-018-ai-boundary.md)** — Security isolation requirements (CRITICAL)
|
||||
- **[ADR-019: Hybrid Identifier Strategy](./ADR-019-hybrid-identifier-strategy.md)** — UUID usage patterns (CRITICAL)
|
||||
|
||||
### Implementation Tasks
|
||||
- **[Task BE-AI-01: Pipeline Infrastructure Setup](../08-Tasks/Task%20BE-AI-01.md)** — n8n + PaddleOCR + Gemma 4 setup
|
||||
- **[Task BE-AI-02: Backend AI Gateway Development](../08-Tasks/Task%20BE-AI-02.md)** — NestJS integration layer
|
||||
- **[Task FE-AI-03: Frontend Human-in-the-Loop Interface](../08-Tasks/Task-FE-AI-03.md)** — User experience and validation
|
||||
|
||||
### Technical Specifications
|
||||
- **[03-05-n8n-migration-setup-guide.md](../03-Data-and-Storage/03-05-n8n-migration-setup-guide.md)** — n8n configuration details
|
||||
- **[05-02-backend-guidelines.md](../05-Engineering-Guidelines/05-02-backend-guidelines.md)** — NestJS patterns and conventions
|
||||
- **[05-03-frontend-guidelines.md](../05-Engineering-Guidelines/05-03-frontend-guidelines.md)** — Next.js patterns and UI standards
|
||||
- **[03-01-data-dictionary.md](../03-Data-and-Storage/03-01-data-dictionary.md)** — Field definitions and business rules
|
||||
|
||||
### Compliance & Security
|
||||
- **[ADR-016: Security & Authentication](./ADR-016-security-authentication.md)** — Overall security framework
|
||||
- **[04-08-release-management-policy.md](../04-Infrastructure-OPS/04-08-release-management-policy.md)** — Deployment procedures
|
||||
|
||||
---
|
||||
|
||||
## Document History
|
||||
|
||||
| Version | Date | Author | Changes |
|
||||
|---------|------|--------|---------|
|
||||
| 1.8.5 | 2026-04-03 | AI Integration Lead | Initial ADR — AI Intelligence Integration Architecture |
|
||||
| 1.8.6 | 2026-04-03 | Tech Lead | Updated — Aligned with detailed task specifications and implementation requirements |
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** 2026-04-03
|
||||
**Status:** Proposed
|
||||
**Review Date:** 2026-04-10
|
||||
**Implementation Target:** v1.9.0
|
||||
@@ -75,6 +75,15 @@ Architecture Decision Records (ADRs) เป็นเอกสารที่บ
|
||||
| -------------------------------------------------- | -------------------------- | ----------- | ---------- | ---------------------------------------------------- |
|
||||
| [ADR-019](./ADR-019-hybrid-identifier-strategy.md) | Hybrid Identifier Strategy | ✅ Accepted | 2026-03-11 | INT PK (internal) + UUIDv7 (public API) on 14 tables |
|
||||
|
||||
### AI & Data Integration
|
||||
|
||||
| ADR | Title | Status | Date | Summary |
|
||||
| ----------------------------------------------- | ---------------------------------- | ------------- | ---------- | ---------------------------------------------------------------------------- |
|
||||
| [ADR-017](./ADR-017-ollama-data-migration.md) | Ollama Data Migration Architecture | ✅ Accepted | 2026-02-26 | On-premise AI (Ollama) สำหรับ Migration 20,000+ PDFs พร้อม Validation Layer |
|
||||
| [ADR-017B](./ADR-017B-ollama.md) | Smart Legacy Document Digitization | ✅ Accepted | 2026-03-27 | AI-powered categorization สำหรับเอกสารเก่า ตาม ADR-018 (AI Isolation) |
|
||||
| [ADR-018](./ADR-018-ai-boundary.md) | AI Boundary Policy | ✅ Accepted | 2026-03-27 | Physical Isolation + API-only communication (Zero Trust for AI) |
|
||||
| [ADR-020](./ADR-020-ai-intelligence-integration.md) | AI Intelligence Integration Architecture | 🔄 Proposed | 2026-04-03 | Unified AI Pipeline สำหรับ RFA-First (Legacy Migration + New Ingestion) |
|
||||
|
||||
---
|
||||
|
||||
## 🔍 ADR Categories
|
||||
@@ -120,6 +129,13 @@ Architecture Decision Records (ADRs) เป็นเอกสารที่บ
|
||||
|
||||
- **ADR-019:** Hybrid Identifier Strategy - INT PK (internal) + UUIDv7 (public API) บน 14 tables
|
||||
|
||||
### 9. AI & Data Integration
|
||||
|
||||
- **ADR-017:** Ollama Data Migration - On-premise AI (Ollama) สำหรับ Migration 20,000+ PDFs
|
||||
- **ADR-017B:** Smart Document Digitization - AI-powered categorization สำหรับเอกสารเก่า
|
||||
- **ADR-018:** AI Boundary Policy - Physical Isolation + API-only communication (Zero Trust)
|
||||
- **ADR-020:** AI Intelligence Integration - Unified AI Pipeline สำหรับ RFA-First approach
|
||||
|
||||
---
|
||||
|
||||
## 📖 How to Read ADRs
|
||||
|
||||
@@ -0,0 +1,247 @@
|
||||
# Task BE-AI-01: Pipeline Infrastructure Setup
|
||||
|
||||
**Phase:** Step 1 - AI Pipeline Foundation (n8n + PaddleOCR + Gemma 4)
|
||||
**ADR Compliance:** ADR-018 (AI Boundary), ADR-019 (UUID Strategy)
|
||||
**Priority:** 🔴 Critical - Foundation for all AI features
|
||||
|
||||
> **Context:** เป็นรากฐานสำคัญของระบบ Document Intelligence ตาม ADR-020 โดยต้องเป็นไปตามนโยบาย AI Isolation และใช้ Identifier ที่ถูกต้อง
|
||||
|
||||
---
|
||||
|
||||
## 📋 Implementation Tasks
|
||||
|
||||
### **AI-1.1: Infrastructure Setup (ADR-018 Compliance)**
|
||||
- [ ] **Docker Environment on Admin Desktop (Desk-5439):**
|
||||
- ติดตั้ง Docker Compose สำหรับ n8n และ PaddleOCR
|
||||
- ตั้งค่า Network Isolation (LAN only, no public access)
|
||||
- ตรวจสอบ Hardware: RTX 2060 Super 8GB VRAM availability
|
||||
- [ ] **n8n Service:**
|
||||
- Docker Compose service พร้อม Basic Authentication
|
||||
- Webhook endpoint: `/webhook/ai-processing`
|
||||
- Environment variables: `N8N_BASIC_AUTH_USER`, `N8N_BASIC_AUTH_PASSWORD`
|
||||
- [ ] **Ollama Service:**
|
||||
- Pull model: `gemma:9b` (GPU optimized, higher accuracy)
|
||||
- API endpoint: `http://localhost:11434`
|
||||
- Health check: `GET /api/tags`
|
||||
- Memory requirement: Minimum 8GB VRAM for 9B model
|
||||
- ollama run gemma4:9b-q5_K_M / gemma4:9b-q4_K_M
|
||||
- สร้างไฟล์ %USERPROFILE%\.ollama\config
|
||||
```config
|
||||
# ใช้ GPU เป็นหลัก
|
||||
gpu: true
|
||||
num_gpu: 1
|
||||
|
||||
# เปิด KV cache เพื่อให้ตอบเร็วขึ้น
|
||||
kv_cache: true
|
||||
|
||||
# จำกัด batch size ให้เหมาะกับ VRAM 8GB
|
||||
gpu_batch_size: 512
|
||||
|
||||
# ปรับ num_thread ให้เหมาะกับ CPU 6–8 คอร์
|
||||
num_thread: 6
|
||||
|
||||
# เปิด mmap เพื่อโหลดโมเดลเร็วขึ้น
|
||||
mmap: true
|
||||
|
||||
# ปรับ max_seq_len ให้เหมาะกับงาน DMS
|
||||
max_seq_len: 4096
|
||||
|
||||
# ปรับ temp ต่ำเพื่อให้ผลลัพธ์เสถียร
|
||||
temperature: 0.2
|
||||
```
|
||||
|
||||
- [ ] **PaddleOCR Service:**
|
||||
- Docker image: `paddlepaddle/paddle:latest-gpu`
|
||||
- Thai language support configuration
|
||||
- API endpoint design: `POST /ocr/extract`
|
||||
|
||||
### **AI-1.2: n8n Workflow Development**
|
||||
- [ ] **Webhook Trigger Node:**
|
||||
- Input: `{ publicId: string, fileUrl: string, context: 'migration'|'ingestion' }`
|
||||
- Validation: Verify `publicId` format (UUIDv7) before processing
|
||||
- Idempotency check: Prevent duplicate processing
|
||||
- [ ] **OCR Integration Node:**
|
||||
- HTTP Request to PaddleOCR service
|
||||
- Input: Binary file data
|
||||
- Output: `{ text: string, confidence: number, language: 'th'|'en' }`
|
||||
- Error handling: Retry logic + fallback to CPU OCR
|
||||
- [ ] **Prompt Engineering Node:**
|
||||
- Function Node to construct Gemma 4 prompt
|
||||
- Template includes: Role definition, validation rules, JSON schema
|
||||
- Thai engineering context keywords
|
||||
- [ ] **Gemma 4 LLM Node:**
|
||||
- HTTP Request to Ollama API
|
||||
- Model: `gemma:9b` (enhanced accuracy for Thai engineering documents)
|
||||
- Parameters: `temperature: 0.1`, `max_tokens: 2048`
|
||||
- Output validation: Ensure valid JSON response
|
||||
- Memory monitoring: Track VRAM usage during inference
|
||||
- [ ] **Result Processing Node:**
|
||||
- Parse and validate AI response
|
||||
- Calculate confidence scores
|
||||
- Format for DMS Backend API callback
|
||||
- [ ] **Callback to DMS:**
|
||||
- HTTP POST to NestJS webhook endpoint
|
||||
- Payload: `{ publicId, extractedData, confidence, processingTime }`
|
||||
- Authentication: Service account JWT
|
||||
|
||||
### **AI-1.3: Prompt Engineering for Thai Engineering Documents**
|
||||
- [ ] **System Prompt Template:**
|
||||
```prompt
|
||||
You are a Senior Document Controller for Laem Chabang Port Phase 3 construction project.
|
||||
|
||||
TASK: Extract metadata from engineering documents with high accuracy.
|
||||
|
||||
RULES:
|
||||
1. Extract: subject, document_date, discipline, drawing_reference, contract_number
|
||||
2. Validate consistency between content and metadata
|
||||
3. Return confidence score (0-100%) for each field
|
||||
4. Support Thai and English engineering terms
|
||||
5. Output MUST be valid JSON only
|
||||
|
||||
OUTPUT FORMAT:
|
||||
{
|
||||
"subject": "string",
|
||||
"document_date": "YYYY-MM-DD",
|
||||
"discipline": "Civil|Mechanical|Electrical|Architectural",
|
||||
"drawing_reference": "string",
|
||||
"contract_number": "string",
|
||||
"confidence": {
|
||||
"overall": 0.95,
|
||||
"field_confidence": {...}
|
||||
}
|
||||
}
|
||||
```
|
||||
- [ ] **Thai Language Optimization:**
|
||||
- Engineering terms: "วิศวกรรมโยธา", "แบบรายละเอียด", "ขออนุมัติ", "แผนผัง"
|
||||
- Date format recognition: Thai Buddhist years (พ.ศ.)
|
||||
- Organization names: "ท่าเรือแหลมฉบัง", "ก.ท.ม.", "การท่าเรือฯ"
|
||||
- [ ] **JSON Schema Validation:**
|
||||
- Zod schema for response validation
|
||||
- Required fields enforcement
|
||||
- Type checking and sanitization
|
||||
|
||||
### **AI-1.4: Integration Testing & Validation**
|
||||
- [ ] **Test Case 1: Legacy Migration Flow:**
|
||||
- Input: Scanned RFA PDF + Excel metadata
|
||||
- Expected: Thai text extraction >90% accuracy
|
||||
- Validation: AI output matches Excel data
|
||||
- [ ] **Test Case 2: Real-time Ingestion Flow:**
|
||||
- Input: New PDF upload from user
|
||||
- Expected: Response time <15 seconds
|
||||
- Validation: Structured JSON response
|
||||
- [ ] **Performance Benchmarking:**
|
||||
- Target: <15 seconds per document
|
||||
- Memory usage monitoring on Admin Desktop
|
||||
- GPU utilization tracking
|
||||
- [ ] **Security Validation:**
|
||||
- Verify no external network calls
|
||||
- Confirm AI services run in isolation
|
||||
- Test authentication between n8n and DMS
|
||||
|
||||
---
|
||||
|
||||
## ✅ Acceptance Criteria
|
||||
|
||||
1. **Pipeline Functionality:**
|
||||
- n8n successfully processes PDF → OCR → AI → JSON flow
|
||||
- Thai text extraction accuracy >90%
|
||||
- Gemma 4 returns valid JSON 100% of time
|
||||
|
||||
2. **Security Compliance (ADR-018):**
|
||||
- All services run on Admin Desktop only
|
||||
- No external network connections
|
||||
- Proper authentication between services
|
||||
|
||||
3. **Data Integrity:**
|
||||
- Extracted metadata matches document content >85%
|
||||
- Confidence scoring implemented and accurate
|
||||
- Idempotency prevents duplicate processing
|
||||
|
||||
4. **Performance:**
|
||||
- Processing time <20 seconds per document (gemma:9b)
|
||||
- GPU memory usage <8GB per document
|
||||
- System remains stable under load
|
||||
|
||||
---
|
||||
|
||||
## � Critical Rules (Non-Negotiable)
|
||||
|
||||
1. **ADR-018 Compliance:** AI services MUST run on Admin Desktop ONLY
|
||||
2. **No Direct DB Access:** Pipeline communicates via DMS API only
|
||||
3. **UUID Strategy:** All document references use `publicId` (UUIDv7)
|
||||
4. **Thai Language Support:** Must handle Thai engineering documents
|
||||
5. **Error Handling:** All failures must log to DMS audit system
|
||||
|
||||
---
|
||||
|
||||
## 📁 Related Specifications
|
||||
|
||||
- **ADR-018:** AI Boundary Policy - Physical isolation requirements
|
||||
- **ADR-019:** Hybrid Identifier Strategy - UUID usage patterns
|
||||
- **ADR-020:** AI Intelligence Integration - Overall architecture
|
||||
- **03-05-n8n-migration-setup-guide.md:** n8n configuration details
|
||||
|
||||
---
|
||||
|
||||
## 📝 Implementation Notes
|
||||
|
||||
### Docker Compose Structure
|
||||
```yaml
|
||||
services:
|
||||
n8n:
|
||||
image: n8nio/n8n:latest
|
||||
ports: ["5678:5678"]
|
||||
environment:
|
||||
- N8N_BASIC_AUTH_ACTIVE=true
|
||||
- N8N_BASIC_AUTH_USER=admin
|
||||
- N8N_BASIC_AUTH_PASSWORD=${N8N_PASSWORD}
|
||||
|
||||
paddleocr:
|
||||
image: paddlepaddle/paddle:latest-gpu
|
||||
ports: ["8866:8866"]
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: 1
|
||||
capabilities: [gpu]
|
||||
device_ids: ["0"] # RTX 2060 Super
|
||||
environment:
|
||||
- CUDA_VISIBLE_DEVICES=0
|
||||
shm_size: 2gb
|
||||
|
||||
ollama:
|
||||
image: ollama/ollama:latest
|
||||
ports: ["11434:11434"]
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: 1
|
||||
capabilities: [gpu]
|
||||
device_ids: ["0"] # RTX 2060 Super
|
||||
environment:
|
||||
- CUDA_VISIBLE_DEVICES=0
|
||||
- OLLAMA_MAX_LOADED_MODELS=1
|
||||
- OLLAMA_NUM_PARALLEL=2
|
||||
volumes:
|
||||
- ollama_data:/root/.ollama
|
||||
pull_policy: always
|
||||
|
||||
volumes:
|
||||
ollama_data:
|
||||
```
|
||||
|
||||
### Hardware Requirements
|
||||
- **GPU:** RTX 2060 Super 8GB VRAM (minimum for gemma:9b)
|
||||
- **RAM:** 32GB system memory recommended
|
||||
- **Storage:** 100GB SSD for models and temporary files
|
||||
- **Network:** Gigabit LAN for file transfers
|
||||
|
||||
### Model Specifications
|
||||
- **gemma:9b** - 9 billion parameters, optimized for Thai
|
||||
- **VRAM Usage:** ~7-8GB for inference
|
||||
- **Performance:** ~15-20 seconds per document
|
||||
- **Accuracy:** Expected 90%+ for Thai engineering documents
|
||||
@@ -0,0 +1,295 @@
|
||||
# Task BE-AI-02: Backend AI Gateway Development
|
||||
|
||||
**Phase:** Step 2 - AI Integration Layer (NestJS)
|
||||
**ADR Compliance:** ADR-018 (AI Boundary), ADR-019 (UUID Strategy)
|
||||
**Priority:** 🔴 Critical - Bridge between DMS and AI Pipeline
|
||||
|
||||
> **Context:** เป็นส่วนเชื่อมโยงระหว่างระบบ DMS และ AI Pipeline ตาม ADR-020 โดยต้องรักษาความปลอดภัยและใช้ Identifier ที่ถูกต้อง
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Implementation Tasks
|
||||
|
||||
### **AI-2.1: Database Schema Design (SQL First Approach)**
|
||||
- [ ] **Create `migration_logs` Table:**
|
||||
```sql
|
||||
CREATE TABLE migration_logs (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
publicId BINARY(16) DEFAULT (UUID_TO_BIN(UUID(), 1)),
|
||||
source_file VARCHAR(255) NOT NULL,
|
||||
source_metadata JSON,
|
||||
ai_extracted_metadata JSON,
|
||||
confidence_score DECIMAL(3,2),
|
||||
status ENUM('PENDING_REVIEW', 'VERIFIED', 'IMPORTED', 'FAILED') DEFAULT 'PENDING_REVIEW',
|
||||
admin_feedback TEXT,
|
||||
reviewed_by INT NULL,
|
||||
reviewed_at TIMESTAMP NULL,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
||||
INDEX idx_status (status),
|
||||
INDEX idx_confidence (confidence_score),
|
||||
INDEX idx_publicId (publicId)
|
||||
);
|
||||
```
|
||||
- [ ] **Create `ai_audit_logs` Table:**
|
||||
```sql
|
||||
CREATE TABLE ai_audit_logs (
|
||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||
publicId BINARY(16) DEFAULT (UUID_TO_BIN(UUID(), 1)),
|
||||
document_publicId BINARY(16),
|
||||
ai_model VARCHAR(50) NOT NULL,
|
||||
processing_time_ms INT,
|
||||
confidence_score DECIMAL(3,2),
|
||||
input_hash VARCHAR(64),
|
||||
output_hash VARCHAR(64),
|
||||
status ENUM('SUCCESS', 'FAILED', 'TIMEOUT') NOT NULL,
|
||||
error_message TEXT,
|
||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||
INDEX idx_document (document_publicId),
|
||||
INDEX idx_model (ai_model),
|
||||
INDEX idx_status (status),
|
||||
FOREIGN KEY (document_publicId) REFERENCES migration_logs(publicId)
|
||||
);
|
||||
```
|
||||
- [ ] **Update Data Dictionary:**
|
||||
- Add field descriptions to `specs/03-Data-and-Storage/03-01-data-dictionary.md`
|
||||
- Include business rules for confidence thresholds
|
||||
- Document status transitions and workflows
|
||||
|
||||
### **AI-2.2: AI Gateway Module Architecture**
|
||||
- [ ] **Module Structure:**
|
||||
```typescript
|
||||
// src/modules/ai/ai.module.ts
|
||||
@Module({
|
||||
imports: [TypeOrmModule.forFeature([MigrationLog, AiAuditLog])],
|
||||
controllers: [AiController],
|
||||
providers: [AiService, AiValidationService],
|
||||
exports: [AiService],
|
||||
})
|
||||
export class AiModule {}
|
||||
```
|
||||
- [ ] **AiService Implementation:**
|
||||
```typescript
|
||||
@Injectable()
|
||||
export class AiService {
|
||||
async triggerProcessing(filePublicId: string, context: ProcessingContext): Promise<void> {
|
||||
// 1. Validate publicId format (ADR-019)
|
||||
// 2. Send HTTP request to n8n webhook
|
||||
// 3. Log request to ai_audit_logs
|
||||
// 4. Return processing token
|
||||
}
|
||||
|
||||
async handleWebhookCallback(payload: AiCallbackDto): Promise<void> {
|
||||
// 1. Validate JWT token from n8n
|
||||
// 2. Update migration_logs with AI results
|
||||
// 3. Calculate confidence scores
|
||||
// 4. Trigger notifications if needed
|
||||
}
|
||||
|
||||
async extractRealtime(filePublicId: string): Promise<ExtractionResult> {
|
||||
// 1. Send to n8n for immediate processing
|
||||
// 2. Wait for response (timeout: 30s)
|
||||
// 3. Return structured suggestions
|
||||
}
|
||||
}
|
||||
```
|
||||
- [ ] **Configuration Management:**
|
||||
```env
|
||||
# .env
|
||||
AI_N8N_WEBHOOK_URL=http://192.168.1.100:5678/webhook/ai-processing
|
||||
AI_N8N_AUTH_TOKEN=service-account-jwt-token
|
||||
AI_OLLAMA_URL=http://192.168.1.100:11434
|
||||
AI_TIMEOUT_MS=30000
|
||||
AI_MAX_RETRIES=3
|
||||
```
|
||||
|
||||
### **AI-2.3: Migration Engine & Business Logic**
|
||||
- [ ] **MigrationService Implementation:**
|
||||
```typescript
|
||||
@Injectable()
|
||||
export class MigrationService {
|
||||
async stageLegacyData(excelData: ExcelImportDto[]): Promise<MigrationLog[]> {
|
||||
// 1. Validate Excel data format
|
||||
// 2. Move PDF files to staging area (via StorageService)
|
||||
// 3. Create migration_logs entries
|
||||
// 4. Trigger AI processing for each file
|
||||
}
|
||||
|
||||
async compareData(excelMetadata: any, aiMetadata: any): Promise<ComparisonResult> {
|
||||
// 1. Field-by-field comparison
|
||||
// 2. Calculate confidence deltas
|
||||
// 3. Flag discrepancies for human review
|
||||
// 4. Generate comparison report
|
||||
}
|
||||
|
||||
async approveMigration(migrationPublicId: string, adminId: number): Promise<void> {
|
||||
// 1. Validate admin permissions (CASL)
|
||||
// 2. Move file from staging to permanent storage
|
||||
// 3. Create actual document records (RFA, Correspondence, etc.)
|
||||
// 4. Update migration_logs status
|
||||
}
|
||||
}
|
||||
```
|
||||
- [ ] **Status Management Workflow:**
|
||||
```typescript
|
||||
enum MigrationStatus {
|
||||
PENDING_REVIEW = 'PENDING_REVIEW',
|
||||
VERIFIED = 'VERIFIED',
|
||||
IMPORTED = 'IMPORTED',
|
||||
FAILED = 'FAILED'
|
||||
}
|
||||
|
||||
// State transition rules
|
||||
const statusTransitions = {
|
||||
[MigrationStatus.PENDING_REVIEW]: [MigrationStatus.VERIFIED, MigrationStatus.FAILED],
|
||||
[MigrationStatus.VERIFIED]: [MigrationStatus.IMPORTED, MigrationStatus.PENDING_REVIEW],
|
||||
[MigrationStatus.IMPORTED]: [], // Terminal state
|
||||
[MigrationStatus.FAILED]: [MigrationStatus.PENDING_REVIEW] // Can retry
|
||||
};
|
||||
```
|
||||
|
||||
### **AI-2.4: API Endpoints & Security Implementation**
|
||||
- [ ] **Admin Migration Endpoints:**
|
||||
```typescript
|
||||
@Controller('admin/migration')
|
||||
@UseGuards(JwtAuthGuard, CaslGuard)
|
||||
export class AdminMigrationController {
|
||||
@Get()
|
||||
@Permissions(PERMISSIONS.MIGRATION_READ)
|
||||
async getMigrationList(@Query() query: MigrationQueryDto): Promise<PaginatedResult<MigrationLog>> {
|
||||
// 1. Validate query parameters
|
||||
// 2. Apply filters (status, confidence, date range)
|
||||
// 3. Return paginated results
|
||||
}
|
||||
|
||||
@Patch(':publicId')
|
||||
@Permissions(PERMISSIONS.MIGRATION_APPROVE)
|
||||
async updateMigration(
|
||||
@Param('publicId') publicId: string,
|
||||
@Body() updateDto: MigrationUpdateDto,
|
||||
@CurrentUser() user: User
|
||||
): Promise<MigrationLog> {
|
||||
// 1. Validate publicId (no parseInt!)
|
||||
// 2. Check admin permissions
|
||||
// 3. Update with audit trail
|
||||
}
|
||||
}
|
||||
```
|
||||
- [ ] **Real-time AI Extraction Endpoint:**
|
||||
```typescript
|
||||
@Controller('ai')
|
||||
export class AiController {
|
||||
@Post('extract')
|
||||
@UseGuards(JwtAuthGuard)
|
||||
@Throttle(5, 60) // 5 requests per minute
|
||||
async extractDocument(@Body() dto: ExtractDocumentDto): Promise<ExtractionResult> {
|
||||
// 1. Validate file access permissions
|
||||
// 2. Send to AI pipeline
|
||||
// 3. Return structured suggestions
|
||||
}
|
||||
}
|
||||
```
|
||||
- [ ] **Security Measures:**
|
||||
- CASL permissions for all endpoints
|
||||
- Idempotency-Key header validation
|
||||
- Rate limiting on AI endpoints
|
||||
- JWT authentication for service accounts
|
||||
- Request/response logging for audit
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Critical Rules (Non-Negotiable)
|
||||
|
||||
1. **ADR-019 UUID Strategy:**
|
||||
- Use `publicId` (UUIDv7) for all document references
|
||||
- NEVER use `parseInt()` or `Number()` on UUID values
|
||||
- All API parameters use string type for UUIDs
|
||||
|
||||
2. **ADR-018 AI Boundary:**
|
||||
- No direct database access from AI services
|
||||
- All communication via DMS API only
|
||||
- AI services run on Admin Desktop (isolated)
|
||||
|
||||
3. **Security Requirements:**
|
||||
- All `POST/PATCH` endpoints must validate `Idempotency-Key`
|
||||
- CASL permissions enforced on all endpoints
|
||||
- Rate limiting on AI endpoints (5 req/min)
|
||||
|
||||
4. **Data Integrity:**
|
||||
- SQL-first approach (no TypeORM migrations)
|
||||
- All file operations via StorageService
|
||||
- Audit logging for all AI interactions
|
||||
|
||||
---
|
||||
|
||||
## 📋 Implementation Sequence
|
||||
|
||||
1. **Phase 1 (AI-2.1):** Database schema and data dictionary updates
|
||||
2. **Phase 2 (AI-2.2):** AI Gateway module and basic service structure
|
||||
3. **Phase 3 (AI-2.3 & AI-2.4):** Business logic and API endpoints (parallel development)
|
||||
4. **Phase 4:** Integration testing with n8n pipeline
|
||||
|
||||
---
|
||||
|
||||
## 📁 Related Specifications
|
||||
|
||||
- **ADR-018:** AI Boundary Policy - Security requirements
|
||||
- **ADR-019:** Hybrid Identifier Strategy - UUID patterns
|
||||
- **ADR-020:** AI Intelligence Integration - Architecture overview
|
||||
- **05-02-backend-guidelines.md:** NestJS patterns and conventions
|
||||
- **03-01-data-dictionary.md:** Field definitions and business rules
|
||||
|
||||
---
|
||||
|
||||
## 📝 Code Templates
|
||||
|
||||
### DTO Examples
|
||||
```typescript
|
||||
// extract-document.dto.ts
|
||||
export class ExtractDocumentDto {
|
||||
@IsUUID()
|
||||
publicId: string;
|
||||
|
||||
@IsEnum(['migration', 'ingestion'])
|
||||
context: string;
|
||||
}
|
||||
|
||||
// migration-update.dto.ts
|
||||
export class MigrationUpdateDto {
|
||||
@IsOptional()
|
||||
@IsEnum(['VERIFIED', 'FAILED'])
|
||||
status?: MigrationStatus;
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
@MaxLength(1000)
|
||||
adminFeedback?: string;
|
||||
}
|
||||
```
|
||||
|
||||
### Entity Example
|
||||
```typescript
|
||||
// migration-log.entity.ts
|
||||
@Entity('migration_logs')
|
||||
export class MigrationLog extends UuidBaseEntity {
|
||||
@Column({ type: 'varchar', length: 255 })
|
||||
sourceFile: string;
|
||||
|
||||
@Column({ type: 'json' })
|
||||
sourceMetadata: any;
|
||||
|
||||
@Column({ type: 'json' })
|
||||
aiExtractedMetadata: any;
|
||||
|
||||
@Column({ type: 'decimal', precision: 3, scale: 2 })
|
||||
confidenceScore: number;
|
||||
|
||||
@Column({
|
||||
type: 'enum',
|
||||
enum: MigrationStatus,
|
||||
default: MigrationStatus.PENDING_REVIEW
|
||||
})
|
||||
status: MigrationStatus;
|
||||
}
|
||||
```
|
||||
|
||||
@@ -0,0 +1,442 @@
|
||||
# Task FE-AI-03: Frontend Human-in-the-Loop Interface
|
||||
|
||||
**Phase:** Step 3 - AI Verification & User Experience (Next.js)
|
||||
**ADR Compliance:** ADR-018 (AI Boundary), ADR-019 (UUID Strategy)
|
||||
**Priority:** 🔴 Critical - Human validation layer for AI outputs
|
||||
|
||||
> **Context:** เป็นส่วนสำคัญที่สุดในการเปลี่ยนข้อมูลที่ AI สกัดได้ให้เป็นข้อมูลที่มีคุณภาพ (Verified Data) ตามกฎ ADR-018 โดยเน้นการสร้าง UI ที่ใช้งานง่ายสำหรับทั้ง Admin (เอกสารเก่า) และ User (เอกสารใหม่)
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ Implementation Tasks
|
||||
|
||||
### **AI-3.1: Reusable AI Review Components**
|
||||
- [ ] **AiSuggestionField Component:**
|
||||
```typescript
|
||||
// components/ai/ai-suggestion-field.tsx
|
||||
interface AiSuggestionFieldProps {
|
||||
value: string;
|
||||
suggestion?: string;
|
||||
confidence?: number;
|
||||
onAccept: () => void;
|
||||
onReject: () => void;
|
||||
onEdit: (newValue: string) => void;
|
||||
}
|
||||
```
|
||||
Features:
|
||||
- AI icon with confidence badge (✨ 95%)
|
||||
- Yellow highlight for AI-suggested values
|
||||
- Accept/Reject/Edit actions
|
||||
- Tooltip showing raw AI extraction
|
||||
|
||||
- [ ] **DocumentComparisonView Component:**
|
||||
```typescript
|
||||
// components/ai/document-comparison-view.tsx
|
||||
interface DocumentComparisonViewProps {
|
||||
fileUrl: string;
|
||||
extractedData: ExtractionResult;
|
||||
formData: FormData;
|
||||
onFieldUpdate: (field: string, value: string) => void;
|
||||
}
|
||||
```
|
||||
Features:
|
||||
- PDF viewer sidebar (react-pdf)
|
||||
- Form fields with AI suggestions
|
||||
- Side-by-side comparison layout
|
||||
- Real-time validation feedback
|
||||
|
||||
- [ ] **Client-side Validation Integration:**
|
||||
```typescript
|
||||
// Validation schema with confidence thresholds
|
||||
const documentSchema = z.object({
|
||||
subject: z.string().min(1, "จำเป็นต้องระบุชื่อเรื่อง"),
|
||||
documentDate: z.string().refine(validateThaiDate),
|
||||
discipline: z.enum(['Civil', 'Mechanical', 'Electrical', 'Architectural'])
|
||||
});
|
||||
|
||||
// React Hook Form integration
|
||||
const form = useForm({
|
||||
resolver: zodResolver(documentSchema),
|
||||
mode: 'onChange',
|
||||
defaultValues: aiSuggestions
|
||||
});
|
||||
```
|
||||
|
||||
### **AI-3.2: Legacy Migration Dashboard (Admin Interface)**
|
||||
- [ ] **Migration List Page:**
|
||||
```typescript
|
||||
// app/(admin)/admin/migration/page.tsx
|
||||
interface MigrationListProps {
|
||||
status?: MigrationStatus;
|
||||
confidenceRange?: [number, number];
|
||||
dateRange?: [Date, Date];
|
||||
}
|
||||
```
|
||||
Features:
|
||||
- Paginated table with sorting/filtering
|
||||
- Status badges (Pending/Verified/Failed)
|
||||
- Confidence score heat map (red/yellow/green)
|
||||
- Bulk selection for actions
|
||||
|
||||
- [ ] **Filter System:**
|
||||
```typescript
|
||||
// Filter components
|
||||
const StatusFilter = () => (
|
||||
<Select value={selectedStatus} onValueChange={setSelectedStatus}>
|
||||
<SelectItem value="PENDING_REVIEW">รอตรวจสอบ</SelectItem>
|
||||
<SelectItem value="VERIFIED">ผ่านการตรวจสอบ</SelectItem>
|
||||
<SelectItem value="FAILED">ล้มเหลว</SelectItem>
|
||||
</Select>
|
||||
);
|
||||
|
||||
const ConfidenceFilter = () => (
|
||||
<Slider
|
||||
min={0}
|
||||
max={100}
|
||||
value={confidenceRange}
|
||||
onValueChange={setConfidenceRange}
|
||||
marks={[{value: 60, label: 'ต่ำ'}, {value: 85, label: 'ปานกลาง'}, {value: 95, label: 'สูง'}]}
|
||||
/>
|
||||
);
|
||||
```
|
||||
|
||||
- [ ] **Bulk Actions Implementation:**
|
||||
```typescript
|
||||
// Bulk verification for high-confidence items
|
||||
const handleBulkVerify = async (selectedIds: string[]) => {
|
||||
const confirmed = await confirm({
|
||||
title: "ยืนยันการนำเข้าข้อมูล",
|
||||
description: `จะยืนยันนำเข้าเอกสาร ${selectedIds.length} รายการที่มีความมั่นใจ >95% หรือไม่?`
|
||||
});
|
||||
|
||||
if (confirmed) {
|
||||
await Promise.all(
|
||||
selectedIds.map(publicId =>
|
||||
api.migration.update(publicId, { status: 'VERIFIED' })
|
||||
)
|
||||
);
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Error Logging UI:**
|
||||
- Error details modal for failed extractions
|
||||
- OCR error screenshots
|
||||
- AI response raw text viewer
|
||||
- Retry mechanism with different parameters
|
||||
|
||||
### **AI-3.3: Real-time Ingestion Integration (User Interface)**
|
||||
- [ ] **RFA Creation Flow Enhancement:**
|
||||
```typescript
|
||||
// app/(dashboard)/rfas/create/page.tsx
|
||||
const [isProcessing, setIsProcessing] = useState(false);
|
||||
const [aiSuggestions, setAiSuggestions] = useState<ExtractionResult | null>(null);
|
||||
|
||||
const handleFileUpload = async (file: File) => {
|
||||
setIsProcessing(true);
|
||||
try {
|
||||
// 1. Upload file to temporary storage
|
||||
const uploadResult = await api.storage.uploadTemp(file);
|
||||
|
||||
// 2. Trigger AI extraction
|
||||
const extraction = await api.ai.extract({
|
||||
publicId: uploadResult.publicId,
|
||||
context: 'ingestion'
|
||||
});
|
||||
|
||||
// 3. Apply suggestions to form
|
||||
setAiSuggestions(extraction);
|
||||
form.reset(extraction.suggestions);
|
||||
} finally {
|
||||
setIsProcessing(false);
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Processing State UI:**
|
||||
```typescript
|
||||
// Loading component during AI processing
|
||||
const AiProcessingIndicator = () => (
|
||||
<Card className="border-yellow-200 bg-yellow-50">
|
||||
<CardContent className="flex items-center space-x-3 p-4">
|
||||
<Loader2 className="h-5 w-5 animate-spin text-yellow-600" />
|
||||
<div>
|
||||
<p className="font-medium text-yellow-800">AI กำลังวิเคราะห์เอกสาร...</p>
|
||||
<p className="text-sm text-yellow-600">กรุณารอสักครู่ (ประมาณ 15-30 วินาที)</p>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
);
|
||||
```
|
||||
|
||||
- [ ] **Auto-fill with User Override:**
|
||||
```typescript
|
||||
// Form field with AI suggestion
|
||||
const FormFieldWithAi = ({ name, label }: { name: string; label: string }) => {
|
||||
const { control, watch } = useFormContext();
|
||||
const value = watch(name);
|
||||
const suggestion = aiSuggestions?.suggestions[name];
|
||||
const confidence = aiSuggestions?.confidence[name];
|
||||
|
||||
return (
|
||||
<FormField
|
||||
control={control}
|
||||
name={name}
|
||||
render={({ field }) => (
|
||||
<FormItem>
|
||||
<FormLabel className="flex items-center gap-2">
|
||||
{label}
|
||||
{suggestion && confidence && (
|
||||
<Badge variant="secondary" className="text-xs">
|
||||
✨ AI {Math.round(confidence * 100)}%
|
||||
</Badge>
|
||||
)}
|
||||
</FormLabel>
|
||||
<FormControl>
|
||||
<Input
|
||||
{...field}
|
||||
className={suggestion && value === suggestion ? 'bg-yellow-50' : ''}
|
||||
/>
|
||||
</FormControl>
|
||||
</FormItem>
|
||||
)}
|
||||
/>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Raw Text Comparison Toggle:**
|
||||
```typescript
|
||||
// Collapsible panel showing OCR text
|
||||
const OcrTextViewer = ({ extractedText }: { extractedText: string }) => (
|
||||
<Collapsible>
|
||||
<CollapsibleTrigger asChild>
|
||||
<Button variant="ghost" size="sm" className="text-blue-600">
|
||||
<Eye className="h-4 w-4 mr-2" />
|
||||
ดูข้อความดิบจาก AI
|
||||
</Button>
|
||||
</CollapsibleTrigger>
|
||||
<CollapsibleContent>
|
||||
<Card className="mt-2">
|
||||
<CardContent className="p-4">
|
||||
<pre className="text-sm bg-gray-50 p-3 rounded overflow-auto max-h-48">
|
||||
{extractedText}
|
||||
</pre>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</CollapsibleContent>
|
||||
</Collapsible>
|
||||
);
|
||||
```
|
||||
|
||||
### **AI-3.4: Human-AI Feedback Loop Implementation**
|
||||
- [ ] **Feedback Collection System:**
|
||||
```typescript
|
||||
// Track user corrections for AI improvement
|
||||
const trackUserCorrection = async (
|
||||
field: string,
|
||||
aiSuggestion: string,
|
||||
userCorrection: string,
|
||||
documentPublicId: string
|
||||
) => {
|
||||
await api.ai.feedback.create({
|
||||
documentPublicId,
|
||||
field,
|
||||
aiSuggestion,
|
||||
userCorrection,
|
||||
timestamp: new Date().toISOString(),
|
||||
userAgent: navigator.userAgent
|
||||
});
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Accuracy Analytics Dashboard:**
|
||||
```typescript
|
||||
// Admin dashboard for AI performance
|
||||
const AiPerformanceDashboard = () => {
|
||||
const [metrics, setMetrics] = useState<PerformanceMetrics>();
|
||||
|
||||
useEffect(() => {
|
||||
const loadMetrics = async () => {
|
||||
const data = await api.ai.analytics.getPerformance();
|
||||
setMetrics(data);
|
||||
};
|
||||
loadMetrics();
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<div className="grid grid-cols-1 md:grid-cols-3 gap-6">
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-sm">ความแม่นยำโดยรวม</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div className="text-2xl font-bold text-green-600">
|
||||
{metrics?.overallAccuracy}%
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-sm">อัตราการแก้ไขโดยผู้ใช้</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div className="text-2xl font-bold text-blue-600">
|
||||
{metrics?.userCorrectionRate}%
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle className="text-sm">เวลาประมวลผลเฉลี่ย</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div className="text-2xl font-bold text-purple-600">
|
||||
{metrics?.avgProcessingTime}s
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Feedback Data Structure:**
|
||||
```typescript
|
||||
// Types for feedback collection
|
||||
interface AiFeedbackDto {
|
||||
documentPublicId: string;
|
||||
field: string;
|
||||
aiSuggestion: string;
|
||||
userCorrection: string;
|
||||
confidence: number;
|
||||
timestamp: string;
|
||||
userAgent: string;
|
||||
}
|
||||
|
||||
interface PerformanceMetrics {
|
||||
overallAccuracy: number;
|
||||
userCorrectionRate: number;
|
||||
avgProcessingTime: number;
|
||||
fieldAccuracy: Record<string, number>;
|
||||
modelPerformance: Record<string, number>;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎨 UX/UI Design Guidelines
|
||||
|
||||
### Design Principles
|
||||
- **Trust through Transparency:** Always show AI confidence and sources
|
||||
- **Human Control First:** User can override any AI suggestion
|
||||
- **Progressive Disclosure:** Hide complexity, show details on demand
|
||||
- **Thai Language First:** All UI text in Thai, engineering terms in context
|
||||
|
||||
### Visual Indicators
|
||||
```typescript
|
||||
// Confidence score color coding
|
||||
const getConfidenceColor = (confidence: number) => {
|
||||
if (confidence >= 0.95) return 'text-green-600 bg-green-50';
|
||||
if (confidence >= 0.85) return 'text-yellow-600 bg-yellow-50';
|
||||
return 'text-red-600 bg-red-50';
|
||||
};
|
||||
|
||||
// AI suggestion highlighting
|
||||
const aiSuggestionStyles = {
|
||||
backgroundColor: '#fef3c7', // yellow-50
|
||||
borderLeft: '3px solid #f59e0b', // yellow-500
|
||||
padding: '0.5rem'
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔴 Critical Rules (Non-Negotiable)
|
||||
|
||||
1. **ADR-019 UUID Strategy:**
|
||||
- All API calls use `publicId` (string) only
|
||||
- NEVER use integer IDs or fallback patterns
|
||||
- Type safety: `publicId?: string` in interfaces
|
||||
|
||||
2. **ADR-018 AI Boundary:**
|
||||
- Frontend communicates with DMS API only
|
||||
- NO direct calls to n8n, Ollama, or PaddleOCR
|
||||
- AI processing via `/api/ai/extract` endpoint only
|
||||
|
||||
3. **Thai Language Standards:**
|
||||
- All UI text in Thai (i18n keys)
|
||||
- Code comments in Thai
|
||||
- Engineering terms preserved in original language
|
||||
|
||||
4. **Security Requirements:**
|
||||
- File uploads through StorageService only
|
||||
- Proper error handling without exposing system details
|
||||
- Rate limiting on AI endpoints
|
||||
|
||||
5. **Data Integrity:**
|
||||
- All AI suggestions require explicit user confirmation
|
||||
- Audit trail for all user corrections
|
||||
- Validation before form submission
|
||||
|
||||
---
|
||||
|
||||
## 📁 Related Specifications
|
||||
|
||||
- **ADR-018:** AI Boundary Policy - Security requirements
|
||||
- **ADR-019:** Hybrid Identifier Strategy - UUID usage patterns
|
||||
- **ADR-020:** AI Intelligence Integration - Architecture overview
|
||||
- **05-03-frontend-guidelines.md:** Next.js patterns and conventions
|
||||
- **05-08-i18n-guidelines.md:** Thai language implementation
|
||||
|
||||
---
|
||||
|
||||
## 📝 Component Library Usage
|
||||
|
||||
### Shadcn/UI Components
|
||||
```typescript
|
||||
// Required components for AI features
|
||||
import {
|
||||
Card,
|
||||
CardContent,
|
||||
CardHeader,
|
||||
CardTitle,
|
||||
Badge,
|
||||
Button,
|
||||
Input,
|
||||
Select,
|
||||
Slider,
|
||||
Collapsible,
|
||||
Dialog,
|
||||
Table,
|
||||
Pagination
|
||||
} from '@/components/ui';
|
||||
|
||||
// Custom AI components
|
||||
import { AiSuggestionField } from '@/components/ai/ai-suggestion-field';
|
||||
import { DocumentComparisonView } from '@/components/ai/document-comparison-view';
|
||||
import { AiProcessingIndicator } from '@/components/ai/processing-indicator';
|
||||
```
|
||||
|
||||
### Tailwind CSS Classes
|
||||
```css
|
||||
/* AI-specific utility classes */
|
||||
.ai-suggestion {
|
||||
@apply bg-yellow-50 border-l-4 border-yellow-500 p-3 rounded;
|
||||
}
|
||||
|
||||
.ai-high-confidence {
|
||||
@apply text-green-600 bg-green-50 border-green-500;
|
||||
}
|
||||
|
||||
.ai-medium-confidence {
|
||||
@apply text-yellow-600 bg-yellow-50 border-yellow-500;
|
||||
}
|
||||
|
||||
.ai-low-confidence {
|
||||
@apply text-red-600 bg-red-50 border-red-500;
|
||||
}
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user