260304:1716 20260304:1700 update n8n
All checks were successful
Build and Deploy / deploy (push) Successful in 1m33s
All checks were successful
Build and Deploy / deploy (push) Successful in 1m33s
This commit is contained in:
admin
@@ -30,7 +30,7 @@ Before generating code or planning a solution, you MUST conceptually load the co
|
|||||||
|
|
||||||
4. **💾 DATABASE & SCHEMA (`specs/03-Data-and-Storage/`)**
|
4. **💾 DATABASE & SCHEMA (`specs/03-Data-and-Storage/`)**
|
||||||
- _Action:_
|
- _Action:_
|
||||||
- **Read `specs/03-Data-and-Storage/lcbp3-v1.8.0-schema.sql`** for exact table structures and constraints.
|
- **Read `specs/03-Data-and-Storage/lcbp3-v1.8.0-schema-02-tables.sql`** for exact table structures and constraints. (Schema split: `01-drop`, `02-tables`, `03-views-indexes`)
|
||||||
- **Consult `specs/03-Data-and-Storage/03-01-data-dictionary.md`** for field meanings and business rules.
|
- **Consult `specs/03-Data-and-Storage/03-01-data-dictionary.md`** for field meanings and business rules.
|
||||||
- **Check `specs/03-Data-and-Storage/lcbp3-v1.8.0-seed-basic.sql`** to understand initial data states.
|
- **Check `specs/03-Data-and-Storage/lcbp3-v1.8.0-seed-basic.sql`** to understand initial data states.
|
||||||
- **Check `specs/03-Data-and-Storage/lcbp3-v1.8.0-seed-permissions.sql`** to understand initial permissions states.
|
- **Check `specs/03-Data-and-Storage/lcbp3-v1.8.0-seed-permissions.sql`** to understand initial permissions states.
|
||||||
@@ -70,7 +70,7 @@ When proposing a change or writing code, you must explicitly reference the sourc
|
|||||||
### 4. Schema Changes
|
### 4. Schema Changes
|
||||||
|
|
||||||
- **DO NOT** create or run TypeORM migration files.
|
- **DO NOT** create or run TypeORM migration files.
|
||||||
- Modify the schema directly in `specs/03-Data-and-Storage/lcbp3-v1.8.0-schema.sql`.
|
- Modify the schema directly in `specs/03-Data-and-Storage/lcbp3-v1.8.0-schema-02-tables.sql` (or `01-drop`/`03-views-indexes` as appropriate).
|
||||||
- Update `specs/03-Data-and-Storage/03-01-data-dictionary.md` if adding/changing columns.
|
- Update `specs/03-Data-and-Storage/03-01-data-dictionary.md` if adding/changing columns.
|
||||||
- Notify the user so they can apply the SQL change to the live database manually.
|
- Notify the user so they can apply the SQL change to the live database manually.
|
||||||
- **AI Isolation (ADR-018):** Ollama runs on ASUSTOR only. AI has NO direct DB access, NO write access to uploads. All writes go through DMS API.
|
- **AI Isolation (ADR-018):** Ollama runs on ASUSTOR only. AI has NO direct DB access, NO write access to uploads. All writes go through DMS API.
|
||||||
|
|||||||
@@ -44,7 +44,7 @@ You value **Data Integrity**, **Security**, and **Clean Architecture**.
|
|||||||
## 📋 Workflow & Spec Guidelines
|
## 📋 Workflow & Spec Guidelines
|
||||||
|
|
||||||
- Always follow specs in `specs/` (v1.8.0). Priority: `06-Decision-Records` > `05-Engineering-Guidelines` > others.
|
- Always follow specs in `specs/` (v1.8.0). Priority: `06-Decision-Records` > `05-Engineering-Guidelines` > others.
|
||||||
- Always verify database schema against **`specs/03-Data-and-Storage/lcbp3-v1.8.0-schema.sql`** before writing queries.
|
- Always verify database schema against **`specs/03-Data-and-Storage/lcbp3-v1.8.0-schema-02-tables.sql`** before writing queries. (Schema split: `01-drop`, `02-tables`, `03-views-indexes`)
|
||||||
- Check data dictionary at **`specs/03-Data-and-Storage/03-01-data-dictionary.md`** for field meanings and business rules.
|
- Check data dictionary at **`specs/03-Data-and-Storage/03-01-data-dictionary.md`** for field meanings and business rules.
|
||||||
- Check seed data: **`lcbp3-v1.8.0-seed-basic.sql`** (reference data), **`lcbp3-v1.8.0-seed-permissions.sql`** (CASL permissions).
|
- Check seed data: **`lcbp3-v1.8.0-seed-basic.sql`** (reference data), **`lcbp3-v1.8.0-seed-permissions.sql`** (CASL permissions).
|
||||||
- For migration context: **`specs/03-Data-and-Storage/03-04-legacy-data-migration.md`** and **`03-05-n8n-migration-setup-guide.md`**.
|
- For migration context: **`specs/03-Data-and-Storage/03-04-legacy-data-migration.md`** and **`03-05-n8n-migration-setup-guide.md`**.
|
||||||
|
|||||||
@@ -42,7 +42,7 @@ You value **Data Integrity**, **Security**, and **Clean Architecture**.
|
|||||||
## 📋 Spec Guidelines
|
## 📋 Spec Guidelines
|
||||||
|
|
||||||
- Always follow specs in `specs/` (v1.8.0). Priority: `06-Decision-Records` > `05-Engineering-Guidelines` > others.
|
- Always follow specs in `specs/` (v1.8.0). Priority: `06-Decision-Records` > `05-Engineering-Guidelines` > others.
|
||||||
- Always verify database schema against **`specs/03-Data-and-Storage/lcbp3-v1.8.0-schema.sql`** before writing queries.
|
- Always verify database schema against **`specs/03-Data-and-Storage/lcbp3-v1.8.0-schema-02-tables.sql`** before writing queries. (Schema split: `01-drop`, `02-tables`, `03-views-indexes`)
|
||||||
- Check data dictionary at **`specs/03-Data-and-Storage/03-01-data-dictionary.md`** for field meanings and business rules.
|
- Check data dictionary at **`specs/03-Data-and-Storage/03-01-data-dictionary.md`** for field meanings and business rules.
|
||||||
|
|
||||||
### ADR Reference (All 17 + Patch)
|
### ADR Reference (All 17 + Patch)
|
||||||
|
|||||||
@@ -40,7 +40,7 @@ You value **Data Integrity**, **Security**, and **Clean Architecture**.
|
|||||||
## 📋 Workflow & Spec Guidelines
|
## 📋 Workflow & Spec Guidelines
|
||||||
|
|
||||||
- Always follow specs in `specs/` (v1.8.0). Priority: `06-Decision-Records` > `05-Engineering-Guidelines` > others.
|
- Always follow specs in `specs/` (v1.8.0). Priority: `06-Decision-Records` > `05-Engineering-Guidelines` > others.
|
||||||
- Always verify database schema against **`specs/03-Data-and-Storage/lcbp3-v1.8.0-schema.sql`** before writing queries.
|
- Always verify database schema against **`specs/03-Data-and-Storage/lcbp3-v1.8.0-schema-02-tables.sql`** before writing queries. (Schema split: `01-drop`, `02-tables`, `03-views-indexes`)
|
||||||
- Check data dictionary at **`specs/03-Data-and-Storage/03-01-data-dictionary.md`** for field meanings and business rules.
|
- Check data dictionary at **`specs/03-Data-and-Storage/03-01-data-dictionary.md`** for field meanings and business rules.
|
||||||
- Check seed data: **`lcbp3-v1.8.0-seed-basic.sql`** (reference data), **`lcbp3-v1.8.0-seed-permissions.sql`** (CASL permissions).
|
- Check seed data: **`lcbp3-v1.8.0-seed-basic.sql`** (reference data), **`lcbp3-v1.8.0-seed-permissions.sql`** (CASL permissions).
|
||||||
- For migration context: **`specs/03-Data-and-Storage/03-04-legacy-data-migration.md`** and **`03-05-n8n-migration-setup-guide.md`**.
|
- For migration context: **`specs/03-Data-and-Storage/03-04-legacy-data-migration.md`** and **`03-05-n8n-migration-setup-guide.md`**.
|
||||||
|
|||||||
975
specs/03-Data-and-Storage/0.md
Normal file
975
specs/03-Data-and-Storage/0.md
Normal file
@@ -0,0 +1,975 @@
|
|||||||
|
ผมจะสร้างไฟล์ JSON สำหรับ n8n Free Plan (ไม่ใช้ Environment Variables) และปรับปรุงเอกสารคู่มือให้สอดคล้อง โดยใช้ **Static Parameters** และ **Set Node** แทนการอ่านจาก Environment
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 ไฟล์ `n8n.workflow` (Free Plan Compatible)
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"name": "LCBP3 Migration Workflow v1.8.0",
|
||||||
|
"meta": {
|
||||||
|
"instanceId": "lcbp3-migration-free"
|
||||||
|
},
|
||||||
|
"settings": {
|
||||||
|
"executionOrder": "v1"
|
||||||
|
},
|
||||||
|
"nodes": [
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"rule": {
|
||||||
|
"interval": [
|
||||||
|
{
|
||||||
|
"field": "cronExpression",
|
||||||
|
"expression": "0 22 * * *"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"id": "trigger-schedule",
|
||||||
|
"name": "Schedule Trigger (22:00)",
|
||||||
|
"type": "n8n-nodes-base.scheduleTrigger",
|
||||||
|
"typeVersion": 1.1,
|
||||||
|
"position": [0, 0],
|
||||||
|
"notes": "เริ่มรันอัตโนมัติทุกวันเวลา 22:00"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "// ============================================\n// CONFIGURATION - แก้ไขค่าที่นี่\n// ============================================\nconst CONFIG = {\n // Ollama Settings\n OLLAMA_HOST: 'http://192.168.20.100:11434',\n OLLAMA_MODEL_PRIMARY: 'llama3.2:3b',\n OLLAMA_MODEL_FALLBACK: 'mistral:7b-instruct-q4_K_M',\n \n // Backend Settings\n BACKEND_URL: 'https://api.np-dms.work',\n MIGRATION_TOKEN: 'Bearer YOUR_MIGRATION_TOKEN_HERE',\n \n // Batch Settings\n BATCH_SIZE: 10,\n BATCH_ID: 'migration_20260226',\n DELAY_MS: 2000,\n \n // Thresholds\n CONFIDENCE_HIGH: 0.85,\n CONFIDENCE_LOW: 0.60,\n MAX_RETRY: 3,\n FALLBACK_THRESHOLD: 5,\n \n // Paths\n STAGING_PATH: '/share/np-dms/staging_ai',\n LOG_PATH: '/share/np-dms/n8n/migration_logs',\n \n // Database\n DB_HOST: '192.168.1.100',\n DB_PORT: 3306,\n DB_NAME: 'lcbp3_production',\n DB_USER: 'migration_bot',\n DB_PASSWORD: 'YOUR_DB_PASSWORD_HERE'\n};\n\n// Store in global workflow data\n$workflow.staticData = $workflow.staticData || {};\n$workflow.staticData.config = CONFIG;\n\nreturn [{ json: { config_loaded: true, timestamp: new Date().toISOString(), config: CONFIG } }];"
|
||||||
|
},
|
||||||
|
"id": "config-setter",
|
||||||
|
"name": "Set Configuration",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [200, 0],
|
||||||
|
"notes": "กำหนดค่า Configuration ทั้งหมด - แก้ไขที่นี่ก่อนรัน"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"method": "GET",
|
||||||
|
"url": "={{$workflow.staticData.config.BACKEND_URL}}/api/meta/categories",
|
||||||
|
"sendHeaders": true,
|
||||||
|
"headerParameters": {
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"name": "Authorization",
|
||||||
|
"value": "={{$workflow.staticData.config.MIGRATION_TOKEN}}"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"options": {
|
||||||
|
"timeout": 10000
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"id": "preflight-categories",
|
||||||
|
"name": "Fetch Categories",
|
||||||
|
"type": "n8n-nodes-base.httpRequest",
|
||||||
|
"typeVersion": 4.1,
|
||||||
|
"position": [400, 0],
|
||||||
|
"notes": "ดึง Categories จาก Backend"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"method": "GET",
|
||||||
|
"url": "={{$workflow.staticData.config.BACKEND_URL}}/api/health",
|
||||||
|
"options": {
|
||||||
|
"timeout": 5000
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"id": "preflight-health",
|
||||||
|
"name": "Check Backend Health",
|
||||||
|
"type": "n8n-nodes-base.httpRequest",
|
||||||
|
"typeVersion": 4.1,
|
||||||
|
"position": [400, 200],
|
||||||
|
"notes": "ตรวจสอบ Backend พร้อมใช้งาน",
|
||||||
|
"onError": "continueErrorOutput"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const fs = require('fs');\nconst config = $workflow.staticData.config;\n\n// Check file mount\ntry {\n const files = fs.readdirSync(config.STAGING_PATH);\n if (files.length === 0) throw new Error('staging_ai is empty');\n \n // Check write permission to log path\n fs.writeFileSync(`${config.LOG_PATH}/.preflight_ok`, new Date().toISOString());\n \n // Store categories\n const categories = $input.first().json.categories || \n ['Correspondence','RFA','Drawing','Transmittal','Report','Other'];\n $workflow.staticData.systemCategories = categories;\n \n return [{ json: { \n preflight_ok: true, \n file_count: files.length,\n system_categories: categories,\n timestamp: new Date().toISOString()\n }}];\n} catch (err) {\n throw new Error(`Pre-flight check failed: ${err.message}`);\n}"
|
||||||
|
},
|
||||||
|
"id": "preflight-check",
|
||||||
|
"name": "File Mount Check",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [600, 0],
|
||||||
|
"notes": "ตรวจสอบ File System และเก็บ Categories"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
n "host": "={{$workflow.staticData.config.DB_HOST}}",
|
||||||
|
"port": "={{$workflow.staticData.config.DB_PORT}}",
|
||||||
|
"database": "={{$workflow.staticData.config.DB_NAME}}",
|
||||||
|
"user": "={{$workflow.staticData.config.DB_USER}}",
|
||||||
|
"password": "={{$workflow.staticData.config.DB_PASSWORD}}",
|
||||||
|
"query": "SELECT last_processed_index, status FROM migration_progress WHERE batch_id = '{{$workflow.staticData.config.BATCH_ID}}' LIMIT 1",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "checkpoint-read",
|
||||||
|
"name": "Read Checkpoint",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4,
|
||||||
|
"position": [800, 0],
|
||||||
|
"notes": "อ่านตำแหน่งล่าสุดที่ประมวลผล",
|
||||||
|
"onError": "continueErrorOutput"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "toData",
|
||||||
|
"binaryProperty": "data",
|
||||||
|
"options": {
|
||||||
|
"sheetName": "Sheet1"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"id": "excel-reader",
|
||||||
|
"name": "Read Excel",
|
||||||
|
"type": "n8n-nodes-base.spreadsheetFile",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [800, 200],
|
||||||
|
"notes": "อ่านไฟล์ Excel รายการเอกสาร"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const checkpoint = $input.first().json[0] || { last_processed_index: 0, status: 'NEW' };\nconst startIndex = checkpoint.last_processed_index || 0;\nconst config = $workflow.staticData.config;\n\nconst allItems = $('Read Excel').all()[0].json.data || [];\nconst remaining = allItems.slice(startIndex);\nconst currentBatch = remaining.slice(0, config.BATCH_SIZE);\n\n// Encoding Normalization\nconst normalize = (str) => {\n if (!str) return '';\n return String(str).normalize('NFC').trim();\n};\n\nreturn currentBatch.map((item, i) => ({\n json: {\n document_number: normalize(item.document_number || item['Document Number']),\n title: normalize(item.title || item.Title || item['Subject']),\n legacy_number: normalize(item.legacy_number || item['Legacy Number']),\n excel_revision: item.revision || item.Revision || 1,\n original_index: startIndex + i,\n batch_id: config.BATCH_ID,\n file_name: `${normalize(item.document_number)}.pdf`\n }\n}));"
|
||||||
|
},
|
||||||
|
"id": "batch-processor",
|
||||||
|
"name": "Process Batch + Encoding",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [1000, 0],
|
||||||
|
"notes": "ตัด Batch + Normalize UTF-8"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const fs = require('fs');\nconst path = require('path');\nconst config = $workflow.staticData.config;\n\nconst items = $input.all();\nconst validated = [];\nconst errors = [];\n\nfor (const item of items) {\n const docNum = item.json.document_number;\n \n // Sanitize filename\n const safeName = path.basename(String(docNum).replace(/[^a-zA-Z0-9\\-_.]/g, '_')).normalize('NFC');\n const filePath = path.resolve(config.STAGING_PATH, `${safeName}.pdf`);\n \n // Path traversal check\n if (!filePath.startsWith(config.STAGING_PATH)) {\n errors.push({\n ...item,\n json: { ...item.json, error: 'Path traversal detected', error_type: 'SECURITY', file_exists: false }\n });\n continue;\n }\n \n try {\n if (fs.existsSync(filePath)) {\n const stats = fs.statSync(filePath);\n validated.push({\n ...item,\n json: { ...item.json, file_exists: true, file_size: stats.size, file_path: filePath }\n });\n } else {\n errors.push({\n ...item,\n json: { ...item.json, error: `File not found: ${safeName}.pdf`, error_type: 'FILE_NOT_FOUND', file_exists: false }\n });\n }\n } catch (err) {\n errors.push({\n ...item,\n json: { ...item.json, error: err.message, error_type: 'FILE_ERROR', file_exists: false }\n });\n }\n}\n\n// Output 0: Validated, Output 1: Errors\nreturn [validated, errors];"
|
||||||
|
},
|
||||||
|
"id": "file-validator",
|
||||||
|
"name": "File Validator",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [1200, 0],
|
||||||
|
"notes": "ตรวจสอบไฟล์ PDF มีอยู่จริง + Sanitize path"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$workflow.staticData.config.DB_HOST}}",
|
||||||
|
"port": "={{$workflow.staticData.config.DB_PORT}}",
|
||||||
|
"database": "={{$workflow.staticData.config.DB_NAME}}",
|
||||||
|
"user": "={{$workflow.staticData.config.DB_USER}}",
|
||||||
|
"password": "={{$workflow.staticData.config.DB_PASSWORD}}",
|
||||||
|
"query": "SELECT is_fallback_active, recent_error_count FROM migration_fallback_state WHERE batch_id = '{{$workflow.staticData.config.BATCH_ID}}' LIMIT 1",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "fallback-check",
|
||||||
|
"name": "Check Fallback State",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4,
|
||||||
|
"position": [1400, -200],
|
||||||
|
"notes": "ตรวจสอบว่าต้องใช้ Fallback Model หรือไม่",
|
||||||
|
"onError": "continueErrorOutput"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const config = $workflow.staticData.config;\nconst fallbackState = $input.first().json[0] || { is_fallback_active: false, recent_error_count: 0 };\n\nconst isFallback = fallbackState.is_fallback_active || false;\nconst model = isFallback ? config.OLLAMA_MODEL_FALLBACK : config.OLLAMA_MODEL_PRIMARY;\n\nconst systemCategories = $workflow.staticData.systemCategories || \n ['Correspondence','RFA','Drawing','Transmittal','Report','Other'];\n\nconst items = $('File Validator').all();\n\nreturn items.map(item => {\n const systemPrompt = `You are a Document Controller for a large construction project.\nYour task is to validate document metadata.\nYou MUST respond ONLY with valid JSON. No explanation, no markdown, no extra text.\nIf there are no issues, \"detected_issues\" must be an empty array [].`;\n\n const userPrompt = `Validate this document metadata and respond in JSON:\n\nDocument Number: ${item.json.document_number}\nTitle: ${item.json.title}\nExpected Pattern: [ORG]-[TYPE]-[SEQ] e.g. \"TCC-COR-0001\"\nCategory List (MUST match system enum exactly): ${JSON.stringify(systemCategories)}\n\nRespond ONLY with this exact JSON structure:\n{\n \"is_valid\": true | false,\n \"confidence\": 0.0 to 1.0,\n \"suggested_category\": \"<one from Category List>\",\n \"detected_issues\": [\"<issue1>\"],\n \"suggested_title\": \"<corrected title or null>\"\n}`;\n\n return {\n json: {\n ...item.json,\n active_model: model,\n is_fallback: isFallback,\n system_categories: systemCategories,\n ollama_payload: {\n model: model,\n prompt: `${systemPrompt}\\n\\n${userPrompt}`,\n stream: false,\n format: 'json'\n }\n }\n };\n});"
|
||||||
|
},
|
||||||
|
"id": "prompt-builder",
|
||||||
|
"name": "Build AI Prompt",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [1400, 0],
|
||||||
|
"notes": "สร้าง Prompt โดยใช้ Categories จาก System"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"method": "POST",
|
||||||
|
"url": "={{$workflow.staticData.config.OLLAMA_HOST}}/api/generate",
|
||||||
|
"sendBody": true,
|
||||||
|
"specifyBody": "json",
|
||||||
|
"jsonBody": "={{ $json.ollama_payload }}",
|
||||||
|
"options": {
|
||||||
|
"timeout": 30000
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"id": "ollama-call",
|
||||||
|
"name": "Ollama AI Analysis",
|
||||||
|
"type": "n8n-nodes-base.httpRequest",
|
||||||
|
"typeVersion": 4.1,
|
||||||
|
"position": [1600, 0],
|
||||||
|
"notes": "เรียก Ollama วิเคราะห์เอกสาร"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const items = $input.all();\nconst parsed = [];\nconst parseErrors = [];\n\nfor (const item of items) {\n try {\n let raw = item.json.response || '';\n \n // Clean markdown\n raw = raw.replace(/```json/gi, '').replace(/```/g, '').trim();\n const result = JSON.parse(raw);\n \n // Schema Validation\n if (typeof result.is_valid !== 'boolean') throw new Error('is_valid must be boolean');\n if (typeof result.confidence !== 'number' || result.confidence < 0 || result.confidence > 1) {\n throw new Error('confidence must be float 0.0-1.0');\n }\n if (!Array.isArray(result.detected_issues)) throw new Error('detected_issues must be array');\n \n // Enum Validation\n const systemCategories = item.json.system_categories || [];\n if (!systemCategories.includes(result.suggested_category)) {\n throw new Error(`Category \"${result.suggested_category}\" not in system enum`);\n }\n \n parsed.push({\n ...item,\n json: { ...item.json, ai_result: result, parse_error: null }\n });\n } catch (err) {\n parseErrors.push({\n ...item,\n json: {\n ...item.json,\n ai_result: null,\n parse_error: err.message,\n raw_ai_response: item.json.response,\n error_type: 'AI_PARSE_ERROR'\n }\n });\n }\n}\n\nreturn [parsed, parseErrors];"
|
||||||
|
},
|
||||||
|
"id": "json-parser",
|
||||||
|
"name": "Parse & Validate AI Response",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [1800, 0],
|
||||||
|
"notes": "Parse JSON + Validate Schema + Enum Check"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$workflow.staticData.config.DB_HOST}}",
|
||||||
|
"port": "={{$workflow.staticData.config.DB_PORT}}",
|
||||||
|
"database": "={{$workflow.staticData.config.DB_NAME}}",
|
||||||
|
"user": "={{$workflow.staticData.config.DB_USER}}",
|
||||||
|
"password": "={{$workflow.staticData.config.DB_PASSWORD}}",
|
||||||
|
"query": "INSERT INTO migration_fallback_state (batch_id, recent_error_count, is_fallback_active) VALUES ('{{$workflow.staticData.config.BATCH_ID}}', 1, FALSE) ON DUPLICATE KEY UPDATE recent_error_count = recent_error_count + 1, is_fallback_active = CASE WHEN recent_error_count + 1 >= {{$workflow.staticData.config.FALLBACK_THRESHOLD}} THEN TRUE ELSE is_fallback_active END, updated_at = NOW()",
|
||||||
|
"options": {}
|
||||||
|
n },
|
||||||
|
"id": "fallback-update",
|
||||||
|
"name": "Update Fallback State",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4,
|
||||||
|
"position": [2000, 200],
|
||||||
|
"notes": "เพิ่ม Error count และตรวจสอบ Fallback threshold"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const config = $workflow.staticData.config;\nconst items = $('Parse & Validate AI Response').all();\n\nconst autoIngest = [];\nconst reviewQueue = [];\nconst rejectLog = [];\nconst errorLog = [];\n\nfor (const item of items) {\n if (item.json.parse_error || !item.json.ai_result) {\n errorLog.push(item);\n continue;\n }\n \n const ai = item.json.ai_result;\n \n // Revision Drift Protection (ถ้ามีข้อมูลจาก DB)\n if (item.json.current_db_revision !== undefined) {\n const expectedRev = item.json.current_db_revision + 1;\n if (parseInt(item.json.excel_revision) !== expectedRev) {\n reviewQueue.push({\n ...item,\n json: { ...item.json, review_reason: `Revision drift: Excel=${item.json.excel_revision}, Expected=${expectedRev}` }\n });\n continue;\n }\n }\n \n // Confidence Routing\n if (ai.confidence >= config.CONFIDENCE_HIGH && ai.is_valid === true) {\n autoIngest.push(item);\n } else if (ai.confidence >= config.CONFIDENCE_LOW) {\n reviewQueue.push({\n ...item,\n json: { ...item.json, review_reason: `Confidence ${ai.confidence.toFixed(2)} < ${config.CONFIDENCE_HIGH}` }\n });\n } else {\n rejectLog.push({\n ...item,\n json: { ...item.json, reject_reason: ai.is_valid === false ? 'AI marked invalid' : `Confidence ${ai.confidence.toFixed(2)} < ${config.CONFIDENCE_LOW}` }\n });\n }\n}\n\n// Output 0: Auto, 1: Review, 2: Reject, 3: Error\nreturn [autoIngest, reviewQueue, rejectLog, errorLog];"
|
||||||
|
},
|
||||||
|
"id": "confidence-router",
|
||||||
|
"name": "Confidence Router",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [2000, 0],
|
||||||
|
"notes": "แยกตาม Confidence: Auto(≥0.85) / Review(≥0.60) / Reject(<0.60)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"method": "POST",
|
||||||
|
"url": "={{$workflow.staticData.config.BACKEND_URL}}/api/correspondences/import",
|
||||||
|
"sendHeaders": true,
|
||||||
|
"headerParameters": {
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"name": "Authorization",
|
||||||
|
"value": "={{$workflow.staticData.config.MIGRATION_TOKEN}}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "Idempotency-Key",
|
||||||
|
"value": "={{$json.document_number}}:{{$workflow.staticData.config.BATCH_ID}}"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"sendBody": true,
|
||||||
|
"specifyBody": "json",
|
||||||
|
"jsonBody": "={\n \"document_number\": \"{{$json.document_number}}\",\n \"title\": \"{{$json.ai_result.suggested_title || $json.title}}\",\n \"category\": \"{{$json.ai_result.suggested_category}}\",\n \"source_file_path\": \"{{$json.file_path}}\",\n \"ai_confidence\": {{$json.ai_result.confidence}},\n \"ai_issues\": {{JSON.stringify($json.ai_result.detected_issues)}},\n \"migrated_by\": \"SYSTEM_IMPORT\",\n \"batch_id\": \"{{$workflow.staticData.config.BATCH_ID}}\",\n \"details\": {\n \"legacy_number\": \"{{$json.legacy_number}}\"\n }\n}",
|
||||||
|
"options": {
|
||||||
|
"timeout": 30000
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"id": "backend-import",
|
||||||
|
"name": "Import to Backend",
|
||||||
|
"type": "n8n-nodes-base.httpRequest",
|
||||||
|
"typeVersion": 4.1,
|
||||||
|
"position": [2200, -200],
|
||||||
|
"notes": "ส่งข้อมูลเข้า LCBP3 Backend พร้อม Idempotency-Key"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const item = $input.first();\nconst shouldCheckpoint = item.json.original_index % 10 === 0;\n\nreturn [{\n json: {\n ...item.json,\n should_update_checkpoint: shouldCheckpoint,\n checkpoint_index: item.json.original_index,\n import_status: 'success',\n timestamp: new Date().toISOString()\n }\n}];"
|
||||||
|
},
|
||||||
|
"id": "checkpoint-flag",
|
||||||
|
"name": "Flag Checkpoint",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [2400, -200],
|
||||||
|
"notes": "กำหนดว่าจะบันทึก Checkpoint หรือไม่ (ทุก 10 records)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$workflow.staticData.config.DB_HOST}}",
|
||||||
|
"port": "={{$workflow.staticData.config.DB_PORT}}",
|
||||||
|
"database": "={{$workflow.staticData.config.DB_NAME}}",
|
||||||
|
"user": "={{$workflow.staticData.config.DB_USER}}",
|
||||||
|
"password": "={{$workflow.staticData.config.DB_PASSWORD}}",
|
||||||
|
"query": "INSERT INTO migration_progress (batch_id, last_processed_index, status) VALUES ('{{$workflow.staticData.config.BATCH_ID}}', {{$json.checkpoint_index}}, 'RUNNING') ON DUPLICATE KEY UPDATE last_processed_index = {{$json.checkpoint_index}}, updated_at = NOW()",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "checkpoint-save",
|
||||||
|
"name": "Save Checkpoint",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4,
|
||||||
|
"position": [2600, -200],
|
||||||
|
"notes": "บันทึกความคืบหน้าลง Database"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$workflow.staticData.config.DB_HOST}}",
|
||||||
|
"port": "={{$workflow.staticData.config.DB_PORT}}",
|
||||||
|
"database": "={{$workflow.staticData.config.DB_NAME}}",
|
||||||
|
"user": "={{$workflow.staticData.config.DB_USER}}",
|
||||||
|
"password": "={{$workflow.staticData.config.DB_PASSWORD}}",
|
||||||
|
"query": "INSERT INTO migration_review_queue (document_number, title, original_title, ai_suggested_category, ai_confidence, ai_issues, review_reason, status, created_at) VALUES ('{{$json.document_number}}', '{{$json.ai_result.suggested_title || $json.title}}', '{{$json.title}}', '{{$json.ai_result.suggested_category}}', {{$json.ai_result.confidence}}, '{{JSON.stringify($json.ai_result.detected_issues)}}', '{{$json.review_reason}}', 'PENDING', NOW()) ON DUPLICATE KEY UPDATE status = 'PENDING', review_reason = '{{$json.review_reason}}', created_at = NOW()",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "review-queue-insert",
|
||||||
|
"name": "Insert Review Queue",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4",
|
||||||
|
"position": [2200, 0],
|
||||||
|
"notes": "บันทึกรายการที่ต้องตรวจสอบโดยคน (ไม่สร้าง Correspondence)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const fs = require('fs');\nconst item = $input.first();\nconst config = $workflow.staticData.config;\n\nconst csvPath = `${config.LOG_PATH}/reject_log.csv`;\nconst header = 'timestamp,document_number,title,reject_reason,ai_confidence,ai_issues\\n';\nconst esc = (s) => `\"${String(s || '').replace(/\"/g, '\"\"')}\"`;\n\nif (!fs.existsSync(csvPath)) {\n fs.writeFileSync(csvPath, header, 'utf8');\n}\n\nconst line = [\n new Date().toISOString(),\n esc(item.json.document_number),\n esc(item.json.title),\n esc(item.json.reject_reason),\n item.json.ai_result?.confidence ?? 'N/A',\n esc(JSON.stringify(item.json.ai_result?.detected_issues || []))\n].join(',') + '\\n';\n\nfs.appendFileSync(csvPath, line, 'utf8');\n\nreturn [$input.first()];"
|
||||||
|
},
|
||||||
|
"id": "reject-logger",
|
||||||
|
"name": "Log Reject to CSV",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [2200, 200],
|
||||||
|
"notes": "บันทึกรายการที่ถูกปฏิเสธลง CSV"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const fs = require('fs');\nconst items = $input.all();\nconst config = $workflow.staticData.config;\n\nconst csvPath = `${config.LOG_PATH}/error_log.csv`;\nconst header = 'timestamp,document_number,error_type,error_message,raw_ai_response\\n';\nconst esc = (s) => `\"${String(s || '').replace(/\"/g, '\"\"')}\"`;\n\nif (!fs.existsSync(csvPath)) {\n fs.writeFileSync(csvPath, header, 'utf8');\n}\n\nfor (const item of items) {\n const line = [\n new Date().toISOString(),\n esc(item.json.document_number),\n esc(item.json.error_type || 'UNKNOWN'),\n esc(item.json.error || item.json.parse_error),\n esc(item.json.raw_ai_response || '')\n ].join(',') + '\\n';\n \n fs.appendFileSync(csvPath, line, 'utf8');\n}\n\nreturn items;"
|
||||||
|
},
|
||||||
|
"id": "error-logger-csv",
|
||||||
|
"name": "Log Error to CSV",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [1400, 400],
|
||||||
|
"notes": "บันทึก Error ลง CSV (จาก File Validator)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$workflow.staticData.config.DB_HOST}}",
|
||||||
|
"port": "={{$workflow.staticData.config.DB_PORT}}",
|
||||||
|
"database": "={{$workflow.staticData.config.DB_NAME}}",
|
||||||
|
"user": "={{$workflow.staticData.config.DB_USER}}",
|
||||||
|
"password": "={{$workflow.staticData.config.DB_PASSWORD}}",
|
||||||
|
"query": "INSERT INTO migration_errors (batch_id, document_number, error_type, error_message, raw_ai_response, created_at) VALUES ('{{$workflow.staticData.config.BATCH_ID}}', '{{$json.document_number}}', '{{$json.error_type || \"UNKNOWN\"}}', '{{$json.error || $json.parse_error}}', '{{$json.raw_ai_response || \"\"}}', NOW())",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "error-logger-db",
|
||||||
|
"name": "Log Error to DB",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4",
|
||||||
|
"position": [2000, 400],
|
||||||
|
"notes": "บันทึก Error ลง MariaDB"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"amount": "={{$workflow.staticData.config.DELAY_MS}}",
|
||||||
|
"unit": "milliseconds"
|
||||||
|
},
|
||||||
|
"id": "delay-node",
|
||||||
|
"name": "Delay",
|
||||||
|
"type": "n8n-nodes-base.wait",
|
||||||
|
"typeVersion": 1,
|
||||||
|
"position": [2800, 0],
|
||||||
|
"notes": "หน่วงเวลาระหว่าง Batches"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"connections": {
|
||||||
|
"Schedule Trigger (22:00)": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Set Configuration",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Set Configuration": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Fetch Categories",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Fetch Categories": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "File Mount Check",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"File Mount Check": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Read Checkpoint",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Read Checkpoint": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Process Batch + Encoding",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Process Batch + Encoding": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "File Validator",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"File Validator": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Build AI Prompt",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Log Error to CSV",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Build AI Prompt": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Ollama AI Analysis",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Ollama AI Analysis": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Parse & Validate AI Response",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Parse & Validate AI Response": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Confidence Router",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Update Fallback State",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Confidence Router": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Import to Backend",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Insert Review Queue",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Log Reject to CSV",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Log Error to DB",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Import to Backend": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Flag Checkpoint",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Flag Checkpoint": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Save Checkpoint",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Log Error to CSV": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Log Error to DB",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📄 เอกสารปรับปรุง `03-05-n8n-migration-setup-guide.md` (Free Plan Edition)
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# 📋 คู่มือการตั้งค่า n8n สำหรับ Legacy Data Migration (Free Plan Edition)
|
||||||
|
|
||||||
|
> **สำหรับ n8n Free Plan (Self-hosted)** - ไม่ใช้ Environment Variables
|
||||||
|
> **Version:** 1.8.0-free | **Last Updated:** 2026-03-03
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ ความแตกต่างจากเวอร์ชัน Enterprise
|
||||||
|
|
||||||
|
| ฟีเจอร์ | Enterprise | Free Plan (นี้) |
|
||||||
|
|---------|-----------|----------------|
|
||||||
|
| Environment Variables | ✅ ใช้ `$env` | ❌ ใช้ `Set Node` + `staticData` |
|
||||||
|
| External Secrets | ✅ Vault/Secrets Manager | ❌ Hardcode ใน Set Node |
|
||||||
|
| Multiple Workflows | ✅ Unlimited | ⚠️ รวมเป็น Workflow เดียว |
|
||||||
|
| Error Handling | ✅ Advanced | ⚠️ Manual Retry |
|
||||||
|
| Webhook Triggers | ✅ | ✅ ใช้ได้ |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🏗️ สถาปัตยกรรมใหม่สำหรับ Free Plan
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
|
│ MIGRATION WORKFLOW v1.8.0-FREE │
|
||||||
|
├─────────────────────────────────────────────────────────────┤
|
||||||
|
│ │
|
||||||
|
│ [Schedule Trigger 22:00] │
|
||||||
|
│ │ │
|
||||||
|
│ ▼ │
|
||||||
|
│ ┌─────────────┐ ค่า Config ทั้งหมดอยู่ที่นี่ │
|
||||||
|
│ │ Set Config │ (แก้ไขใน Code Node นี้เท่านั้น) │
|
||||||
|
│ │ (Node 0) │ │
|
||||||
|
│ └──────┬──────┘ │
|
||||||
|
│ │ │
|
||||||
|
│ ┌──────▼──────┐ ┌──────────────┐ ┌──────────────┐ │
|
||||||
|
│ │Pre-flight │───▶│Fetch Categories│──▶│File Validator│ │
|
||||||
|
│ │Checks │ │from Backend │ │+ Sanitize │ │
|
||||||
|
│ └─────────────┘ └──────────────┘ └──────┬───────┘ │
|
||||||
|
│ │ │
|
||||||
|
│ ┌────────────────────────────┤ │
|
||||||
|
│ │ │ │
|
||||||
|
│ Valid │ Error │ │
|
||||||
|
│ ▼ ▼ │
|
||||||
|
│ ┌─────────────────┐ ┌─────────────────┐ │
|
||||||
|
│ │ AI Analysis │ │ Error Logger │ │
|
||||||
|
│ │ (Ollama) │ │ (CSV + DB) │ │
|
||||||
|
│ └────────┬────────┘ └─────────────────┘ │
|
||||||
|
│ │ │
|
||||||
|
│ ┌────────▼────────┐ │
|
||||||
|
│ │ Confidence │ │
|
||||||
|
│ │ Router │ │
|
||||||
|
│ │ (4 outputs) │ │
|
||||||
|
│ └────┬───┬───┬────┘ │
|
||||||
|
│ │ │ │ │
|
||||||
|
│ ┌─────────┘ │ └─────────┐ │
|
||||||
|
│ ▼ ▼ ▼ │
|
||||||
|
│ ┌──────┐ ┌──────────┐ ┌────────┐ │
|
||||||
|
│ │Auto │ │ Review │ │Reject │ │
|
||||||
|
│ │Ingest│ │ Queue │ │Log │ │
|
||||||
|
│ │+Chkpt│ │(DB only) │ │(CSV) │ │
|
||||||
|
│ └──────┘ └──────────┘ └────────┘ │
|
||||||
|
│ │
|
||||||
|
└─────────────────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 การตั้งค่า Configuration (สำคัญมาก)
|
||||||
|
|
||||||
|
### ขั้นตอนที่ 1: แก้ไข Node "Set Configuration"
|
||||||
|
|
||||||
|
**เปิด Workflow → คลิก Node "Set Configuration" → แก้ไข Code:**
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// ============================================
|
||||||
|
// CONFIGURATION - แก้ไขค่าที่นี่เท่านั้น
|
||||||
|
// ============================================
|
||||||
|
const CONFIG = {
|
||||||
|
// 🔴 สำคัญ: เปลี่ยนทุกค่าที่มี <...>
|
||||||
|
|
||||||
|
// Ollama Settings
|
||||||
|
OLLAMA_HOST: 'http://192.168.20.100:11434',
|
||||||
|
OLLAMA_MODEL_PRIMARY: 'llama3.2:3b',
|
||||||
|
OLLAMA_MODEL_FALLBACK: 'mistral:7b-instruct-q4_K_M',
|
||||||
|
|
||||||
|
// Backend Settings
|
||||||
|
BACKEND_URL: 'https://api.np-dms.work',
|
||||||
|
MIGRATION_TOKEN: 'Bearer YOUR_MIGRATION_TOKEN_HERE', // 🔴 เปลี่ยน
|
||||||
|
|
||||||
|
// Batch Settings
|
||||||
|
BATCH_SIZE: 10,
|
||||||
|
BATCH_ID: 'migration_20260226',
|
||||||
|
DELAY_MS: 2000,
|
||||||
|
|
||||||
|
// Thresholds
|
||||||
|
CONFIDENCE_HIGH: 0.85,
|
||||||
|
CONFIDENCE_LOW: 0.60,
|
||||||
|
MAX_RETRY: 3,
|
||||||
|
FALLBACK_THRESHOLD: 5,
|
||||||
|
|
||||||
|
// Paths (QNAP NAS)
|
||||||
|
STAGING_PATH: '/share/np-dms/staging_ai',
|
||||||
|
LOG_PATH: '/share/np-dms/n8n/migration_logs',
|
||||||
|
|
||||||
|
// Database (MariaDB)
|
||||||
|
DB_HOST: '192.168.1.100',
|
||||||
|
DB_PORT: 3306,
|
||||||
|
DB_NAME: 'lcbp3_production',
|
||||||
|
DB_USER: 'migration_bot',
|
||||||
|
DB_PASSWORD: 'YOUR_DB_PASSWORD_HERE' // 🔴 เปลี่ยน
|
||||||
|
};
|
||||||
|
|
||||||
|
// อย่าแก้โค้ดด้านล่างนี้
|
||||||
|
$workflow.staticData = $workflow.staticData || {};
|
||||||
|
$workflow.staticData.config = CONFIG;
|
||||||
|
|
||||||
|
return [{ json: { config_loaded: true, timestamp: new Date().toISOString() }}];
|
||||||
|
```
|
||||||
|
|
||||||
|
### ขั้นตอนที่ 2: ตั้งค่า Credentials ใน n8n UI
|
||||||
|
|
||||||
|
เนื่องจาก Free Plan ไม่สามารถซ่อน Sensitive Data ได้ทั้งหมด แนะนำให้:
|
||||||
|
|
||||||
|
1. **สร้าง Dedicated User สำหรับ Migration เท่านั้น**
|
||||||
|
2. **ใช้ Token ที่มีสิทธิ์จำกัด** (เฉพาะ API ที่จำเป็น)
|
||||||
|
3. **Rotate Token ทันทีหลัง Migration เสร็จ**
|
||||||
|
|
||||||
|
**การตั้งค่า Credentials (ถ้าใช้):**
|
||||||
|
|
||||||
|
| Credential | Type | ใช้ใน Node |
|
||||||
|
|-----------|------|-----------|
|
||||||
|
| Ollama API | HTTP Request | Ollama AI Analysis |
|
||||||
|
| LCBP3 Backend | HTTP Request | Import to Backend, Fetch Categories |
|
||||||
|
| MariaDB | MySQL | ทุก Database Node |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🗄️ การเตรียม Database (เหมือนเดิม)
|
||||||
|
|
||||||
|
รัน SQL นี้บน MariaDB **ก่อน** เริ่มใช้งาน:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
-- Checkpoint
|
||||||
|
CREATE TABLE IF NOT EXISTS migration_progress (
|
||||||
|
batch_id VARCHAR(50) PRIMARY KEY,
|
||||||
|
last_processed_index INT DEFAULT 0,
|
||||||
|
status ENUM('RUNNING','COMPLETED','FAILED') DEFAULT 'RUNNING',
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Review Queue
|
||||||
|
CREATE TABLE IF NOT EXISTS migration_review_queue (
|
||||||
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
document_number VARCHAR(100) NOT NULL,
|
||||||
|
title TEXT,
|
||||||
|
original_title TEXT,
|
||||||
|
ai_suggested_category VARCHAR(50),
|
||||||
|
ai_confidence DECIMAL(4,3),
|
||||||
|
ai_issues JSON,
|
||||||
|
review_reason VARCHAR(255),
|
||||||
|
status ENUM('PENDING','APPROVED','REJECTED') DEFAULT 'PENDING',
|
||||||
|
reviewed_by VARCHAR(100),
|
||||||
|
reviewed_at TIMESTAMP NULL,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
UNIQUE KEY uq_doc_number (document_number)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Error Log
|
||||||
|
CREATE TABLE IF NOT EXISTS migration_errors (
|
||||||
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
batch_id VARCHAR(50),
|
||||||
|
document_number VARCHAR(100),
|
||||||
|
error_type ENUM('FILE_NOT_FOUND','AI_PARSE_ERROR','API_ERROR','DB_ERROR','SECURITY','UNKNOWN'),
|
||||||
|
error_message TEXT,
|
||||||
|
raw_ai_response TEXT,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
INDEX idx_batch_id (batch_id),
|
||||||
|
INDEX idx_error_type (error_type)
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Fallback State
|
||||||
|
CREATE TABLE IF NOT EXISTS migration_fallback_state (
|
||||||
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
batch_id VARCHAR(50) UNIQUE,
|
||||||
|
recent_error_count INT DEFAULT 0,
|
||||||
|
is_fallback_active BOOLEAN DEFAULT FALSE,
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
||||||
|
);
|
||||||
|
|
||||||
|
-- Idempotency
|
||||||
|
CREATE TABLE IF NOT EXISTS import_transactions (
|
||||||
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
idempotency_key VARCHAR(255) UNIQUE NOT NULL,
|
||||||
|
document_number VARCHAR(100),
|
||||||
|
batch_id VARCHAR(100),
|
||||||
|
status_code INT DEFAULT 201,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
INDEX idx_idem_key (idempotency_key)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🐳 Docker Compose สำหรับ QNAP (Free Plan)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
version: '3.8'
|
||||||
|
|
||||||
|
services:
|
||||||
|
n8n:
|
||||||
|
image: n8nio/n8n:1.78.0
|
||||||
|
container_name: n8n-lcbp3
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "5678:5678"
|
||||||
|
environment:
|
||||||
|
- TZ=Asia/Bangkok
|
||||||
|
- NODE_ENV=production
|
||||||
|
- N8N_BASIC_AUTH_ACTIVE=true
|
||||||
|
- N8N_BASIC_AUTH_USER=admin
|
||||||
|
- N8N_BASIC_AUTH_PASSWORD=YOUR_N8N_PASSWORD_HERE
|
||||||
|
- N8N_ENCRYPTION_KEY=YOUR_ENCRYPTION_KEY_HERE
|
||||||
|
volumes:
|
||||||
|
- /share/np-dms/n8n:/home/node/.n8n
|
||||||
|
- /share/np-dms/n8n/cache:/home/node/.cache
|
||||||
|
# อ่านอย่างเดียว: ไฟล์ต้นฉบับ
|
||||||
|
- /share/np-dms/staging_ai:/share/np-dms/staging_ai:ro
|
||||||
|
# เขียนได้: Logs และ CSV
|
||||||
|
- /share/np-dms/n8n/migration_logs:/share/np-dms/n8n/migration_logs:rw
|
||||||
|
networks:
|
||||||
|
- lcbp3-network
|
||||||
|
|
||||||
|
networks:
|
||||||
|
lcbp3-network:
|
||||||
|
external: true
|
||||||
|
```
|
||||||
|
|
||||||
|
> **หมายเหตุ:** Free Plan ไม่ต้องใช้ PostgreSQL สำหรับ n8n (ใช้ SQLite ได้)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔄 การทำงานของแต่ละ Node
|
||||||
|
|
||||||
|
### Node 0: Set Configuration
|
||||||
|
- เก็บค่า Config ทั้งหมดใน `$workflow.staticData.config`
|
||||||
|
- อ่านผ่าน `$workflow.staticData.config.KEY` ใน Node อื่น
|
||||||
|
|
||||||
|
### Node 1-2: Pre-flight Checks
|
||||||
|
- ตรวจสอบ Backend Health
|
||||||
|
- ดึง Categories จาก `/api/meta/categories`
|
||||||
|
- ตรวจ File Mount (Read-only)
|
||||||
|
- เก็บ Categories ใน `$workflow.staticData.systemCategories`
|
||||||
|
|
||||||
|
### Node 3: Read Checkpoint
|
||||||
|
- อ่าน `last_processed_index` จาก `migration_progress`
|
||||||
|
- ถ้าไม่มี เริ่มจาก 0
|
||||||
|
|
||||||
|
### Node 4: Process Batch
|
||||||
|
- อ่าน Excel
|
||||||
|
- Normalize UTF-8 (NFC)
|
||||||
|
- ตัด Batch ตาม `BATCH_SIZE`
|
||||||
|
|
||||||
|
### Node 5: File Validator
|
||||||
|
- Sanitize filename (replace special chars)
|
||||||
|
- Path traversal check
|
||||||
|
- ตรวจสอบไฟล์มีอยู่จริง
|
||||||
|
- **Output 2 ทาง**: Valid → AI, Error → Log
|
||||||
|
|
||||||
|
### Node 6: Build AI Prompt
|
||||||
|
- ดึง Categories จาก `staticData` (ไม่ hardcode)
|
||||||
|
- เลือก Model ตาม Fallback State
|
||||||
|
- สร้าง Prompt ตาม Template
|
||||||
|
|
||||||
|
### Node 7: Ollama AI Analysis
|
||||||
|
- เรียก `POST /api/generate`
|
||||||
|
- Timeout 30 วินาที
|
||||||
|
- Retry 3 ครั้ง (n8n built-in)
|
||||||
|
|
||||||
|
### Node 8: Parse & Validate
|
||||||
|
- Parse JSON Response
|
||||||
|
- Schema Validation (is_valid, confidence, detected_issues)
|
||||||
|
- Enum Validation (ตรวจ Category ว่าอยู่ใน List หรือไม่)
|
||||||
|
- **Output 2 ทาง**: Success → Router, Error → Fallback
|
||||||
|
|
||||||
|
### Node 9: Confidence Router
|
||||||
|
- **4 Outputs**:
|
||||||
|
1. Auto Ingest (confidence ≥ 0.85 && is_valid)
|
||||||
|
2. Review Queue (0.60 ≤ confidence < 0.85)
|
||||||
|
3. Reject Log (confidence < 0.60 หรือ is_valid = false)
|
||||||
|
4. Error Log (parse error)
|
||||||
|
|
||||||
|
### Node 10A: Auto Ingest
|
||||||
|
- POST `/api/correspondences/import`
|
||||||
|
- Header: `Idempotency-Key: {doc_num}:{batch_id}`
|
||||||
|
- บันทึก Checkpoint ทุก 10 records
|
||||||
|
|
||||||
|
### Node 10B: Review Queue
|
||||||
|
- INSERT เข้า `migration_review_queue` เท่านั้น
|
||||||
|
- ยังไม่สร้าง Correspondence
|
||||||
|
|
||||||
|
### Node 10C: Reject Log
|
||||||
|
- เขียน CSV ที่ `/share/np-dms/n8n/migration_logs/reject_log.csv`
|
||||||
|
|
||||||
|
### Node 10D: Error Log
|
||||||
|
- เขียน CSV + INSERT เข้า `migration_errors`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚨 ข้อควรระวังสำหรับ Free Plan
|
||||||
|
|
||||||
|
### 1. Security
|
||||||
|
- **อย่า Commit ไฟล์นี้เข้า Git** ถ้ามี Password/Token
|
||||||
|
- ใช้ `.gitignore` สำหรับไฟล์ JSON ที่มี Config
|
||||||
|
- Rotate Token ทันทีหลังใช้งาน
|
||||||
|
|
||||||
|
### 2. Limitations
|
||||||
|
- **Execution Timeout**: ตรวจสอบ n8n execution timeout (default 5 นาที)
|
||||||
|
- **Memory**: จำกัดที่ 2GB (ตาม Docker Compose)
|
||||||
|
- **Concurrent**: รัน Batch ต่อเนื่อง ไม่ parallel
|
||||||
|
|
||||||
|
### 3. Backup
|
||||||
|
- สำรอง SQLite database ของ n8n ที่ `/home/node/.n8n`
|
||||||
|
- สำรอง Logs ที่ `/share/np-dms/n8n/migration_logs`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ✅ Pre-Production Checklist (Free Plan)
|
||||||
|
|
||||||
|
| ลำดับ | รายการ | วิธีตรวจสอบ |
|
||||||
|
|-------|--------|-------------|
|
||||||
|
| 1 | Config ถูกต้อง | รัน Test Execution ดูผลลัพธ์ Node 0 |
|
||||||
|
| 2 | Database Connect ได้ | Test Step ใน Node Read Checkpoint |
|
||||||
|
| 3 | Ollama พร้อม | `curl http://<OLLAMA_HOST>/api/tags` |
|
||||||
|
| 4 | Backend Token ใช้ได้ | Test Step ใน Node Fetch Categories |
|
||||||
|
| 5 | File Mount RO ถูกต้อง | `docker exec n8n ls /share/np-dms/staging_ai` |
|
||||||
|
| 6 | Log Mount RW ถูกต้อง | `docker exec n8n touch /share/np-dms/n8n/migration_logs/test` |
|
||||||
|
| 7 | Categories ไม่ hardcode | ดูผลลัพธ์ Node Fetch Categories |
|
||||||
|
| 8 | Idempotency Key ถูกต้อง | ตรวจ Header ใน Node Import |
|
||||||
|
| 9 | Checkpoint บันทึก | ตรวจสอบ `migration_progress` หลังรัน |
|
||||||
|
| 10 | Error Log สร้างไฟล์ | ตรวจสอบ `error_log.csv` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 การแก้ไขปัญหาเฉพาะหน้า
|
||||||
|
|
||||||
|
### ปัญหา: Config ไม่ถูกต้อง
|
||||||
|
**แก้ไข:** แก้ที่ Node "Set Configuration" แล้ว Save → Execute Workflow ใหม่
|
||||||
|
|
||||||
|
### ปัญหา: Database Connection Error
|
||||||
|
**ตรวจสอบ:**
|
||||||
|
```javascript
|
||||||
|
// ใส่ใน Code Node ชั่วคราวเพื่อ Debug
|
||||||
|
const config = $workflow.staticData.config;
|
||||||
|
return [{ json: {
|
||||||
|
host: config.DB_HOST,
|
||||||
|
port: config.DB_PORT,
|
||||||
|
// อย่าแสดง password ใน Production!
|
||||||
|
test: 'Config loaded: ' + (config ? 'YES' : 'NO')
|
||||||
|
}}];
|
||||||
|
```
|
||||||
|
|
||||||
|
### ปัญหา: Ollama Timeout
|
||||||
|
**แก้ไข:**
|
||||||
|
- เพิ่ม `DELAY_MS` เป็น 3000 หรือ 5000
|
||||||
|
- ลด `BATCH_SIZE` เหลือ 5
|
||||||
|
- ตรวจสอบ GPU/CPU ของ Ollama Server
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 การ Monitor (Manual)
|
||||||
|
|
||||||
|
เนื่องจาก Free Plan ไม่มี Advanced Monitoring:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# ดู Progress ล่าสุด
|
||||||
|
docker exec n8n-lcbp3 sh -c "tail -5 /share/np-dms/n8n/migration_logs/reject_log.csv"
|
||||||
|
|
||||||
|
# ดู Error ล่าสุด
|
||||||
|
docker exec n8n-lcbp3 sh -c "tail -10 /share/np-dms/n8n/migration_logs/error_log.csv"
|
||||||
|
|
||||||
|
# ดู Checkpoint ใน DB
|
||||||
|
mysql -h <DB_HOST> -u migration_bot -p -e "SELECT * FROM migration_progress WHERE batch_id = 'migration_20260226'"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**เอกสารฉบับนี้จัดทำขึ้นสำหรับ n8n Free Plan (Self-hosted)**
|
||||||
|
**Version:** 1.8.0-free | **Last Updated:** 2026-03-03
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📥 วิธี Import Workflow
|
||||||
|
|
||||||
|
1. บันทึก JSON ด้านบนเป็นไฟล์ `lcbp3-migration-free.json`
|
||||||
|
2. เข้า n8n UI → **Workflows** → **Import from File**
|
||||||
|
3. เลือกไฟล์ `lcbp3-migration-free.json`
|
||||||
|
4. เปิด Workflow → แก้ไข Node **"Set Configuration"** ตามข้อมูลจริง
|
||||||
|
5. ตั้งค่า **Schedule Trigger** หรือเปลี่ยนเป็น **Manual Trigger** สำหรับทดสอบ
|
||||||
|
6. **Save** → **Execute Workflow** เพื่อทดสอบ
|
||||||
|
|
||||||
|
ต้องการให้ช่วยปรับแต่งเพิ่มเติมหรือไม่ครับ? เช่น:
|
||||||
|
- เพิ่ม Node สำหรับส่ง Email แจ้งเตือน
|
||||||
|
- เพิ่ม Rollback Workflow
|
||||||
|
- ปรับ Batch Size อัตโนมัติตาม Error Rate
|
||||||
File diff suppressed because it is too large
Load Diff
109
specs/03-Data-and-Storage/lcbp3-v1.8.0-migration.sql
Normal file
109
specs/03-Data-and-Storage/lcbp3-v1.8.0-migration.sql
Normal file
@@ -0,0 +1,109 @@
|
|||||||
|
-- ==========================================================
|
||||||
|
-- DMS v1.8.0 Migration Support Tables
|
||||||
|
-- ไฟล์นี้แยกจาก schema หลัก (lcbp3-v1.8.0-schema-01/02/03)
|
||||||
|
-- ใช้สำหรับ n8n Migration Workflow เท่านั้น
|
||||||
|
-- ลบได้ทั้งหมดหลัง Migration เสร็จสิ้น
|
||||||
|
-- ==========================================================
|
||||||
|
-- รันบน MariaDB **ก่อน** เริ่ม n8n Workflow
|
||||||
|
SET NAMES utf8mb4;
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- 1. Checkpoint — ติดตามความคืบหน้าของ Batch
|
||||||
|
-- =====================================================
|
||||||
|
CREATE TABLE IF NOT EXISTS migration_progress (
|
||||||
|
batch_id VARCHAR(50) PRIMARY KEY,
|
||||||
|
last_processed_index INT DEFAULT 0,
|
||||||
|
STATUS ENUM('RUNNING', 'COMPLETED', 'FAILED') DEFAULT 'RUNNING',
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
||||||
|
) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4 COLLATE = utf8mb4_general_ci COMMENT = 'Migration: ติดตามความคืบหน้า Batch (ลบได้หลัง Migration เสร็จ)';
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- 2. Review Queue — รายการที่ต้องตรวจสอบโดยคน
|
||||||
|
-- =====================================================
|
||||||
|
CREATE TABLE IF NOT EXISTS migration_review_queue (
|
||||||
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
document_number VARCHAR(100) NOT NULL,
|
||||||
|
title TEXT,
|
||||||
|
original_title TEXT,
|
||||||
|
ai_suggested_category VARCHAR(50),
|
||||||
|
ai_confidence DECIMAL(4, 3),
|
||||||
|
ai_issues JSON,
|
||||||
|
review_reason VARCHAR(255),
|
||||||
|
STATUS ENUM('PENDING', 'APPROVED', 'REJECTED') DEFAULT 'PENDING',
|
||||||
|
reviewed_by VARCHAR(100),
|
||||||
|
reviewed_at TIMESTAMP NULL,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
UNIQUE KEY uq_doc_number (document_number)
|
||||||
|
) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4 COLLATE = utf8mb4_general_ci COMMENT = 'Migration: Review Queue ชั่วคราว (ลบได้หลัง Migration เสร็จ)';
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- 3. Error Log — บันทึก Error ระหว่าง Migration
|
||||||
|
-- =====================================================
|
||||||
|
CREATE TABLE IF NOT EXISTS migration_errors (
|
||||||
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
batch_id VARCHAR(50),
|
||||||
|
document_number VARCHAR(100),
|
||||||
|
error_type ENUM(
|
||||||
|
'FILE_NOT_FOUND',
|
||||||
|
'AI_PARSE_ERROR',
|
||||||
|
'API_ERROR',
|
||||||
|
'DB_ERROR',
|
||||||
|
'SECURITY',
|
||||||
|
'UNKNOWN'
|
||||||
|
),
|
||||||
|
error_message TEXT,
|
||||||
|
raw_ai_response TEXT,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
INDEX idx_batch_id (batch_id),
|
||||||
|
INDEX idx_error_type (error_type)
|
||||||
|
) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4 COLLATE = utf8mb4_general_ci COMMENT = 'Migration: Error Log (ลบได้หลัง Migration เสร็จ)';
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- 4. Fallback State — สถานะ AI Model Fallback
|
||||||
|
-- =====================================================
|
||||||
|
CREATE TABLE IF NOT EXISTS migration_fallback_state (
|
||||||
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
batch_id VARCHAR(50) UNIQUE,
|
||||||
|
recent_error_count INT DEFAULT 0,
|
||||||
|
is_fallback_active BOOLEAN DEFAULT FALSE,
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
||||||
|
) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4 COLLATE = utf8mb4_general_ci COMMENT = 'Migration: Fallback Model State (ลบได้หลัง Migration เสร็จ)';
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- 5. Idempotency — ป้องกัน Import ซ้ำ
|
||||||
|
-- =====================================================
|
||||||
|
CREATE TABLE IF NOT EXISTS import_transactions (
|
||||||
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
idempotency_key VARCHAR(255) UNIQUE NOT NULL,
|
||||||
|
document_number VARCHAR(100),
|
||||||
|
batch_id VARCHAR(100),
|
||||||
|
status_code INT DEFAULT 201,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
INDEX idx_idem_key (idempotency_key)
|
||||||
|
) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4 COLLATE = utf8mb4_general_ci COMMENT = 'Migration: Idempotency Tracking (ลบได้หลัง Migration เสร็จ)';
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- 6. Daily Summary — สรุปผลรายวัน
|
||||||
|
-- =====================================================
|
||||||
|
CREATE TABLE IF NOT EXISTS migration_daily_summary (
|
||||||
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
batch_id VARCHAR(50),
|
||||||
|
summary_date DATE,
|
||||||
|
total_processed INT DEFAULT 0,
|
||||||
|
auto_ingested INT DEFAULT 0,
|
||||||
|
sent_to_review INT DEFAULT 0,
|
||||||
|
rejected INT DEFAULT 0,
|
||||||
|
errors INT DEFAULT 0,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
UNIQUE KEY uq_batch_date (batch_id, summary_date)
|
||||||
|
) ENGINE = InnoDB DEFAULT CHARSET = utf8mb4 COLLATE = utf8mb4_general_ci COMMENT = 'Migration: Daily Summary (ลบได้หลัง Migration เสร็จ)';
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- Cleanup Script (รันหลัง Migration เสร็จสิ้นทั้งหมด)
|
||||||
|
-- =====================================================
|
||||||
|
-- DROP TABLE IF EXISTS migration_daily_summary;
|
||||||
|
-- DROP TABLE IF EXISTS import_transactions;
|
||||||
|
-- DROP TABLE IF EXISTS migration_fallback_state;
|
||||||
|
-- DROP TABLE IF EXISTS migration_errors;
|
||||||
|
-- DROP TABLE IF EXISTS migration_review_queue;
|
||||||
|
-- DROP TABLE IF EXISTS migration_progress;
|
||||||
215
specs/03-Data-and-Storage/lcbp3-v1.8.0-schema-01-drop.sql
Normal file
215
specs/03-Data-and-Storage/lcbp3-v1.8.0-schema-01-drop.sql
Normal file
@@ -0,0 +1,215 @@
|
|||||||
|
-- ==========================================================
|
||||||
|
-- DMS v1.8.0 Schema Part 1/3: DROP Statements
|
||||||
|
-- รันไฟล์นี้ก่อน เพื่อล้างตารางเดิมทั้งหมด
|
||||||
|
-- ==========================================================
|
||||||
|
-- ==========================================================
|
||||||
|
-- DMS v1.8.0 Document Management System Database
|
||||||
|
-- Deploy Script Schema
|
||||||
|
-- Server: Container Station on QNAP TS-473A
|
||||||
|
-- Database service: MariaDB 11.8
|
||||||
|
-- database web ui: phpmyadmin 5-apache
|
||||||
|
-- database development ui: DBeaver
|
||||||
|
-- backend service: NestJS
|
||||||
|
-- frontend service: next.js
|
||||||
|
-- reverse proxy: jc21/nginx-proxy-manager:latest
|
||||||
|
-- cron service: n8n
|
||||||
|
-- ==========================================================
|
||||||
|
-- [v1.8.0 UPDATE] Prepare migration
|
||||||
|
-- Update: Upgraded from v1.7.0
|
||||||
|
-- Last Updated: 2026-02-27
|
||||||
|
-- Major Changes:
|
||||||
|
-- 1. ปรับปรุง:
|
||||||
|
-- 1.1 TABLE correspondences
|
||||||
|
-- - INDEX idx_doc_number (correspondence_number),
|
||||||
|
-- - INDEX idx_deleted_at (deleted_at),
|
||||||
|
-- - INDEX idx_created_by (created_by),
|
||||||
|
-- ==========================================================
|
||||||
|
SET NAMES utf8mb4;
|
||||||
|
|
||||||
|
SET time_zone = '+07:00';
|
||||||
|
|
||||||
|
-- ปิดการตรวจสอบ Foreign Key ชั่วคราวเพื่อให้สามารถลบตารางได้ทั้งหมด
|
||||||
|
SET FOREIGN_KEY_CHECKS = 0;
|
||||||
|
|
||||||
|
DROP VIEW IF EXISTS v_document_statistics;
|
||||||
|
|
||||||
|
DROP VIEW IF EXISTS v_documents_with_attachments;
|
||||||
|
|
||||||
|
DROP VIEW IF EXISTS v_user_all_permissions;
|
||||||
|
|
||||||
|
DROP VIEW IF EXISTS v_audit_log_details;
|
||||||
|
|
||||||
|
DROP VIEW IF EXISTS v_user_tasks;
|
||||||
|
|
||||||
|
DROP VIEW IF EXISTS v_contract_parties_all;
|
||||||
|
|
||||||
|
DROP VIEW IF EXISTS v_current_rfas;
|
||||||
|
|
||||||
|
DROP VIEW IF EXISTS v_current_correspondences;
|
||||||
|
|
||||||
|
-- DROP PROCEDURE IF EXISTS sp_get_next_document_number;
|
||||||
|
-- 🗑️ DROP TABLE SCRIPT: LCBP3-DMS v1.4.2
|
||||||
|
-- คำเตือน: ข้อมูลทั้งหมดจะหายไป กรุณา Backup ก่อนรันบน Production
|
||||||
|
SET FOREIGN_KEY_CHECKS = 0;
|
||||||
|
|
||||||
|
-- ============================================================
|
||||||
|
-- ส่วนที่ 1: ตาราง System, Logs & Preferences (ตารางปลายทาง/ส่วนเสริม)
|
||||||
|
-- ============================================================
|
||||||
|
DROP TABLE IF EXISTS backup_logs;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS search_indices;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS notifications;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS audit_logs;
|
||||||
|
|
||||||
|
-- [NEW v1.4.2] ตารางการตั้งค่าส่วนตัวของผู้ใช้ (FK -> users)
|
||||||
|
DROP TABLE IF EXISTS user_preferences;
|
||||||
|
|
||||||
|
-- [NEW v1.4.2] ตารางเก็บ Schema สำหรับ Validate JSON (Stand-alone)
|
||||||
|
DROP TABLE IF EXISTS json_schemas;
|
||||||
|
|
||||||
|
-- [v1.5.1 NEW] ตาราง Audit และ Error Log สำหรับ Document Numbering
|
||||||
|
DROP TABLE IF EXISTS document_number_errors;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS document_number_audit;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS document_number_reservations;
|
||||||
|
|
||||||
|
-- ============================================================
|
||||||
|
-- ส่วนที่ 2: ตาราง Junction (เชื่อมโยงข้อมูล M:N)
|
||||||
|
-- ============================================================
|
||||||
|
DROP TABLE IF EXISTS correspondence_tags;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS asbuilt_revision_shop_revisions_refs;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS shop_drawing_revision_contract_refs;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS contract_drawing_subcat_cat_maps;
|
||||||
|
|
||||||
|
-- ============================================================
|
||||||
|
-- ส่วนที่ 3: ตารางไฟล์แนบและการเชื่อมโยง (Attachments)
|
||||||
|
-- ============================================================
|
||||||
|
DROP TABLE IF EXISTS contract_drawing_attachments;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS circulation_attachments;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS shop_drawing_revision_attachments;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS asbuilt_drawing_revision_attachments;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS correspondence_attachments;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS attachments;
|
||||||
|
|
||||||
|
-- ตารางหลักเก็บ path ไฟล์
|
||||||
|
-- ============================================================
|
||||||
|
-- ส่วนที่ 4: ตาราง Workflow & Routing (Process Logic)
|
||||||
|
-- ============================================================
|
||||||
|
-- Correspondence Workflow
|
||||||
|
-- ============================================================
|
||||||
|
-- ส่วนที่ 5: ตาราง Mapping สิทธิ์และโครงสร้าง (Access Control)
|
||||||
|
-- ============================================================
|
||||||
|
DROP TABLE IF EXISTS role_permissions;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS user_assignments;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS contract_organizations;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS project_organizations;
|
||||||
|
|
||||||
|
-- ============================================================
|
||||||
|
-- ส่วนที่ 6: ตารางรายละเอียดของเอกสาร (Revisions & Items)
|
||||||
|
-- ============================================================
|
||||||
|
DROP TABLE IF EXISTS transmittal_items;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS shop_drawing_revisions;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS asbuilt_drawing_revisions;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS rfa_items;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS rfa_revisions;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS correspondence_references;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS correspondence_recipients;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS correspondence_revisions;
|
||||||
|
|
||||||
|
-- [Modified v1.4.2] มี Virtual Columns
|
||||||
|
-- ============================================================
|
||||||
|
-- ส่วนที่ 7: ตารางเอกสารหลัก (Core Documents)
|
||||||
|
-- ============================================================
|
||||||
|
DROP TABLE IF EXISTS circulations;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS transmittals;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS contract_drawings;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS shop_drawings;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS asbuilt_drawings;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS rfas;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS correspondences;
|
||||||
|
|
||||||
|
-- ============================================================
|
||||||
|
-- ส่วนที่ 8: ตารางหมวดหมู่และข้อมูลหลัก (Master Data)
|
||||||
|
-- ============================================================
|
||||||
|
-- [NEW 6B] ลบตารางใหม่ที่เพิ่มเข้ามาเพื่อป้องกัน Error เวลา Re-deploy
|
||||||
|
DROP TABLE IF EXISTS correspondence_sub_types;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS disciplines;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS shop_drawing_sub_categories;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS shop_drawing_main_categories;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS contract_drawing_sub_cats;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS contract_drawing_cats;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS contract_drawing_volumes;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS circulation_status_codes;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS rfa_approve_codes;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS rfa_status_codes;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS rfa_types;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS correspondence_status;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS correspondence_types;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS document_number_counters;
|
||||||
|
|
||||||
|
-- [Modified v1.4.2] มี version column
|
||||||
|
DROP TABLE IF EXISTS document_number_formats;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS tags;
|
||||||
|
|
||||||
|
-- ============================================================
|
||||||
|
-- ส่วนที่ 9: ตารางผู้ใช้ บทบาท และโครงสร้างรากฐาน (Root Tables)
|
||||||
|
-- ============================================================
|
||||||
|
DROP TABLE IF EXISTS organization_roles;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS roles;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS permissions;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS contracts;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS projects;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS refresh_tokens;
|
||||||
|
|
||||||
|
DROP TABLE IF EXISTS users;
|
||||||
|
|
||||||
|
-- Referenced by user_preferences, audit_logs, etc.
|
||||||
|
DROP TABLE IF EXISTS organizations;
|
||||||
|
|
||||||
|
-- Referenced by users, projects, etc.
|
||||||
1344
specs/03-Data-and-Storage/lcbp3-v1.8.0-schema-02-tables.sql
Normal file
1344
specs/03-Data-and-Storage/lcbp3-v1.8.0-schema-02-tables.sql
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,516 @@
|
|||||||
|
-- ==========================================================
|
||||||
|
-- DMS v1.8.0 Schema Part 3/3: Views, Indexes, Partitioning
|
||||||
|
-- รัน: หลังจาก 02-schema-tables.sql เสร็จ
|
||||||
|
-- ==========================================================
|
||||||
|
SET NAMES utf8mb4;
|
||||||
|
SET time_zone = '+07:00';
|
||||||
|
|
||||||
|
-- ============================================================
|
||||||
|
-- 5. PARTITIONING PREPARATION (Advance - Optional)
|
||||||
|
-- ============================================================
|
||||||
|
-- หมายเหตุ: การทำ Partitioning บนตารางที่มีอยู่แล้ว (audit_logs, notifications)
|
||||||
|
-- มักจะต้อง Drop Primary Key เดิม แล้วสร้างใหม่โดยรวม Partition Key (created_at) เข้าไป
|
||||||
|
-- ขั้นตอนนี้ควรทำแยกต่างหากเมื่อระบบเริ่มมีข้อมูลเยอะ หรือทำใน Maintenance Window
|
||||||
|
--
|
||||||
|
-- ตัวอย่าง SQL สำหรับ Audit Logs (Reference Only):
|
||||||
|
-- ALTER TABLE audit_logs DROP PRIMARY KEY, ADD PRIMARY KEY (audit_id, created_at);
|
||||||
|
-- ALTER TABLE audit_logs PARTITION BY RANGE (YEAR(created_at)) (
|
||||||
|
-- PARTITION p2024 VALUES LESS THAN (2025),
|
||||||
|
-- PARTITION p2025 VALUES LESS THAN (2026),
|
||||||
|
-- PARTITION p_future VALUES LESS THAN MAXVALUE
|
||||||
|
-- );
|
||||||
|
-- =====================================================
|
||||||
|
-- CREATE INDEXES
|
||||||
|
-- =====================================================
|
||||||
|
-- Indexes for correspondences
|
||||||
|
CREATE INDEX idx_corr_type ON correspondences(correspondence_type_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_corr_project ON correspondences(project_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_rfa_rev_v_drawing_count ON rfa_revisions (v_ref_drawing_count);
|
||||||
|
|
||||||
|
-- Indexes for document_number_formats
|
||||||
|
CREATE INDEX idx_document_number_formats_project ON document_number_formats (project_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_document_number_formats_type ON document_number_formats (correspondence_type_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_document_number_formats_project_type ON document_number_formats (project_id, correspondence_type_id);
|
||||||
|
|
||||||
|
-- Indexes for document_number_counters
|
||||||
|
CREATE INDEX idx_document_number_counters_project ON document_number_counters (project_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_document_number_counters_org ON document_number_counters (originator_organization_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_document_number_counters_type ON document_number_counters (correspondence_type_id);
|
||||||
|
|
||||||
|
-- Indexes for tags
|
||||||
|
CREATE INDEX idx_tags_name ON tags (tag_name);
|
||||||
|
|
||||||
|
CREATE INDEX idx_tags_created_at ON tags (created_at);
|
||||||
|
|
||||||
|
-- Indexes for correspondence_tags
|
||||||
|
CREATE INDEX idx_correspondence_tags_correspondence ON correspondence_tags (correspondence_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_correspondence_tags_tag ON correspondence_tags (tag_id);
|
||||||
|
|
||||||
|
-- Indexes for audit_logs
|
||||||
|
CREATE INDEX idx_audit_logs_user ON audit_logs (user_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_audit_logs_action ON audit_logs (ACTION);
|
||||||
|
|
||||||
|
CREATE INDEX idx_audit_logs_entity ON audit_logs (entity_type, entity_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_audit_logs_created_at ON audit_logs (created_at);
|
||||||
|
|
||||||
|
CREATE INDEX idx_audit_logs_ip ON audit_logs (ip_address);
|
||||||
|
|
||||||
|
-- Indexes for notifications
|
||||||
|
CREATE INDEX idx_notifications_user ON notifications (user_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_notifications_type ON notifications (notification_type);
|
||||||
|
|
||||||
|
CREATE INDEX idx_notifications_read ON notifications (is_read);
|
||||||
|
|
||||||
|
CREATE INDEX idx_notifications_entity ON notifications (entity_type, entity_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_notifications_created_at ON notifications (created_at);
|
||||||
|
|
||||||
|
-- Indexes for search_indices
|
||||||
|
CREATE INDEX idx_search_indices_entity ON search_indices (entity_type, entity_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_search_indices_indexed_at ON search_indices (indexed_at);
|
||||||
|
|
||||||
|
-- Indexes for backup_logs
|
||||||
|
CREATE INDEX idx_backup_logs_type ON backup_logs (backup_type);
|
||||||
|
|
||||||
|
CREATE INDEX idx_backup_logs_status ON backup_logs (STATUS);
|
||||||
|
|
||||||
|
CREATE INDEX idx_backup_logs_started_at ON backup_logs (started_at);
|
||||||
|
|
||||||
|
CREATE INDEX idx_backup_logs_completed_at ON backup_logs (completed_at);
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- Additional Composite Indexes for Performance
|
||||||
|
-- =====================================================
|
||||||
|
-- Composite index for document_number_counters for faster lookups
|
||||||
|
-- Composite index for notifications for user-specific queries
|
||||||
|
CREATE INDEX idx_notifications_user_unread ON notifications (user_id, is_read, created_at);
|
||||||
|
|
||||||
|
-- Composite index for audit_logs for reporting
|
||||||
|
CREATE INDEX idx_audit_logs_reporting ON audit_logs (created_at, entity_type, ACTION);
|
||||||
|
|
||||||
|
-- Composite index for search_indices for entity-based queries
|
||||||
|
CREATE INDEX idx_search_entities ON search_indices (entity_type, entity_id, indexed_at);
|
||||||
|
|
||||||
|
-- สร้าง Index สำหรับ Cleanup Job
|
||||||
|
CREATE INDEX idx_attachments_temp_cleanup ON attachments (is_temporary, expires_at);
|
||||||
|
|
||||||
|
CREATE INDEX idx_attachments_temp_id ON attachments (temp_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_audit_request_id ON audit_logs (request_id);
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- SQL Script for LCBP3-DMS (V1.4.0) - MariaDB
|
||||||
|
-- Generated from Data Dictionary
|
||||||
|
-- =====================================================
|
||||||
|
-- =====================================================
|
||||||
|
-- 11. 📊 Views & Procedures (วิว และ โปรซีเดอร์)
|
||||||
|
-- =====================================================
|
||||||
|
-- View แสดง Revision "ปัจจุบัน" ของ correspondences ทั้งหมด (ที่ไม่ใช่ RFA)
|
||||||
|
CREATE VIEW v_current_correspondences AS
|
||||||
|
SELECT c.id AS correspondence_id,
|
||||||
|
c.correspondence_number,
|
||||||
|
c.correspondence_type_id,
|
||||||
|
ct.type_code AS correspondence_type_code,
|
||||||
|
ct.type_name AS correspondence_type_name,
|
||||||
|
c.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name,
|
||||||
|
c.originator_id,
|
||||||
|
org.organization_code AS originator_code,
|
||||||
|
org.organization_name AS originator_name,
|
||||||
|
cr.id AS revision_id,
|
||||||
|
cr.revision_number,
|
||||||
|
cr.revision_label,
|
||||||
|
cr.subject,
|
||||||
|
cr.document_date,
|
||||||
|
cr.issued_date,
|
||||||
|
cr.received_date,
|
||||||
|
cr.due_date,
|
||||||
|
cr.correspondence_status_id,
|
||||||
|
cs.status_code,
|
||||||
|
cs.status_name,
|
||||||
|
cr.created_by,
|
||||||
|
u.username AS created_by_username,
|
||||||
|
cr.created_at AS revision_created_at
|
||||||
|
FROM correspondences c
|
||||||
|
INNER JOIN correspondence_types ct ON c.correspondence_type_id = ct.id
|
||||||
|
INNER JOIN projects p ON c.project_id = p.id
|
||||||
|
LEFT JOIN organizations org ON c.originator_id = org.id
|
||||||
|
INNER JOIN correspondence_revisions cr ON c.id = cr.correspondence_id
|
||||||
|
INNER JOIN correspondence_status cs ON cr.correspondence_status_id = cs.id
|
||||||
|
LEFT JOIN users u ON cr.created_by = u.user_id
|
||||||
|
WHERE cr.is_current = TRUE
|
||||||
|
AND c.correspondence_type_id NOT IN(
|
||||||
|
SELECT id
|
||||||
|
FROM correspondence_types
|
||||||
|
WHERE type_code = 'RFA'
|
||||||
|
)
|
||||||
|
AND c.deleted_at IS NULL;
|
||||||
|
|
||||||
|
-- View แสดง Revision "ปัจจุบัน" ของ rfa_revisions ทั้งหมด
|
||||||
|
CREATE VIEW v_current_rfas AS
|
||||||
|
SELECT r.id AS rfa_id,
|
||||||
|
r.rfa_type_id,
|
||||||
|
rt.type_code AS rfa_type_code,
|
||||||
|
rt.type_name_th AS rfa_type_name_th,
|
||||||
|
rt.type_name_en AS rfa_type_name_en,
|
||||||
|
c.correspondence_number,
|
||||||
|
c.discipline_id,
|
||||||
|
-- ✅ ดึงจาก Correspondences
|
||||||
|
d.discipline_code,
|
||||||
|
-- ✅ Join เพิ่มเพื่อแสดง code
|
||||||
|
c.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name,
|
||||||
|
c.originator_id,
|
||||||
|
org.organization_name AS originator_name,
|
||||||
|
rr.id AS revision_id,
|
||||||
|
rr.revision_number,
|
||||||
|
rr.revision_label,
|
||||||
|
rr.subject,
|
||||||
|
rr.document_date,
|
||||||
|
rr.issued_date,
|
||||||
|
rr.received_date,
|
||||||
|
rr.approved_date,
|
||||||
|
rr.rfa_status_code_id,
|
||||||
|
rsc.status_code AS rfa_status_code,
|
||||||
|
rsc.status_name AS rfa_status_name,
|
||||||
|
rr.rfa_approve_code_id,
|
||||||
|
rac.approve_code AS rfa_approve_code,
|
||||||
|
rac.approve_name AS rfa_approve_name,
|
||||||
|
rr.created_by,
|
||||||
|
u.username AS created_by_username,
|
||||||
|
rr.created_at AS revision_created_at
|
||||||
|
FROM rfas r
|
||||||
|
INNER JOIN rfa_types rt ON r.rfa_type_id = rt.id
|
||||||
|
INNER JOIN rfa_revisions rr ON r.id = rr.rfa_id -- RFA uses shared primary key with correspondences (1:1)
|
||||||
|
INNER JOIN correspondences c ON r.id = c.id -- [FIX 1] เพิ่มการ Join ตาราง disciplines
|
||||||
|
LEFT JOIN disciplines d ON c.discipline_id = d.id
|
||||||
|
INNER JOIN projects p ON c.project_id = p.id
|
||||||
|
INNER JOIN organizations org ON c.originator_id = org.id
|
||||||
|
INNER JOIN rfa_status_codes rsc ON rr.rfa_status_code_id = rsc.id
|
||||||
|
LEFT JOIN rfa_approve_codes rac ON rr.rfa_approve_code_id = rac.id
|
||||||
|
LEFT JOIN users u ON rr.created_by = u.user_id
|
||||||
|
WHERE rr.is_current = TRUE
|
||||||
|
AND r.deleted_at IS NULL
|
||||||
|
AND c.deleted_at IS NULL;
|
||||||
|
|
||||||
|
-- View แสดงความสัมพันธ์ทั้งหมดระหว่าง Contract, Project, และ Organization
|
||||||
|
CREATE VIEW v_contract_parties_all AS
|
||||||
|
SELECT c.id AS contract_id,
|
||||||
|
c.contract_code,
|
||||||
|
c.contract_name,
|
||||||
|
p.id AS project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name,
|
||||||
|
o.id AS organization_id,
|
||||||
|
o.organization_code,
|
||||||
|
o.organization_name,
|
||||||
|
co.role_in_contract
|
||||||
|
FROM contracts c
|
||||||
|
INNER JOIN projects p ON c.project_id = p.id
|
||||||
|
INNER JOIN contract_organizations co ON c.id = co.contract_id
|
||||||
|
INNER JOIN organizations o ON co.organization_id = o.id
|
||||||
|
WHERE c.is_active = TRUE;
|
||||||
|
|
||||||
|
-- ============================================================
|
||||||
|
-- View: v_user_tasks (Unified Workflow Engine Edition)
|
||||||
|
-- ============================================================
|
||||||
|
-- หน้าที่: รวมรายการงานที่ยังค้างอยู่ (Status = ACTIVE) จากทุกระบบ (RFA, Circulation, Correspondence)
|
||||||
|
-- เพื่อนำไปแสดงในหน้า Dashboard "My Tasks"
|
||||||
|
-- ============================================================
|
||||||
|
CREATE OR REPLACE VIEW v_user_tasks AS
|
||||||
|
SELECT -- 1. Workflow Instance Info
|
||||||
|
wi.id AS instance_id,
|
||||||
|
wd.workflow_code,
|
||||||
|
wi.current_state,
|
||||||
|
wi.status AS workflow_status,
|
||||||
|
wi.created_at AS assigned_at,
|
||||||
|
-- 2. Entity Info (Polymorphic Identity)
|
||||||
|
wi.entity_type,
|
||||||
|
wi.entity_id,
|
||||||
|
-- 3. Normalized Document Info (ดึงข้อมูลจริงจากตารางลูกตามประเภท)
|
||||||
|
-- ใช้ CASE WHEN เพื่อรวมคอลัมน์ที่ชื่อต่างกันให้เป็นชื่อกลาง (document_number, subject)
|
||||||
|
CASE
|
||||||
|
WHEN wi.entity_type = 'rfa_revision' THEN rfa_corr.correspondence_number
|
||||||
|
WHEN wi.entity_type = 'circulation' THEN circ.circulation_no
|
||||||
|
WHEN wi.entity_type = 'correspondence_revision' THEN corr_corr.correspondence_number
|
||||||
|
ELSE 'N/A'
|
||||||
|
END AS document_number,
|
||||||
|
CASE
|
||||||
|
WHEN wi.entity_type = 'rfa_revision' THEN rfa_rev.subject
|
||||||
|
WHEN wi.entity_type = 'circulation' THEN circ.circulation_subject
|
||||||
|
WHEN wi.entity_type = 'correspondence_revision' THEN corr_rev.subject
|
||||||
|
ELSE 'Unknown Document'
|
||||||
|
END AS subject,
|
||||||
|
-- 4. Context Info (สำหรับ Filter สิทธิ์การมองเห็นที่ Backend)
|
||||||
|
-- ดึงเป็น JSON String เพื่อให้ Backend ไป Parse หรือใช้ JSON_CONTAINS
|
||||||
|
JSON_UNQUOTE(JSON_EXTRACT(wi.context, '$.ownerId')) AS owner_id,
|
||||||
|
JSON_EXTRACT(wi.context, '$.assigneeIds') AS assignee_ids_json
|
||||||
|
FROM workflow_instances wi
|
||||||
|
JOIN workflow_definitions wd ON wi.definition_id = wd.id -- 5. Joins for RFA (ซับซ้อนหน่อยเพราะ RFA ผูกกับ Correspondence อีกที)
|
||||||
|
LEFT JOIN rfa_revisions rfa_rev ON wi.entity_type = 'rfa_revision'
|
||||||
|
AND wi.entity_id = CAST(rfa_rev.id AS CHAR)
|
||||||
|
LEFT JOIN correspondences rfa_corr ON rfa_rev.id = rfa_corr.id -- 6. Joins for Circulation
|
||||||
|
LEFT JOIN circulations circ ON wi.entity_type = 'circulation'
|
||||||
|
AND wi.entity_id = CAST(circ.id AS CHAR) -- 7. Joins for Correspondence
|
||||||
|
LEFT JOIN correspondence_revisions corr_rev ON wi.entity_type = 'correspondence_revision'
|
||||||
|
AND wi.entity_id = CAST(corr_rev.id AS CHAR)
|
||||||
|
LEFT JOIN correspondences corr_corr ON corr_rev.correspondence_id = corr_corr.id -- 8. Filter เฉพาะงานที่ยัง Active อยู่
|
||||||
|
WHERE wi.status = 'ACTIVE';
|
||||||
|
|
||||||
|
-- View แสดง audit_logs พร้อมข้อมูล username และ email ของผู้กระทำ
|
||||||
|
CREATE VIEW v_audit_log_details AS
|
||||||
|
SELECT al.audit_id,
|
||||||
|
al.user_id,
|
||||||
|
u.username,
|
||||||
|
u.email,
|
||||||
|
u.first_name,
|
||||||
|
u.last_name,
|
||||||
|
al.action,
|
||||||
|
al.entity_type,
|
||||||
|
al.entity_id,
|
||||||
|
al.details_json,
|
||||||
|
al.ip_address,
|
||||||
|
al.user_agent,
|
||||||
|
al.created_at
|
||||||
|
FROM audit_logs al
|
||||||
|
LEFT JOIN users u ON al.user_id = u.user_id;
|
||||||
|
|
||||||
|
-- View รวมสิทธิ์ทั้งหมด (Global + Project) ของผู้ใช้ทุกคน
|
||||||
|
CREATE VIEW v_user_all_permissions AS -- Global Permissions
|
||||||
|
SELECT ua.user_id,
|
||||||
|
ua.role_id,
|
||||||
|
r.role_name,
|
||||||
|
rp.permission_id,
|
||||||
|
p.permission_name,
|
||||||
|
p.module,
|
||||||
|
p.scope_level,
|
||||||
|
ua.organization_id,
|
||||||
|
NULL AS project_id,
|
||||||
|
NULL AS contract_id,
|
||||||
|
'GLOBAL' AS permission_scope
|
||||||
|
FROM user_assignments ua
|
||||||
|
INNER JOIN roles r ON ua.role_id = r.role_id
|
||||||
|
INNER JOIN role_permissions rp ON ua.role_id = rp.role_id
|
||||||
|
INNER JOIN permissions p ON rp.permission_id = p.permission_id -- Global scope
|
||||||
|
WHERE p.is_active = 1
|
||||||
|
AND ua.organization_id IS NULL
|
||||||
|
AND ua.project_id IS NULL
|
||||||
|
AND ua.contract_id IS NULL
|
||||||
|
UNION ALL
|
||||||
|
-- Organization-specific Permissions
|
||||||
|
SELECT ua.user_id,
|
||||||
|
ua.role_id,
|
||||||
|
r.role_name,
|
||||||
|
rp.permission_id,
|
||||||
|
p.permission_name,
|
||||||
|
p.module,
|
||||||
|
p.scope_level,
|
||||||
|
ua.organization_id,
|
||||||
|
NULL AS project_id,
|
||||||
|
NULL AS contract_id,
|
||||||
|
'ORGANIZATION' AS permission_scope
|
||||||
|
FROM user_assignments ua
|
||||||
|
INNER JOIN roles r ON ua.role_id = r.role_id
|
||||||
|
INNER JOIN role_permissions rp ON ua.role_id = rp.role_id
|
||||||
|
INNER JOIN permissions p ON rp.permission_id = p.permission_id -- Organization scope
|
||||||
|
WHERE p.is_active = 1
|
||||||
|
AND ua.organization_id IS NOT NULL
|
||||||
|
AND ua.project_id IS NULL
|
||||||
|
AND ua.contract_id IS NULL
|
||||||
|
UNION ALL
|
||||||
|
-- Project-specific Permissions
|
||||||
|
SELECT ua.user_id,
|
||||||
|
ua.role_id,
|
||||||
|
r.role_name,
|
||||||
|
rp.permission_id,
|
||||||
|
p.permission_name,
|
||||||
|
p.module,
|
||||||
|
p.scope_level,
|
||||||
|
ua.organization_id,
|
||||||
|
ua.project_id,
|
||||||
|
NULL AS contract_id,
|
||||||
|
'PROJECT' AS permission_scope
|
||||||
|
FROM user_assignments ua
|
||||||
|
INNER JOIN roles r ON ua.role_id = r.role_id
|
||||||
|
INNER JOIN role_permissions rp ON ua.role_id = rp.role_id
|
||||||
|
INNER JOIN permissions p ON rp.permission_id = p.permission_id -- Project scope
|
||||||
|
WHERE p.is_active = 1
|
||||||
|
AND ua.project_id IS NOT NULL
|
||||||
|
AND ua.contract_id IS NULL
|
||||||
|
UNION ALL
|
||||||
|
-- Contract-specific Permissions
|
||||||
|
SELECT ua.user_id,
|
||||||
|
ua.role_id,
|
||||||
|
r.role_name,
|
||||||
|
rp.permission_id,
|
||||||
|
p.permission_name,
|
||||||
|
p.module,
|
||||||
|
p.scope_level,
|
||||||
|
ua.organization_id,
|
||||||
|
ua.project_id,
|
||||||
|
ua.contract_id,
|
||||||
|
'CONTRACT' AS permission_scope
|
||||||
|
FROM user_assignments ua
|
||||||
|
INNER JOIN roles r ON ua.role_id = r.role_id
|
||||||
|
INNER JOIN role_permissions rp ON ua.role_id = rp.role_id
|
||||||
|
INNER JOIN permissions p ON rp.permission_id = p.permission_id -- Contract scope
|
||||||
|
WHERE p.is_active = 1
|
||||||
|
AND ua.contract_id IS NOT NULL;
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- Additional Useful Views
|
||||||
|
-- =====================================================
|
||||||
|
-- View แสดงเอกสารทั้งหมดที่มีไฟล์แนบ
|
||||||
|
CREATE VIEW v_documents_with_attachments AS
|
||||||
|
SELECT 'CORRESPONDENCE' AS document_type,
|
||||||
|
c.id AS document_id,
|
||||||
|
c.correspondence_number AS document_number,
|
||||||
|
c.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name,
|
||||||
|
COUNT(ca.attachment_id) AS attachment_count,
|
||||||
|
MAX(a.created_at) AS latest_attachment_date
|
||||||
|
FROM correspondences c
|
||||||
|
INNER JOIN projects p ON c.project_id = p.id
|
||||||
|
LEFT JOIN correspondence_attachments ca ON c.id = ca.correspondence_id
|
||||||
|
LEFT JOIN attachments a ON ca.attachment_id = a.id
|
||||||
|
WHERE c.deleted_at IS NULL
|
||||||
|
GROUP BY c.id,
|
||||||
|
c.correspondence_number,
|
||||||
|
c.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name
|
||||||
|
UNION ALL
|
||||||
|
SELECT 'CIRCULATION' AS document_type,
|
||||||
|
circ.id AS document_id,
|
||||||
|
circ.circulation_no AS document_number,
|
||||||
|
corr.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name,
|
||||||
|
COUNT(ca.attachment_id) AS attachment_count,
|
||||||
|
MAX(a.created_at) AS latest_attachment_date
|
||||||
|
FROM circulations circ
|
||||||
|
INNER JOIN correspondences corr ON circ.correspondence_id = corr.id
|
||||||
|
INNER JOIN projects p ON corr.project_id = p.id
|
||||||
|
LEFT JOIN circulation_attachments ca ON circ.id = ca.circulation_id
|
||||||
|
LEFT JOIN attachments a ON ca.attachment_id = a.id
|
||||||
|
GROUP BY circ.id,
|
||||||
|
circ.circulation_no,
|
||||||
|
corr.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name
|
||||||
|
UNION ALL
|
||||||
|
SELECT 'SHOP_DRAWING' AS document_type,
|
||||||
|
sdr.id AS document_id,
|
||||||
|
sd.drawing_number AS document_number,
|
||||||
|
sd.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name,
|
||||||
|
COUNT(sdra.attachment_id) AS attachment_count,
|
||||||
|
MAX(a.created_at) AS latest_attachment_date
|
||||||
|
FROM shop_drawing_revisions sdr
|
||||||
|
INNER JOIN shop_drawings sd ON sdr.shop_drawing_id = sd.id
|
||||||
|
INNER JOIN projects p ON sd.project_id = p.id
|
||||||
|
LEFT JOIN shop_drawing_revision_attachments sdra ON sdr.id = sdra.shop_drawing_revision_id
|
||||||
|
LEFT JOIN attachments a ON sdra.attachment_id = a.id
|
||||||
|
WHERE sd.deleted_at IS NULL
|
||||||
|
GROUP BY sdr.id,
|
||||||
|
sd.drawing_number,
|
||||||
|
sd.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name
|
||||||
|
UNION ALL
|
||||||
|
SELECT 'CONTRACT_DRAWING' AS document_type,
|
||||||
|
cd.id AS document_id,
|
||||||
|
cd.condwg_no AS document_number,
|
||||||
|
cd.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name,
|
||||||
|
COUNT(cda.attachment_id) AS attachment_count,
|
||||||
|
MAX(a.created_at) AS latest_attachment_date
|
||||||
|
FROM contract_drawings cd
|
||||||
|
INNER JOIN projects p ON cd.project_id = p.id
|
||||||
|
LEFT JOIN contract_drawing_attachments cda ON cd.id = cda.contract_drawing_id
|
||||||
|
LEFT JOIN attachments a ON cda.attachment_id = a.id
|
||||||
|
WHERE cd.deleted_at IS NULL
|
||||||
|
GROUP BY cd.id,
|
||||||
|
cd.condwg_no,
|
||||||
|
cd.project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name;
|
||||||
|
|
||||||
|
-- View แสดงสถิติเอกสารตามประเภทและสถานะ
|
||||||
|
CREATE VIEW v_document_statistics AS
|
||||||
|
SELECT p.id AS project_id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name,
|
||||||
|
ct.id AS correspondence_type_id,
|
||||||
|
ct.type_code,
|
||||||
|
ct.type_name,
|
||||||
|
cs.id AS status_id,
|
||||||
|
cs.status_code,
|
||||||
|
cs.status_name,
|
||||||
|
COUNT(DISTINCT c.id) AS document_count,
|
||||||
|
COUNT(DISTINCT cr.id) AS revision_count
|
||||||
|
FROM projects p
|
||||||
|
CROSS JOIN correspondence_types ct
|
||||||
|
CROSS JOIN correspondence_status cs
|
||||||
|
LEFT JOIN correspondences c ON p.id = c.project_id
|
||||||
|
AND ct.id = c.correspondence_type_id
|
||||||
|
LEFT JOIN correspondence_revisions cr ON c.id = cr.correspondence_id
|
||||||
|
AND cs.id = cr.correspondence_status_id
|
||||||
|
AND cr.is_current = TRUE
|
||||||
|
WHERE p.is_active = 1
|
||||||
|
AND ct.is_active = 1
|
||||||
|
AND cs.is_active = 1
|
||||||
|
GROUP BY p.id,
|
||||||
|
p.project_code,
|
||||||
|
p.project_name,
|
||||||
|
ct.id,
|
||||||
|
ct.type_code,
|
||||||
|
ct.type_name,
|
||||||
|
cs.id,
|
||||||
|
cs.status_code,
|
||||||
|
cs.status_name;
|
||||||
|
|
||||||
|
-- =====================================================
|
||||||
|
-- Indexes for View Performance Optimization
|
||||||
|
-- =====================================================
|
||||||
|
-- Indexes for v_current_correspondences performance
|
||||||
|
CREATE INDEX idx_correspondences_type_project ON correspondences (correspondence_type_id, project_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_corr_revisions_current_status ON correspondence_revisions (is_current, correspondence_status_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_corr_revisions_correspondence_current ON correspondence_revisions (correspondence_id, is_current);
|
||||||
|
|
||||||
|
-- Indexes for v_current_rfas performance
|
||||||
|
CREATE INDEX idx_rfa_revisions_current_status ON rfa_revisions (is_current, rfa_status_code_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_rfa_revisions_rfa_current ON rfa_revisions (rfa_id, is_current);
|
||||||
|
|
||||||
|
-- Indexes for document statistics performance
|
||||||
|
CREATE INDEX idx_correspondences_project_type ON correspondences (project_id, correspondence_type_id);
|
||||||
|
|
||||||
|
CREATE INDEX idx_corr_revisions_status_current ON correspondence_revisions (correspondence_status_id, is_current);
|
||||||
|
|
||||||
|
CREATE INDEX IDX_AUDIT_DOC_ID ON document_number_audit (document_id);
|
||||||
|
|
||||||
|
CREATE INDEX IDX_AUDIT_STATUS ON document_number_audit (STATUS);
|
||||||
|
|
||||||
|
CREATE INDEX IDX_AUDIT_OPERATION ON document_number_audit (operation);
|
||||||
|
|
||||||
|
SET FOREIGN_KEY_CHECKS = 1;
|
||||||
|
|
||||||
@@ -1,53 +1,206 @@
|
|||||||
{
|
{
|
||||||
|
"name": "LCBP3 Migration Workflow v1.8.0",
|
||||||
"meta": {
|
"meta": {
|
||||||
"instanceId": "lcbp3-migration"
|
"instanceId": "lcbp3-migration-free"
|
||||||
|
},
|
||||||
|
"settings": {
|
||||||
|
"executionOrder": "v1"
|
||||||
},
|
},
|
||||||
"nodes": [
|
"nodes": [
|
||||||
{
|
{
|
||||||
"parameters": {},
|
"parameters": {},
|
||||||
"id": "trigger-1",
|
"id": "trigger-manual",
|
||||||
"name": "When clicking ‘Execute Workflow’",
|
"name": "Manual Trigger",
|
||||||
"type": "n8n-nodes-base.manualTrigger",
|
"type": "n8n-nodes-base.manualTrigger",
|
||||||
"typeVersion": 1,
|
"typeVersion": 1,
|
||||||
"position": [0, 0]
|
"position": [
|
||||||
|
0,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "กดรันด้วยตนเอง"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"parameters": {
|
"parameters": {
|
||||||
"operation": "read",
|
"jsCode": "// ============================================\n// CONFIGURATION - แก้ไขค่าที่นี่\n// ============================================\nconst CONFIG = {\n // Ollama Settings\n OLLAMA_HOST: 'http://192.168.20.100:11434',\n OLLAMA_MODEL_PRIMARY: 'llama3.2:3b',\n OLLAMA_MODEL_FALLBACK: 'mistral:7b-instruct-q4_K_M',\n \n // Backend Settings\n BACKEND_URL: 'https://api.np-dms.work',\n MIGRATION_TOKEN: 'Bearer YOUR_MIGRATION_TOKEN_HERE',\n \n // Batch Settings\n BATCH_SIZE: 10,\n BATCH_ID: 'migration_20260226',\n DELAY_MS: 2000,\n \n // Thresholds\n CONFIDENCE_HIGH: 0.85,\n CONFIDENCE_LOW: 0.60,\n MAX_RETRY: 3,\n FALLBACK_THRESHOLD: 5,\n \n // Paths\n STAGING_PATH: '/home/node/.n8n-files/staging_ai',\n LOG_PATH: '/home/node/.n8n-files/migration_logs',\n \n // Database\n DB_HOST: '192.168.1.100',\n DB_PORT: 3306,\n DB_NAME: 'lcbp3_db',\n DB_USER: 'migration_bot',\n DB_PASSWORD: 'YOUR_DB_PASSWORD_HERE'\n};\n\nreturn [{ json: { config_loaded: true, timestamp: new Date().toISOString(), config: CONFIG } }];"
|
||||||
"fileFormat": "xlsx",
|
|
||||||
"options": {}
|
|
||||||
},
|
},
|
||||||
"id": "spreadsheet-1",
|
"id": "config-setter",
|
||||||
"name": "Read Excel Data",
|
"name": "Set Configuration",
|
||||||
"type": "n8n-nodes-base.spreadsheetFile",
|
|
||||||
"typeVersion": 2,
|
|
||||||
"position": [200, 0]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"parameters": {
|
|
||||||
"batchSize": 10,
|
|
||||||
"options": {}
|
|
||||||
},
|
|
||||||
"id": "split-in-batches-1",
|
|
||||||
"name": "Split In Batches",
|
|
||||||
"type": "n8n-nodes-base.splitInBatches",
|
|
||||||
"typeVersion": 3,
|
|
||||||
"position": [400, 0]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"parameters": {
|
|
||||||
"jsCode": "const item = $input.first();\n\nconst prompt = `You are a Document Controller for a large construction project.\nYour task is to validate document metadata.\nYou MUST respond ONLY with valid JSON. No explanation, no markdown, no extra text.\n\nDocument Number: ${item.json.document_number}\nTitle: ${item.json.title}\nCategory List: [\"Correspondence\",\"RFA\",\"Drawing\",\"Transmittal\",\"Report\",\"Other\"]\n\nRespond ONLY with this exact JSON structure:\n{\n \"is_valid\": true,\n \"confidence\": 0.95,\n \"suggested_category\": \"Correspondence\",\n \"detected_issues\": [],\n \"suggested_title\": null\n}`;\n\nreturn [{\n json: {\n ...item.json,\n ollama_payload: {\n model: \"llama3.2:3b\",\n format: \"json\",\n stream: false,\n prompt: prompt\n }\n }\n}];"
|
|
||||||
},
|
|
||||||
"id": "code-1",
|
|
||||||
"name": "Build Prompt",
|
|
||||||
"type": "n8n-nodes-base.code",
|
"type": "n8n-nodes-base.code",
|
||||||
"typeVersion": 2,
|
"typeVersion": 2,
|
||||||
"position": [620, 0]
|
"position": [
|
||||||
|
200,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "กำหนดค่า Configuration ทั้งหมด - แก้ไขที่นี่ก่อนรัน"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"method": "GET",
|
||||||
|
"url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/master/correspondence-types",
|
||||||
|
"sendHeaders": true,
|
||||||
|
"headerParameters": {
|
||||||
|
"parameters": [
|
||||||
|
{
|
||||||
|
"name": "Authorization",
|
||||||
|
"value": "={{$workflow.staticData.config.MIGRATION_TOKEN}}"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"options": {
|
||||||
|
"timeout": 10000
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"id": "preflight-categories",
|
||||||
|
"name": "Fetch Categories",
|
||||||
|
"type": "n8n-nodes-base.httpRequest",
|
||||||
|
"typeVersion": 4.1,
|
||||||
|
"position": [
|
||||||
|
400,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "ดึง Categories จาก Backend"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"method": "GET",
|
||||||
|
"url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/health",
|
||||||
|
"options": {
|
||||||
|
"timeout": 5000
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"id": "preflight-health",
|
||||||
|
"name": "Check Backend Health",
|
||||||
|
"type": "n8n-nodes-base.httpRequest",
|
||||||
|
"typeVersion": 4.1,
|
||||||
|
"position": [
|
||||||
|
400,
|
||||||
|
200
|
||||||
|
],
|
||||||
|
"notes": "ตรวจสอบ Backend พร้อมใช้งาน",
|
||||||
|
"onError": "continueErrorOutput"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const fs = require('fs');\nconst config = $('Set Configuration').first().json.config;\n\n// Check file mount\ntry {\n const files = fs.readdirSync(config.STAGING_PATH);\n if (files.length === 0) throw new Error('staging_ai is empty');\n \n // Check write permission to log path\n fs.writeFileSync(`${config.LOG_PATH}/.preflight_ok`, new Date().toISOString());\n \n // Store categories\n const categories = $input.first().json.categories || \n ['Correspondence','RFA','Drawing','Transmittal','Report','Other'];\n $('File Mount Check').first().json.system_categories = categories;\n \n return [{ json: { \n preflight_ok: true, \n file_count: files.length,\n system_categories: categories,\n timestamp: new Date().toISOString()\n }}];\n} catch (err) {\n throw new Error(`Pre-flight check failed: ${err.message}`);\n}"
|
||||||
|
},
|
||||||
|
"id": "preflight-check",
|
||||||
|
"name": "File Mount Check",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [
|
||||||
|
600,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "ตรวจสอบ File System และเก็บ Categories"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$('Set Configuration').first().json.config.DB_HOST}}",
|
||||||
|
"port": "={{$('Set Configuration').first().json.config.DB_PORT}}",
|
||||||
|
"database": "={{$('Set Configuration').first().json.config.DB_NAME}}",
|
||||||
|
"user": "={{$('Set Configuration').first().json.config.DB_USER}}",
|
||||||
|
"password": "={{$('Set Configuration').first().json.config.DB_PASSWORD}}",
|
||||||
|
"query": "SELECT last_processed_index, status FROM migration_progress WHERE batch_id = '{{$('Set Configuration').first().json.config.BATCH_ID}}' LIMIT 1",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "checkpoint-read",
|
||||||
|
"name": "Read Checkpoint",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4,
|
||||||
|
"position": [
|
||||||
|
800,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "อ่านตำแหน่งล่าสุดที่ประมวลผล",
|
||||||
|
"onError": "continueErrorOutput"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "toData",
|
||||||
|
"binaryProperty": "data",
|
||||||
|
"options": {
|
||||||
|
"sheetName": "Sheet1"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"id": "excel-reader",
|
||||||
|
"name": "Read Excel",
|
||||||
|
"type": "n8n-nodes-base.spreadsheetFile",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [
|
||||||
|
800,
|
||||||
|
200
|
||||||
|
],
|
||||||
|
"notes": "อ่านไฟล์ Excel รายการเอกสาร"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const checkpoint = $input.first().json[0] || { last_processed_index: 0, status: 'NEW' };\nconst startIndex = checkpoint.last_processed_index || 0;\nconst config = $('Set Configuration').first().json.config;\n\nconst allItems = $('Read Excel').all()[0].json.data || [];\nconst remaining = allItems.slice(startIndex);\nconst currentBatch = remaining.slice(0, config.BATCH_SIZE);\n\n// Encoding Normalization\nconst normalize = (str) => {\n if (!str) return '';\n return String(str).normalize('NFC').trim();\n};\n\nreturn currentBatch.map((item, i) => ({\n json: {\n document_number: normalize(item.document_number || item['Document Number']),\n title: normalize(item.title || item.Title || item['Subject']),\n legacy_number: normalize(item.legacy_number || item['Legacy Number']),\n excel_revision: item.revision || item.Revision || 1,\n original_index: startIndex + i,\n batch_id: config.BATCH_ID,\n file_name: `${normalize(item.document_number)}.pdf`\n }\n}));"
|
||||||
|
},
|
||||||
|
"id": "batch-processor",
|
||||||
|
"name": "Process Batch + Encoding",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [
|
||||||
|
1000,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "ตัด Batch + Normalize UTF-8"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const fs = require('fs');\nconst path = require('path');\nconst config = $('Set Configuration').first().json.config;\n\nconst items = $input.all();\nconst validated = [];\nconst errors = [];\n\nfor (const item of items) {\n const docNum = item.json.document_number;\n \n // Sanitize filename\n const safeName = path.basename(String(docNum).replace(/[^a-zA-Z0-9\\-_.]/g, '_')).normalize('NFC');\n const filePath = path.resolve(config.STAGING_PATH, `${safeName}.pdf`);\n \n // Path traversal check\n if (!filePath.startsWith(config.STAGING_PATH)) {\n errors.push({\n ...item,\n json: { ...item.json, error: 'Path traversal detected', error_type: 'SECURITY', file_exists: false }\n });\n continue;\n }\n \n try {\n if (fs.existsSync(filePath)) {\n const stats = fs.statSync(filePath);\n validated.push({\n ...item,\n json: { ...item.json, file_exists: true, file_size: stats.size, file_path: filePath }\n });\n } else {\n errors.push({\n ...item,\n json: { ...item.json, error: `File not found: ${safeName}.pdf`, error_type: 'FILE_NOT_FOUND', file_exists: false }\n });\n }\n } catch (err) {\n errors.push({\n ...item,\n json: { ...item.json, error: err.message, error_type: 'FILE_ERROR', file_exists: false }\n });\n }\n}\n\n// Output 0: Validated, Output 1: Errors\nreturn [validated, errors];"
|
||||||
|
},
|
||||||
|
"id": "file-validator",
|
||||||
|
"name": "File Validator",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [
|
||||||
|
1200,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "ตรวจสอบไฟล์ PDF มีอยู่จริง + Sanitize path"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$('Set Configuration').first().json.config.DB_HOST}}",
|
||||||
|
"port": "={{$('Set Configuration').first().json.config.DB_PORT}}",
|
||||||
|
"database": "={{$('Set Configuration').first().json.config.DB_NAME}}",
|
||||||
|
"user": "={{$('Set Configuration').first().json.config.DB_USER}}",
|
||||||
|
"password": "={{$('Set Configuration').first().json.config.DB_PASSWORD}}",
|
||||||
|
"query": "SELECT is_fallback_active, recent_error_count FROM migration_fallback_state WHERE batch_id = '{{$('Set Configuration').first().json.config.BATCH_ID}}' LIMIT 1",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "fallback-check",
|
||||||
|
"name": "Check Fallback State",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4,
|
||||||
|
"position": [
|
||||||
|
1400,
|
||||||
|
-200
|
||||||
|
],
|
||||||
|
"notes": "ตรวจสอบว่าต้องใช้ Fallback Model หรือไม่",
|
||||||
|
"onError": "continueErrorOutput"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const config = $('Set Configuration').first().json.config;\nconst fallbackState = $input.first().json[0] || { is_fallback_active: false, recent_error_count: 0 };\n\nconst isFallback = fallbackState.is_fallback_active || false;\nconst model = isFallback ? config.OLLAMA_MODEL_FALLBACK : config.OLLAMA_MODEL_PRIMARY;\n\nconst systemCategories = $('File Mount Check').first().json.system_categories || \n ['Correspondence','RFA','Drawing','Transmittal','Report','Other'];\n\nconst items = $('File Validator').all();\n\nreturn items.map(item => {\n const systemPrompt = `You are a Document Controller for a large construction project.\nYour task is to validate document metadata.\nYou MUST respond ONLY with valid JSON. No explanation, no markdown, no extra text.\nIf there are no issues, \"detected_issues\" must be an empty array [].`;\n\n const userPrompt = `Validate this document metadata and respond in JSON:\n\nDocument Number: ${item.json.document_number}\nTitle: ${item.json.title}\nExpected Pattern: [ORG]-[TYPE]-[SEQ] e.g. \"TCC-COR-0001\"\nCategory List (MUST match system enum exactly): ${JSON.stringify(systemCategories)}\n\nRespond ONLY with this exact JSON structure:\n{\n \"is_valid\": true | false,\n \"confidence\": 0.0 to 1.0,\n \"suggested_category\": \"<one from Category List>\",\n \"detected_issues\": [\"<issue1>\"],\n \"suggested_title\": \"<corrected title or null>\"\n}`;\n\n return {\n json: {\n ...item.json,\n active_model: model,\n is_fallback: isFallback,\n system_categories: systemCategories,\n ollama_payload: {\n model: model,\n prompt: `${systemPrompt}\\n\\n${userPrompt}`,\n stream: false,\n format: 'json'\n }\n }\n };\n});"
|
||||||
|
},
|
||||||
|
"id": "prompt-builder",
|
||||||
|
"name": "Build AI Prompt",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [
|
||||||
|
1400,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "สร้าง Prompt โดยใช้ Categories จาก System"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"parameters": {
|
"parameters": {
|
||||||
"method": "POST",
|
"method": "POST",
|
||||||
"url": "http://192.168.20.100:11434/api/generate",
|
"url": "={{$('Set Configuration').first().json.config.OLLAMA_HOST}}/api/generate",
|
||||||
"sendBody": true,
|
"sendBody": true,
|
||||||
"specifyBody": "json",
|
"specifyBody": "json",
|
||||||
"jsonBody": "={{ $json.ollama_payload }}",
|
"jsonBody": "={{ $json.ollama_payload }}",
|
||||||
@@ -55,157 +208,404 @@
|
|||||||
"timeout": 30000
|
"timeout": 30000
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"id": "http-1",
|
"id": "ollama-call",
|
||||||
"name": "Ollama Local API",
|
"name": "Ollama AI Analysis",
|
||||||
"type": "n8n-nodes-base.httpRequest",
|
"type": "n8n-nodes-base.httpRequest",
|
||||||
"typeVersion": 4.1,
|
"typeVersion": 4.1,
|
||||||
"position": [840, 0]
|
"position": [
|
||||||
|
1600,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "เรียก Ollama วิเคราะห์เอกสาร"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"parameters": {
|
"parameters": {
|
||||||
"jsCode": "const items = $input.all();\nconst parsed = [];\n\nfor (const item of items) {\n try {\n let raw = item.json.response || '';\n raw = raw.replace(/```json/gi, '').replace(/```/g, '').trim();\n const aiResult = JSON.parse(raw);\n parsed.push({ json: { ...item.json, ai_result: aiResult } });\n } catch (err) {\n parsed.push({ json: { ...item.json, ai_result: { confidence: 0, is_valid: false, error: err.message } } });\n }\n}\nreturn parsed;"
|
"jsCode": "const items = $input.all();\nconst parsed = [];\nconst parseErrors = [];\n\nfor (const item of items) {\n try {\n let raw = item.json.response || '';\n \n // Clean markdown\n raw = raw.replace(/```json/gi, '').replace(/```/g, '').trim();\n const result = JSON.parse(raw);\n \n // Schema Validation\n if (typeof result.is_valid !== 'boolean') throw new Error('is_valid must be boolean');\n if (typeof result.confidence !== 'number' || result.confidence < 0 || result.confidence > 1) {\n throw new Error('confidence must be float 0.0-1.0');\n }\n if (!Array.isArray(result.detected_issues)) throw new Error('detected_issues must be array');\n \n // Enum Validation\n const systemCategories = item.json.system_categories || [];\n if (!systemCategories.includes(result.suggested_category)) {\n throw new Error(`Category \"${result.suggested_category}\" not in system enum`);\n }\n \n parsed.push({\n ...item,\n json: { ...item.json, ai_result: result, parse_error: null }\n });\n } catch (err) {\n parseErrors.push({\n ...item,\n json: {\n ...item.json,\n ai_result: null,\n parse_error: err.message,\n raw_ai_response: item.json.response,\n error_type: 'AI_PARSE_ERROR'\n }\n });\n }\n}\n\nreturn [parsed, parseErrors];"
|
||||||
},
|
},
|
||||||
"id": "code-2",
|
"id": "json-parser",
|
||||||
"name": "Parse JSON",
|
"name": "Parse & Validate AI Response",
|
||||||
"type": "n8n-nodes-base.code",
|
"type": "n8n-nodes-base.code",
|
||||||
"typeVersion": 2,
|
"typeVersion": 2,
|
||||||
"position": [1040, 0]
|
"position": [
|
||||||
|
1800,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "Parse JSON + Validate Schema + Enum Check"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"parameters": {
|
"parameters": {
|
||||||
"conditions": {
|
"operation": "executeQuery",
|
||||||
"boolean": [
|
"host": "={{$('Set Configuration').first().json.config.DB_HOST}}",
|
||||||
{
|
"port": "={{$('Set Configuration').first().json.config.DB_PORT}}",
|
||||||
"value1": "={{ $json.ai_result.confidence >= 0.85 && $json.ai_result.is_valid }}",
|
"database": "={{$('Set Configuration').first().json.config.DB_NAME}}",
|
||||||
"value2": true
|
"user": "={{$('Set Configuration').first().json.config.DB_USER}}",
|
||||||
}
|
"password": "={{$('Set Configuration').first().json.config.DB_PASSWORD}}",
|
||||||
]
|
"query": "INSERT INTO migration_fallback_state (batch_id, recent_error_count, is_fallback_active) VALUES ('{{$('Set Configuration').first().json.config.BATCH_ID}}', 1, FALSE) ON DUPLICATE KEY UPDATE recent_error_count = recent_error_count + 1, is_fallback_active = CASE WHEN recent_error_count + 1 >= {{$('Set Configuration').first().json.config.FALLBACK_THRESHOLD}} THEN TRUE ELSE is_fallback_active END, updated_at = NOW()",
|
||||||
}
|
"options": {}
|
||||||
},
|
},
|
||||||
"id": "if-1",
|
"id": "fallback-update",
|
||||||
"name": "Confidence >= 0.85?",
|
"name": "Update Fallback State",
|
||||||
"type": "n8n-nodes-base.if",
|
"type": "n8n-nodes-base.mySql",
|
||||||
"typeVersion": 1,
|
"typeVersion": 2.4,
|
||||||
"position": [1240, 0]
|
"position": [
|
||||||
|
2000,
|
||||||
|
200
|
||||||
|
],
|
||||||
|
"notes": "เพิ่ม Error count และตรวจสอบ Fallback threshold"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const config = $('Set Configuration').first().json.config;\nconst items = $('Parse & Validate AI Response').all();\n\nconst autoIngest = [];\nconst reviewQueue = [];\nconst rejectLog = [];\nconst errorLog = [];\n\nfor (const item of items) {\n if (item.json.parse_error || !item.json.ai_result) {\n errorLog.push(item);\n continue;\n }\n \n const ai = item.json.ai_result;\n \n // Revision Drift Protection (ถ้ามีข้อมูลจาก DB)\n if (item.json.current_db_revision !== undefined) {\n const expectedRev = item.json.current_db_revision + 1;\n if (parseInt(item.json.excel_revision) !== expectedRev) {\n reviewQueue.push({\n ...item,\n json: { ...item.json, review_reason: `Revision drift: Excel=${item.json.excel_revision}, Expected=${expectedRev}` }\n });\n continue;\n }\n }\n \n // Confidence Routing\n if (ai.confidence >= config.CONFIDENCE_HIGH && ai.is_valid === true) {\n autoIngest.push(item);\n } else if (ai.confidence >= config.CONFIDENCE_LOW) {\n reviewQueue.push({\n ...item,\n json: { ...item.json, review_reason: `Confidence ${ai.confidence.toFixed(2)} < ${config.CONFIDENCE_HIGH}` }\n });\n } else {\n rejectLog.push({\n ...item,\n json: { ...item.json, reject_reason: ai.is_valid === false ? 'AI marked invalid' : `Confidence ${ai.confidence.toFixed(2)} < ${config.CONFIDENCE_LOW}` }\n });\n }\n}\n\n// Output 0: Auto, 1: Review, 2: Reject, 3: Error\nreturn [autoIngest, reviewQueue, rejectLog, errorLog];"
|
||||||
|
},
|
||||||
|
"id": "confidence-router",
|
||||||
|
"name": "Confidence Router",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [
|
||||||
|
2000,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "แยกตาม Confidence: Auto(≥0.85) / Review(≥0.60) / Reject(<0.60)"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"parameters": {
|
"parameters": {
|
||||||
"method": "POST",
|
"method": "POST",
|
||||||
"url": "http://<YOUR_BACKEND_IP>:3000/api/migration/import",
|
"url": "={{$('Set Configuration').first().json.config.BACKEND_URL}}/api/migration/import",
|
||||||
"sendHeaders": true,
|
"sendHeaders": true,
|
||||||
"headerParameters": {
|
"headerParameters": {
|
||||||
"parameters": [
|
"parameters": [
|
||||||
{
|
{
|
||||||
"name": "Idempotency-Key",
|
"name": "Authorization",
|
||||||
"value": "={{ $json.document_number }}:BATCH-001"
|
"value": "={{$workflow.staticData.config.MIGRATION_TOKEN}}"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"name": "Authorization",
|
"name": "Idempotency-Key",
|
||||||
"value": "Bearer <YOUR_MIGRATION_TOKEN>"
|
"value": "={{$json.document_number}}:{{$workflow.staticData.config.BATCH_ID}}"
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"sendBody": true,
|
"sendBody": true,
|
||||||
"specifyBody": "json",
|
"specifyBody": "json",
|
||||||
"jsonBody": "={\n \"source_file_path\": \"/share/np-dms/staging_ai/{{$json.document_number}}.pdf\",\n \"document_number\": \"{{$json.document_number}}\",\n \"title\": \"{{$json.ai_result.suggested_title || $json.title}}\",\n \"category\": \"{{$json.ai_result.suggested_category}}\",\n \"revision\": 1, \n \"batch_id\": \"BATCH_001\",\n \"ai_confidence\": {{$json.ai_result.confidence}},\n \"ai_issues\": {{$json.ai_result.detected_issues}},\n \"legacy_document_number\": \"{{$json.legacy_number}}\"\n}",
|
"jsonBody": "={\n \"document_number\": \"{{$json.document_number}}\",\n \"title\": \"{{$json.ai_result.suggested_title || $json.title}}\",\n \"category\": \"{{$json.ai_result.suggested_category}}\",\n \"source_file_path\": \"{{$json.file_path}}\",\n \"ai_confidence\": {{$json.ai_result.confidence}},\n \"ai_issues\": {{JSON.stringify($json.ai_result.detected_issues)}},\n \"migrated_by\": \"SYSTEM_IMPORT\",\n \"batch_id\": \"{{$('Set Configuration').first().json.config.BATCH_ID}}\",\n \"details\": {\n \"legacy_number\": \"{{$json.legacy_number}}\"\n }\n}",
|
||||||
"options": {}
|
"options": {
|
||||||
|
"timeout": 30000
|
||||||
|
}
|
||||||
},
|
},
|
||||||
"id": "http-2",
|
"id": "backend-import",
|
||||||
"name": "LCBP3 Backend (Auto Ingest)",
|
"name": "Import to Backend",
|
||||||
"type": "n8n-nodes-base.httpRequest",
|
"type": "n8n-nodes-base.httpRequest",
|
||||||
"typeVersion": 4.1,
|
"typeVersion": 4.1,
|
||||||
"position": [1460, -100]
|
"position": [
|
||||||
|
2200,
|
||||||
|
-200
|
||||||
|
],
|
||||||
|
"notes": "ส่งข้อมูลเข้า LCBP3 Backend พร้อม Idempotency-Key"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"parameters": {
|
"parameters": {
|
||||||
"jsCode": "return [{ json: { message: \"Sent to Human Review Queue OR Check AI Error Log\", data: $input.first().json } }];"
|
"jsCode": "const item = $input.first();\nconst shouldCheckpoint = item.json.original_index % 10 === 0;\n\nreturn [{\n json: {\n ...item.json,\n should_update_checkpoint: shouldCheckpoint,\n checkpoint_index: item.json.original_index,\n import_status: 'success',\n timestamp: new Date().toISOString()\n }\n}];"
|
||||||
},
|
},
|
||||||
"id": "code-3",
|
"id": "checkpoint-flag",
|
||||||
"name": "Review Queue / Reject Log",
|
"name": "Flag Checkpoint",
|
||||||
"type": "n8n-nodes-base.code",
|
"type": "n8n-nodes-base.code",
|
||||||
"typeVersion": 2,
|
"typeVersion": 2,
|
||||||
"position": [1460, 100]
|
"position": [
|
||||||
|
2400,
|
||||||
|
-200
|
||||||
|
],
|
||||||
|
"notes": "กำหนดว่าจะบันทึก Checkpoint หรือไม่ (ทุก 10 records)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$('Set Configuration').first().json.config.DB_HOST}}",
|
||||||
|
"port": "={{$('Set Configuration').first().json.config.DB_PORT}}",
|
||||||
|
"database": "={{$('Set Configuration').first().json.config.DB_NAME}}",
|
||||||
|
"user": "={{$('Set Configuration').first().json.config.DB_USER}}",
|
||||||
|
"password": "={{$('Set Configuration').first().json.config.DB_PASSWORD}}",
|
||||||
|
"query": "INSERT INTO migration_progress (batch_id, last_processed_index, status) VALUES ('{{$('Set Configuration').first().json.config.BATCH_ID}}', {{$json.checkpoint_index}}, 'RUNNING') ON DUPLICATE KEY UPDATE last_processed_index = {{$json.checkpoint_index}}, updated_at = NOW()",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "checkpoint-save",
|
||||||
|
"name": "Save Checkpoint",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4,
|
||||||
|
"position": [
|
||||||
|
2600,
|
||||||
|
-200
|
||||||
|
],
|
||||||
|
"notes": "บันทึกความคืบหน้าลง Database"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$('Set Configuration').first().json.config.DB_HOST}}",
|
||||||
|
"port": "={{$('Set Configuration').first().json.config.DB_PORT}}",
|
||||||
|
"database": "={{$('Set Configuration').first().json.config.DB_NAME}}",
|
||||||
|
"user": "={{$('Set Configuration').first().json.config.DB_USER}}",
|
||||||
|
"password": "={{$('Set Configuration').first().json.config.DB_PASSWORD}}",
|
||||||
|
"query": "INSERT INTO migration_review_queue (document_number, title, original_title, ai_suggested_category, ai_confidence, ai_issues, review_reason, status, created_at) VALUES ('{{$json.document_number}}', '{{$json.ai_result.suggested_title || $json.title}}', '{{$json.title}}', '{{$json.ai_result.suggested_category}}', {{$json.ai_result.confidence}}, '{{JSON.stringify($json.ai_result.detected_issues)}}', '{{$json.review_reason}}', 'PENDING', NOW()) ON DUPLICATE KEY UPDATE status = 'PENDING', review_reason = '{{$json.review_reason}}', created_at = NOW()",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "review-queue-insert",
|
||||||
|
"name": "Insert Review Queue",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4,
|
||||||
|
"position": [
|
||||||
|
2200,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "บันทึกรายการที่ต้องตรวจสอบโดยคน (ไม่สร้าง Correspondence)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const fs = require('fs');\nconst item = $input.first();\nconst config = $('Set Configuration').first().json.config;\n\nconst csvPath = `${config.LOG_PATH}/reject_log.csv`;\nconst header = 'timestamp,document_number,title,reject_reason,ai_confidence,ai_issues\\n';\nconst esc = (s) => `\"${String(s || '').replace(/\"/g, '\"\"')}\"`;\n\nif (!fs.existsSync(csvPath)) {\n fs.writeFileSync(csvPath, header, 'utf8');\n}\n\nconst line = [\n new Date().toISOString(),\n esc(item.json.document_number),\n esc(item.json.title),\n esc(item.json.reject_reason),\n item.json.ai_result?.confidence ?? 'N/A',\n esc(JSON.stringify(item.json.ai_result?.detected_issues || []))\n].join(',') + '\\n';\n\nfs.appendFileSync(csvPath, line, 'utf8');\n\nreturn [$input.first()];"
|
||||||
|
},
|
||||||
|
"id": "reject-logger",
|
||||||
|
"name": "Log Reject to CSV",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [
|
||||||
|
2200,
|
||||||
|
200
|
||||||
|
],
|
||||||
|
"notes": "บันทึกรายการที่ถูกปฏิเสธลง CSV"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"jsCode": "const fs = require('fs');\nconst items = $input.all();\nconst config = $('Set Configuration').first().json.config;\n\nconst csvPath = `${config.LOG_PATH}/error_log.csv`;\nconst header = 'timestamp,document_number,error_type,error_message,raw_ai_response\\n';\nconst esc = (s) => `\"${String(s || '').replace(/\"/g, '\"\"')}\"`;\n\nif (!fs.existsSync(csvPath)) {\n fs.writeFileSync(csvPath, header, 'utf8');\n}\n\nfor (const item of items) {\n const line = [\n new Date().toISOString(),\n esc(item.json.document_number),\n esc(item.json.error_type || 'UNKNOWN'),\n esc(item.json.error || item.json.parse_error),\n esc(item.json.raw_ai_response || '')\n ].join(',') + '\\n';\n \n fs.appendFileSync(csvPath, line, 'utf8');\n}\n\nreturn items;"
|
||||||
|
},
|
||||||
|
"id": "error-logger-csv",
|
||||||
|
"name": "Log Error to CSV",
|
||||||
|
"type": "n8n-nodes-base.code",
|
||||||
|
"typeVersion": 2,
|
||||||
|
"position": [
|
||||||
|
1400,
|
||||||
|
400
|
||||||
|
],
|
||||||
|
"notes": "บันทึก Error ลง CSV (จาก File Validator)"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"operation": "executeQuery",
|
||||||
|
"host": "={{$('Set Configuration').first().json.config.DB_HOST}}",
|
||||||
|
"port": "={{$('Set Configuration').first().json.config.DB_PORT}}",
|
||||||
|
"database": "={{$('Set Configuration').first().json.config.DB_NAME}}",
|
||||||
|
"user": "={{$('Set Configuration').first().json.config.DB_USER}}",
|
||||||
|
"password": "={{$('Set Configuration').first().json.config.DB_PASSWORD}}",
|
||||||
|
"query": "INSERT INTO migration_errors (batch_id, document_number, error_type, error_message, raw_ai_response, created_at) VALUES ('{{$('Set Configuration').first().json.config.BATCH_ID}}', '{{$json.document_number}}', '{{$json.error_type || \"UNKNOWN\"}}', '{{$json.error || $json.parse_error}}', '{{$json.raw_ai_response || \"\"}}', NOW())",
|
||||||
|
"options": {}
|
||||||
|
},
|
||||||
|
"id": "error-logger-db",
|
||||||
|
"name": "Log Error to DB",
|
||||||
|
"type": "n8n-nodes-base.mySql",
|
||||||
|
"typeVersion": 2.4,
|
||||||
|
"position": [
|
||||||
|
2000,
|
||||||
|
400
|
||||||
|
],
|
||||||
|
"notes": "บันทึก Error ลง MariaDB"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"parameters": {
|
||||||
|
"amount": "={{$('Set Configuration').first().json.config.DELAY_MS}}",
|
||||||
|
"unit": "milliseconds"
|
||||||
|
},
|
||||||
|
"id": "delay-node",
|
||||||
|
"name": "Delay",
|
||||||
|
"type": "n8n-nodes-base.wait",
|
||||||
|
"typeVersion": 1,
|
||||||
|
"position": [
|
||||||
|
2800,
|
||||||
|
0
|
||||||
|
],
|
||||||
|
"notes": "หน่วงเวลาระหว่าง Batches"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"connections": {
|
"connections": {
|
||||||
"When clicking ‘Execute Workflow’": {
|
"Manual Trigger": {
|
||||||
"main": [
|
"main": [
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
"node": "Read Excel Data",
|
"node": "Set Configuration",
|
||||||
"type": "main",
|
"type": "main",
|
||||||
"index": 0
|
"index": 0
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"Read Excel Data": {
|
"Set Configuration": {
|
||||||
"main": [
|
"main": [
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
"node": "Split In Batches",
|
"node": "Fetch Categories",
|
||||||
"type": "main",
|
"type": "main",
|
||||||
"index": 0
|
"index": 0
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"Split In Batches": {
|
"Fetch Categories": {
|
||||||
"main": [
|
"main": [
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
"node": "Build Prompt",
|
"node": "File Mount Check",
|
||||||
"type": "main",
|
"type": "main",
|
||||||
"index": 0
|
"index": 0
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"Build Prompt": {
|
"File Mount Check": {
|
||||||
"main": [
|
"main": [
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
"node": "Ollama Local API",
|
"node": "Read Checkpoint",
|
||||||
"type": "main",
|
"type": "main",
|
||||||
"index": 0
|
"index": 0
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"Ollama Local API": {
|
"Read Checkpoint": {
|
||||||
"main": [
|
"main": [
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
"node": "Parse JSON",
|
"node": "Process Batch + Encoding",
|
||||||
"type": "main",
|
"type": "main",
|
||||||
"index": 0
|
"index": 0
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"Parse JSON": {
|
"Process Batch + Encoding": {
|
||||||
"main": [
|
"main": [
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
"node": "Confidence >= 0.85?",
|
"node": "File Validator",
|
||||||
"type": "main",
|
"type": "main",
|
||||||
"index": 0
|
"index": 0
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"Confidence >= 0.85?": {
|
"File Validator": {
|
||||||
"main": [
|
"main": [
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
"node": "LCBP3 Backend (Auto Ingest)",
|
"node": "Build AI Prompt",
|
||||||
"type": "main",
|
"type": "main",
|
||||||
"index": 0
|
"index": 0
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
[
|
[
|
||||||
{
|
{
|
||||||
"node": "Review Queue / Reject Log",
|
"node": "Log Error to CSV",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Build AI Prompt": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Ollama AI Analysis",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Ollama AI Analysis": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Parse & Validate AI Response",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Parse & Validate AI Response": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Confidence Router",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Update Fallback State",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Confidence Router": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Import to Backend",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Insert Review Queue",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Log Reject to CSV",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Log Error to DB",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Import to Backend": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Flag Checkpoint",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Flag Checkpoint": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Save Checkpoint",
|
||||||
|
"type": "main",
|
||||||
|
"index": 0
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"Log Error to CSV": {
|
||||||
|
"main": [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"node": "Log Error to DB",
|
||||||
"type": "main",
|
"type": "main",
|
||||||
"index": 0
|
"index": 0
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -19,13 +19,6 @@
|
|||||||
-- - INDEX idx_doc_number (correspondence_number),
|
-- - INDEX idx_doc_number (correspondence_number),
|
||||||
-- - INDEX idx_deleted_at (deleted_at),
|
-- - INDEX idx_deleted_at (deleted_at),
|
||||||
-- - INDEX idx_created_by (created_by),
|
-- - INDEX idx_created_by (created_by),
|
||||||
-- 2. เพิ่ม:
|
|
||||||
-- 2.1 TABLE migration_progress
|
|
||||||
-- 2.2 TABLE import_transactions
|
|
||||||
-- 2.3 TABLE migration_review_queue
|
|
||||||
-- 2.4 TABLE migration_errors
|
|
||||||
-- 2.5 TABLE migration_fallback_state
|
|
||||||
-- 2.6 TABLE migration_daily_summary
|
|
||||||
-- ==========================================================
|
-- ==========================================================
|
||||||
SET NAMES utf8mb4;
|
SET NAMES utf8mb4;
|
||||||
|
|
||||||
@@ -55,18 +48,6 @@ DROP VIEW IF EXISTS v_current_correspondences;
|
|||||||
-- คำเตือน: ข้อมูลทั้งหมดจะหายไป กรุณา Backup ก่อนรันบน Production
|
-- คำเตือน: ข้อมูลทั้งหมดจะหายไป กรุณา Backup ก่อนรันบน Production
|
||||||
SET FOREIGN_KEY_CHECKS = 0;
|
SET FOREIGN_KEY_CHECKS = 0;
|
||||||
|
|
||||||
DROP TABLE IF EXISTS migration_progress;
|
|
||||||
|
|
||||||
DROP TABLE IF EXISTS import_transactions;
|
|
||||||
|
|
||||||
DROP TABLE IF EXISTS migration_review_queue;
|
|
||||||
|
|
||||||
DROP TABLE IF EXISTS migration_errors;
|
|
||||||
|
|
||||||
DROP TABLE IF EXISTS migration_fallback_state;
|
|
||||||
|
|
||||||
DROP TABLE IF EXISTS migration_daily_summary;
|
|
||||||
|
|
||||||
-- ============================================================
|
-- ============================================================
|
||||||
-- ส่วนที่ 1: ตาราง System, Logs & Preferences (ตารางปลายทาง/ส่วนเสริม)
|
-- ส่วนที่ 1: ตาราง System, Logs & Preferences (ตารางปลายทาง/ส่วนเสริม)
|
||||||
-- ============================================================
|
-- ============================================================
|
||||||
@@ -1563,24 +1544,6 @@ CREATE INDEX idx_wf_hist_instance ON workflow_histories (instance_id);
|
|||||||
|
|
||||||
CREATE INDEX idx_wf_hist_user ON workflow_histories (action_by_user_id);
|
CREATE INDEX idx_wf_hist_user ON workflow_histories (action_by_user_id);
|
||||||
|
|
||||||
-- Checkpoint Table:
|
|
||||||
CREATE TABLE IF NOT EXISTS migration_progress (
|
|
||||||
batch_id VARCHAR(50) PRIMARY KEY,
|
|
||||||
last_processed_index INT DEFAULT 0,
|
|
||||||
STATUS ENUM('RUNNING', 'COMPLETED', 'FAILED') DEFAULT 'RUNNING',
|
|
||||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Idempotency Table :
|
|
||||||
CREATE TABLE IF NOT EXISTS import_transactions (
|
|
||||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
|
||||||
idempotency_key VARCHAR(255) UNIQUE NOT NULL,
|
|
||||||
document_number VARCHAR(100),
|
|
||||||
batch_id VARCHAR(100),
|
|
||||||
status_code INT DEFAULT 201,
|
|
||||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
INDEX idx_idem_key (idempotency_key)
|
|
||||||
);
|
|
||||||
|
|
||||||
-- ============================================================
|
-- ============================================================
|
||||||
-- 5. PARTITIONING PREPARATION (Advance - Optional)
|
-- 5. PARTITIONING PREPARATION (Advance - Optional)
|
||||||
@@ -2071,87 +2034,6 @@ CREATE INDEX idx_correspondences_type_project ON correspondences (correspondence
|
|||||||
|
|
||||||
CREATE INDEX idx_corr_revisions_current_status ON correspondence_revisions (is_current, correspondence_status_id);
|
CREATE INDEX idx_corr_revisions_current_status ON correspondence_revisions (is_current, correspondence_status_id);
|
||||||
|
|
||||||
-- =====================================================
|
|
||||||
-- Migration Tracking Tables (Temporary)
|
|
||||||
-- =====================================================
|
|
||||||
-- Checkpoint
|
|
||||||
CREATE TABLE IF NOT EXISTS migration_progress (
|
|
||||||
batch_id VARCHAR(50) PRIMARY KEY,
|
|
||||||
last_processed_index INT DEFAULT 0,
|
|
||||||
STATUS ENUM('RUNNING', 'COMPLETED', 'FAILED') DEFAULT 'RUNNING',
|
|
||||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Review Queue (Temporary — ไม่ใช่ Business Schema)
|
|
||||||
CREATE TABLE IF NOT EXISTS migration_review_queue (
|
|
||||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
|
||||||
document_number VARCHAR(100) NOT NULL,
|
|
||||||
title TEXT,
|
|
||||||
original_title TEXT,
|
|
||||||
ai_suggested_category VARCHAR(50),
|
|
||||||
ai_confidence DECIMAL(4, 3),
|
|
||||||
ai_issues JSON,
|
|
||||||
review_reason VARCHAR(255),
|
|
||||||
STATUS ENUM('PENDING', 'APPROVED', 'REJECTED') DEFAULT 'PENDING',
|
|
||||||
reviewed_by VARCHAR(100),
|
|
||||||
reviewed_at TIMESTAMP NULL,
|
|
||||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
UNIQUE KEY uq_doc_number (document_number)
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Error Log
|
|
||||||
CREATE TABLE IF NOT EXISTS migration_errors (
|
|
||||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
|
||||||
batch_id VARCHAR(50),
|
|
||||||
document_number VARCHAR(100),
|
|
||||||
error_type ENUM(
|
|
||||||
'FILE_NOT_FOUND',
|
|
||||||
'AI_PARSE_ERROR',
|
|
||||||
'API_ERROR',
|
|
||||||
'DB_ERROR',
|
|
||||||
'UNKNOWN'
|
|
||||||
),
|
|
||||||
error_message TEXT,
|
|
||||||
raw_ai_response TEXT,
|
|
||||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
INDEX idx_batch_id (batch_id),
|
|
||||||
INDEX idx_error_type (error_type)
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Fallback State
|
|
||||||
CREATE TABLE IF NOT EXISTS migration_fallback_state (
|
|
||||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
|
||||||
batch_id VARCHAR(50) UNIQUE,
|
|
||||||
recent_error_count INT DEFAULT 0,
|
|
||||||
is_fallback_active BOOLEAN DEFAULT FALSE,
|
|
||||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Idempotency (Patch)
|
|
||||||
CREATE TABLE IF NOT EXISTS import_transactions (
|
|
||||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
|
||||||
idempotency_key VARCHAR(255) UNIQUE NOT NULL,
|
|
||||||
document_number VARCHAR(100),
|
|
||||||
batch_id VARCHAR(100),
|
|
||||||
status_code INT DEFAULT 201,
|
|
||||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
INDEX idx_idem_key (idempotency_key)
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Daily Summary
|
|
||||||
CREATE TABLE IF NOT EXISTS migration_daily_summary (
|
|
||||||
id INT AUTO_INCREMENT PRIMARY KEY,
|
|
||||||
batch_id VARCHAR(50),
|
|
||||||
summary_date DATE,
|
|
||||||
total_processed INT DEFAULT 0,
|
|
||||||
auto_ingested INT DEFAULT 0,
|
|
||||||
sent_to_review INT DEFAULT 0,
|
|
||||||
rejected INT DEFAULT 0,
|
|
||||||
errors INT DEFAULT 0,
|
|
||||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
UNIQUE KEY uq_batch_date (batch_id, summary_date)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_corr_revisions_correspondence_current ON correspondence_revisions (correspondence_id, is_current);
|
CREATE INDEX idx_corr_revisions_correspondence_current ON correspondence_revisions (correspondence_id, is_current);
|
||||||
|
|
||||||
-- Indexes for v_current_rfas performance
|
-- Indexes for v_current_rfas performance
|
||||||
Reference in New Issue
Block a user