296 lines
9.1 KiB
Markdown
296 lines
9.1 KiB
Markdown
# Task BE-AI-02: Backend AI Gateway Development
|
|
|
|
**Phase:** Step 2 - AI Integration Layer (NestJS)
|
|
**ADR Compliance:** ADR-018 (AI Boundary), ADR-019 (UUID Strategy)
|
|
**Priority:** 🔴 Critical - Bridge between DMS and AI Pipeline
|
|
|
|
> **Context:** เป็นส่วนเชื่อมโยงระหว่างระบบ DMS และ AI Pipeline ตาม ADR-020 โดยต้องรักษาความปลอดภัยและใช้ Identifier ที่ถูกต้อง
|
|
|
|
---
|
|
|
|
## 🛠️ Implementation Tasks
|
|
|
|
### **AI-2.1: Database Schema Design (SQL First Approach)**
|
|
- [ ] **Create `migration_logs` Table:**
|
|
```sql
|
|
CREATE TABLE migration_logs (
|
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
|
publicId BINARY(16) DEFAULT (UUID_TO_BIN(UUID(), 1)),
|
|
source_file VARCHAR(255) NOT NULL,
|
|
source_metadata JSON,
|
|
ai_extracted_metadata JSON,
|
|
confidence_score DECIMAL(3,2),
|
|
status ENUM('PENDING_REVIEW', 'VERIFIED', 'IMPORTED', 'FAILED') DEFAULT 'PENDING_REVIEW',
|
|
admin_feedback TEXT,
|
|
reviewed_by INT NULL,
|
|
reviewed_at TIMESTAMP NULL,
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
|
|
INDEX idx_status (status),
|
|
INDEX idx_confidence (confidence_score),
|
|
INDEX idx_publicId (publicId)
|
|
);
|
|
```
|
|
- [ ] **Create `ai_audit_logs` Table:**
|
|
```sql
|
|
CREATE TABLE ai_audit_logs (
|
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
|
publicId BINARY(16) DEFAULT (UUID_TO_BIN(UUID(), 1)),
|
|
document_publicId BINARY(16),
|
|
ai_model VARCHAR(50) NOT NULL,
|
|
processing_time_ms INT,
|
|
confidence_score DECIMAL(3,2),
|
|
input_hash VARCHAR(64),
|
|
output_hash VARCHAR(64),
|
|
status ENUM('SUCCESS', 'FAILED', 'TIMEOUT') NOT NULL,
|
|
error_message TEXT,
|
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
INDEX idx_document (document_publicId),
|
|
INDEX idx_model (ai_model),
|
|
INDEX idx_status (status),
|
|
FOREIGN KEY (document_publicId) REFERENCES migration_logs(publicId)
|
|
);
|
|
```
|
|
- [ ] **Update Data Dictionary:**
|
|
- Add field descriptions to `specs/03-Data-and-Storage/03-01-data-dictionary.md`
|
|
- Include business rules for confidence thresholds
|
|
- Document status transitions and workflows
|
|
|
|
### **AI-2.2: AI Gateway Module Architecture**
|
|
- [ ] **Module Structure:**
|
|
```typescript
|
|
// src/modules/ai/ai.module.ts
|
|
@Module({
|
|
imports: [TypeOrmModule.forFeature([MigrationLog, AiAuditLog])],
|
|
controllers: [AiController],
|
|
providers: [AiService, AiValidationService],
|
|
exports: [AiService],
|
|
})
|
|
export class AiModule {}
|
|
```
|
|
- [ ] **AiService Implementation:**
|
|
```typescript
|
|
@Injectable()
|
|
export class AiService {
|
|
async triggerProcessing(filePublicId: string, context: ProcessingContext): Promise<void> {
|
|
// 1. Validate publicId format (ADR-019)
|
|
// 2. Send HTTP request to n8n webhook
|
|
// 3. Log request to ai_audit_logs
|
|
// 4. Return processing token
|
|
}
|
|
|
|
async handleWebhookCallback(payload: AiCallbackDto): Promise<void> {
|
|
// 1. Validate JWT token from n8n
|
|
// 2. Update migration_logs with AI results
|
|
// 3. Calculate confidence scores
|
|
// 4. Trigger notifications if needed
|
|
}
|
|
|
|
async extractRealtime(filePublicId: string): Promise<ExtractionResult> {
|
|
// 1. Send to n8n for immediate processing
|
|
// 2. Wait for response (timeout: 30s)
|
|
// 3. Return structured suggestions
|
|
}
|
|
}
|
|
```
|
|
- [ ] **Configuration Management:**
|
|
```env
|
|
# .env
|
|
AI_N8N_WEBHOOK_URL=http://192.168.1.100:5678/webhook/ai-processing
|
|
AI_N8N_AUTH_TOKEN=service-account-jwt-token
|
|
AI_OLLAMA_URL=http://192.168.1.100:11434
|
|
AI_TIMEOUT_MS=30000
|
|
AI_MAX_RETRIES=3
|
|
```
|
|
|
|
### **AI-2.3: Migration Engine & Business Logic**
|
|
- [ ] **MigrationService Implementation:**
|
|
```typescript
|
|
@Injectable()
|
|
export class MigrationService {
|
|
async stageLegacyData(excelData: ExcelImportDto[]): Promise<MigrationLog[]> {
|
|
// 1. Validate Excel data format
|
|
// 2. Move PDF files to staging area (via StorageService)
|
|
// 3. Create migration_logs entries
|
|
// 4. Trigger AI processing for each file
|
|
}
|
|
|
|
async compareData(excelMetadata: any, aiMetadata: any): Promise<ComparisonResult> {
|
|
// 1. Field-by-field comparison
|
|
// 2. Calculate confidence deltas
|
|
// 3. Flag discrepancies for human review
|
|
// 4. Generate comparison report
|
|
}
|
|
|
|
async approveMigration(migrationPublicId: string, adminId: number): Promise<void> {
|
|
// 1. Validate admin permissions (CASL)
|
|
// 2. Move file from staging to permanent storage
|
|
// 3. Create actual document records (RFA, Correspondence, etc.)
|
|
// 4. Update migration_logs status
|
|
}
|
|
}
|
|
```
|
|
- [ ] **Status Management Workflow:**
|
|
```typescript
|
|
enum MigrationStatus {
|
|
PENDING_REVIEW = 'PENDING_REVIEW',
|
|
VERIFIED = 'VERIFIED',
|
|
IMPORTED = 'IMPORTED',
|
|
FAILED = 'FAILED'
|
|
}
|
|
|
|
// State transition rules
|
|
const statusTransitions = {
|
|
[MigrationStatus.PENDING_REVIEW]: [MigrationStatus.VERIFIED, MigrationStatus.FAILED],
|
|
[MigrationStatus.VERIFIED]: [MigrationStatus.IMPORTED, MigrationStatus.PENDING_REVIEW],
|
|
[MigrationStatus.IMPORTED]: [], // Terminal state
|
|
[MigrationStatus.FAILED]: [MigrationStatus.PENDING_REVIEW] // Can retry
|
|
};
|
|
```
|
|
|
|
### **AI-2.4: API Endpoints & Security Implementation**
|
|
- [ ] **Admin Migration Endpoints:**
|
|
```typescript
|
|
@Controller('admin/migration')
|
|
@UseGuards(JwtAuthGuard, CaslGuard)
|
|
export class AdminMigrationController {
|
|
@Get()
|
|
@Permissions(PERMISSIONS.MIGRATION_READ)
|
|
async getMigrationList(@Query() query: MigrationQueryDto): Promise<PaginatedResult<MigrationLog>> {
|
|
// 1. Validate query parameters
|
|
// 2. Apply filters (status, confidence, date range)
|
|
// 3. Return paginated results
|
|
}
|
|
|
|
@Patch(':publicId')
|
|
@Permissions(PERMISSIONS.MIGRATION_APPROVE)
|
|
async updateMigration(
|
|
@Param('publicId') publicId: string,
|
|
@Body() updateDto: MigrationUpdateDto,
|
|
@CurrentUser() user: User
|
|
): Promise<MigrationLog> {
|
|
// 1. Validate publicId (no parseInt!)
|
|
// 2. Check admin permissions
|
|
// 3. Update with audit trail
|
|
}
|
|
}
|
|
```
|
|
- [ ] **Real-time AI Extraction Endpoint:**
|
|
```typescript
|
|
@Controller('ai')
|
|
export class AiController {
|
|
@Post('extract')
|
|
@UseGuards(JwtAuthGuard)
|
|
@Throttle(5, 60) // 5 requests per minute
|
|
async extractDocument(@Body() dto: ExtractDocumentDto): Promise<ExtractionResult> {
|
|
// 1. Validate file access permissions
|
|
// 2. Send to AI pipeline
|
|
// 3. Return structured suggestions
|
|
}
|
|
}
|
|
```
|
|
- [ ] **Security Measures:**
|
|
- CASL permissions for all endpoints
|
|
- Idempotency-Key header validation
|
|
- Rate limiting on AI endpoints
|
|
- JWT authentication for service accounts
|
|
- Request/response logging for audit
|
|
|
|
---
|
|
|
|
## 🔴 Critical Rules (Non-Negotiable)
|
|
|
|
1. **ADR-019 UUID Strategy:**
|
|
- Use `publicId` (UUIDv7) for all document references
|
|
- NEVER use `parseInt()` or `Number()` on UUID values
|
|
- All API parameters use string type for UUIDs
|
|
|
|
2. **ADR-018 AI Boundary:**
|
|
- No direct database access from AI services
|
|
- All communication via DMS API only
|
|
- AI services run on Admin Desktop (isolated)
|
|
|
|
3. **Security Requirements:**
|
|
- All `POST/PATCH` endpoints must validate `Idempotency-Key`
|
|
- CASL permissions enforced on all endpoints
|
|
- Rate limiting on AI endpoints (5 req/min)
|
|
|
|
4. **Data Integrity:**
|
|
- SQL-first approach (no TypeORM migrations)
|
|
- All file operations via StorageService
|
|
- Audit logging for all AI interactions
|
|
|
|
---
|
|
|
|
## 📋 Implementation Sequence
|
|
|
|
1. **Phase 1 (AI-2.1):** Database schema and data dictionary updates
|
|
2. **Phase 2 (AI-2.2):** AI Gateway module and basic service structure
|
|
3. **Phase 3 (AI-2.3 & AI-2.4):** Business logic and API endpoints (parallel development)
|
|
4. **Phase 4:** Integration testing with n8n pipeline
|
|
|
|
---
|
|
|
|
## 📁 Related Specifications
|
|
|
|
- **ADR-018:** AI Boundary Policy - Security requirements
|
|
- **ADR-019:** Hybrid Identifier Strategy - UUID patterns
|
|
- **ADR-020:** AI Intelligence Integration - Architecture overview
|
|
- **05-02-backend-guidelines.md:** NestJS patterns and conventions
|
|
- **03-01-data-dictionary.md:** Field definitions and business rules
|
|
|
|
---
|
|
|
|
## 📝 Code Templates
|
|
|
|
### DTO Examples
|
|
```typescript
|
|
// extract-document.dto.ts
|
|
export class ExtractDocumentDto {
|
|
@IsUUID()
|
|
publicId: string;
|
|
|
|
@IsEnum(['migration', 'ingestion'])
|
|
context: string;
|
|
}
|
|
|
|
// migration-update.dto.ts
|
|
export class MigrationUpdateDto {
|
|
@IsOptional()
|
|
@IsEnum(['VERIFIED', 'FAILED'])
|
|
status?: MigrationStatus;
|
|
|
|
@IsOptional()
|
|
@IsString()
|
|
@MaxLength(1000)
|
|
adminFeedback?: string;
|
|
}
|
|
```
|
|
|
|
### Entity Example
|
|
```typescript
|
|
// migration-log.entity.ts
|
|
@Entity('migration_logs')
|
|
export class MigrationLog extends UuidBaseEntity {
|
|
@Column({ type: 'varchar', length: 255 })
|
|
sourceFile: string;
|
|
|
|
@Column({ type: 'json' })
|
|
sourceMetadata: any;
|
|
|
|
@Column({ type: 'json' })
|
|
aiExtractedMetadata: any;
|
|
|
|
@Column({ type: 'decimal', precision: 3, scale: 2 })
|
|
confidenceScore: number;
|
|
|
|
@Column({
|
|
type: 'enum',
|
|
enum: MigrationStatus,
|
|
default: MigrationStatus.PENDING_REVIEW
|
|
})
|
|
status: MigrationStatus;
|
|
}
|
|
```
|
|
|