lcbp3/specs/08-Tasks/Task BE-AI-02.md

Task BE-AI-02: Backend AI Gateway Development

Phase: Step 2 - AI Integration Layer (NestJS) ADR Compliance: ADR-018 (AI Boundary), ADR-019 (UUID Strategy) Priority: 🔴 Critical - Bridge between DMS and AI Pipeline

Context: เป็นส่วนเชื่อมโยงระหว่างระบบ DMS และ AI Pipeline ตาม ADR-020 โดยต้องรักษาความปลอดภัยและใช้ Identifier ที่ถูกต้อง

---

🛠️ Implementation Tasks

AI-2.1: Database Schema Design (SQL First Approach)

• Create migration_logs Table:

  CREATE TABLE migration_logs (
    id INT AUTO_INCREMENT PRIMARY KEY,
    publicId BINARY(16) DEFAULT (UUID_TO_BIN(UUID(), 1)),
    source_file VARCHAR(255) NOT NULL,
    source_metadata JSON,
    ai_extracted_metadata JSON,
    confidence_score DECIMAL(3,2),
    status ENUM('PENDING_REVIEW', 'VERIFIED', 'IMPORTED', 'FAILED') DEFAULT 'PENDING_REVIEW',
    admin_feedback TEXT,
    reviewed_by INT NULL,
    reviewed_at TIMESTAMP NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
    INDEX idx_status (status),
    INDEX idx_confidence (confidence_score),
    INDEX idx_publicId (publicId)
  );
  

• Create ai_audit_logs Table:

  CREATE TABLE ai_audit_logs (
    id INT AUTO_INCREMENT PRIMARY KEY,
    publicId BINARY(16) DEFAULT (UUID_TO_BIN(UUID(), 1)),
    document_publicId BINARY(16),
    ai_model VARCHAR(50) NOT NULL,
    processing_time_ms INT,
    confidence_score DECIMAL(3,2),
    input_hash VARCHAR(64),
    output_hash VARCHAR(64),
    status ENUM('SUCCESS', 'FAILED', 'TIMEOUT') NOT NULL,
    error_message TEXT,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    INDEX idx_document (document_publicId),
    INDEX idx_model (ai_model),
    INDEX idx_status (status),
    FOREIGN KEY (document_publicId) REFERENCES migration_logs(publicId)
  );
  

• Update Data Dictionary:
  • Add field descriptions to specs/03-Data-and-Storage/03-01-data-dictionary.md
  • Include business rules for confidence thresholds
  • Document status transitions and workflows

AI-2.2: AI Gateway Module Architecture

• Module Structure:

  // src/modules/ai/ai.module.ts
  @Module({
    imports: [TypeOrmModule.forFeature([MigrationLog, AiAuditLog])],
    controllers: [AiController],
    providers: [AiService, AiValidationService],
    exports: [AiService],
  })
  export class AiModule {}
  

• AiService Implementation:

  @Injectable()
  export class AiService {
    async triggerProcessing(filePublicId: string, context: ProcessingContext): Promise<void> {
      // 1. Validate publicId format (ADR-019)
      // 2. Send HTTP request to n8n webhook
      // 3. Log request to ai_audit_logs
      // 4. Return processing token
    }
  
    async handleWebhookCallback(payload: AiCallbackDto): Promise<void> {
      // 1. Validate JWT token from n8n
      // 2. Update migration_logs with AI results
      // 3. Calculate confidence scores
      // 4. Trigger notifications if needed
    }
  
    async extractRealtime(filePublicId: string): Promise<ExtractionResult> {
      // 1. Send to n8n for immediate processing
      // 2. Wait for response (timeout: 30s)
      // 3. Return structured suggestions
    }
  }
  

• Configuration Management:

  # .env
  AI_N8N_WEBHOOK_URL=http://192.168.1.100:5678/webhook/ai-processing
  AI_N8N_AUTH_TOKEN=service-account-jwt-token
  AI_OLLAMA_URL=http://192.168.1.100:11434
  AI_TIMEOUT_MS=30000
  AI_MAX_RETRIES=3
  

AI-2.3: Migration Engine & Business Logic

• MigrationService Implementation:

  @Injectable()
  export class MigrationService {
    async stageLegacyData(excelData: ExcelImportDto[]): Promise<MigrationLog[]> {
      // 1. Validate Excel data format
      // 2. Move PDF files to staging area (via StorageService)
      // 3. Create migration_logs entries
      // 4. Trigger AI processing for each file
    }
  
    async compareData(excelMetadata: any, aiMetadata: any): Promise<ComparisonResult> {
      // 1. Field-by-field comparison
      // 2. Calculate confidence deltas
      // 3. Flag discrepancies for human review
      // 4. Generate comparison report
    }
  
    async approveMigration(migrationPublicId: string, adminId: number): Promise<void> {
      // 1. Validate admin permissions (CASL)
      // 2. Move file from staging to permanent storage
      // 3. Create actual document records (RFA, Correspondence, etc.)
      // 4. Update migration_logs status
    }
  }
  

• Status Management Workflow:

  enum MigrationStatus {
    PENDING_REVIEW = 'PENDING_REVIEW',
    VERIFIED = 'VERIFIED',
    IMPORTED = 'IMPORTED',
    FAILED = 'FAILED'
  }
  
  // State transition rules
  const statusTransitions = {
    [MigrationStatus.PENDING_REVIEW]: [MigrationStatus.VERIFIED, MigrationStatus.FAILED],
    [MigrationStatus.VERIFIED]: [MigrationStatus.IMPORTED, MigrationStatus.PENDING_REVIEW],
    [MigrationStatus.IMPORTED]: [], // Terminal state
    [MigrationStatus.FAILED]: [MigrationStatus.PENDING_REVIEW] // Can retry
  };
  

AI-2.4: API Endpoints & Security Implementation

• Admin Migration Endpoints:

  @Controller('admin/migration')
  @UseGuards(JwtAuthGuard, CaslGuard)
  export class AdminMigrationController {
    @Get()
    @Permissions(PERMISSIONS.MIGRATION_READ)
    async getMigrationList(@Query() query: MigrationQueryDto): Promise<PaginatedResult<MigrationLog>> {
      // 1. Validate query parameters
      // 2. Apply filters (status, confidence, date range)
      // 3. Return paginated results
    }
  
    @Patch(':publicId')
    @Permissions(PERMISSIONS.MIGRATION_APPROVE)
    async updateMigration(
      @Param('publicId') publicId: string,
      @Body() updateDto: MigrationUpdateDto,
      @CurrentUser() user: User
    ): Promise<MigrationLog> {
      // 1. Validate publicId (no parseInt!)
      // 2. Check admin permissions
      // 3. Update with audit trail
    }
  }
  

• Real-time AI Extraction Endpoint:

  @Controller('ai')
  export class AiController {
    @Post('extract')
    @UseGuards(JwtAuthGuard)
    @Throttle(5, 60) // 5 requests per minute
    async extractDocument(@Body() dto: ExtractDocumentDto): Promise<ExtractionResult> {
      // 1. Validate file access permissions
      // 2. Send to AI pipeline
      // 3. Return structured suggestions
    }
  }
  

• Security Measures:
  • CASL permissions for all endpoints
  • Idempotency-Key header validation
  • Rate limiting on AI endpoints
  • JWT authentication for service accounts
  • Request/response logging for audit

---

🔴 Critical Rules (Non-Negotiable)

1. ADR-019 UUID Strategy:

   • Use publicId (UUIDv7) for all document references
   • NEVER use parseInt() or Number() on UUID values
   • All API parameters use string type for UUIDs

2. ADR-018 AI Boundary:

   • No direct database access from AI services
   • All communication via DMS API only
   • AI services run on Admin Desktop (isolated)

3. Security Requirements:

   • All POST/PATCH endpoints must validate Idempotency-Key
   • CASL permissions enforced on all endpoints
   • Rate limiting on AI endpoints (5 req/min)

4. Data Integrity:

   • SQL-first approach (no TypeORM migrations)
   • All file operations via StorageService
   • Audit logging for all AI interactions

---

📋 Implementation Sequence

1. Phase 1 (AI-2.1): Database schema and data dictionary updates
2. Phase 2 (AI-2.2): AI Gateway module and basic service structure
3. Phase 3 (AI-2.3 & AI-2.4): Business logic and API endpoints (parallel development)
4. Phase 4: Integration testing with n8n pipeline

---

📁 Related Specifications

• ADR-018: AI Boundary Policy - Security requirements
• ADR-019: Hybrid Identifier Strategy - UUID patterns
• ADR-020: AI Intelligence Integration - Architecture overview
• 05-02-backend-guidelines.md: NestJS patterns and conventions
• 03-01-data-dictionary.md: Field definitions and business rules

---

📝 Code Templates

DTO Examples

// extract-document.dto.ts
export class ExtractDocumentDto {
  @IsUUID()
  publicId: string;

  @IsEnum(['migration', 'ingestion'])
  context: string;
}

// migration-update.dto.ts
export class MigrationUpdateDto {
  @IsOptional()
  @IsEnum(['VERIFIED', 'FAILED'])
  status?: MigrationStatus;

  @IsOptional()
  @IsString()
  @MaxLength(1000)
  adminFeedback?: string;
}

Entity Example

// migration-log.entity.ts
@Entity('migration_logs')
export class MigrationLog extends UuidBaseEntity {
  @Column({ type: 'varchar', length: 255 })
  sourceFile: string;

  @Column({ type: 'json' })
  sourceMetadata: any;

  @Column({ type: 'json' })
  aiExtractedMetadata: any;

  @Column({ type: 'decimal', precision: 3, scale: 2 })
  confidenceScore: number;

  @Column({
    type: 'enum',
    enum: MigrationStatus,
    default: MigrationStatus.PENDING_REVIEW
  })
  status: MigrationStatus;
}