690618:1444 237 #02

2026-06-18 14:44:46 +07:00
parent 037fbb65f5
commit 09e304de84
52 changed files with 4471 additions and 1038 deletions
@@ -70,6 +70,12 @@ This meta-workflow orchestrates the **complete development lifecycle**, from spe
 /speckit.all "Build a user authentication system with OAuth2 support"
 ```

+For OCR & AI Extraction prompt management (ADR-037 3-Step Pipeline), use the specialized workflow:
+
+```
+/speckit.ocr-prompt-management
+```
+
 ## Pipeline Comparison

 | Pipeline           | Steps                     | Use When                               |
@@ -26,3 +26,15 @@ This workflow orchestrates the sequential execution of the Speckit preparation p
 5. **Step 5: Analyze (Skill 06)**
   - Goal: Validate consistency across all design artifacts (spec, plan, tasks).
   - Action: Read and execute `.agents/skills/speckit-analyze/SKILL.md`.
+
+## OCR-Specific Considerations
+
+For OCR & AI Extraction prompt management features (ADR-037), consider:
+
+- **Infrastructure**: Verify OCR sidecar (Desk-5439) and `/embed` endpoint availability
+- **Database**: Check for `ai_prompts` table with `version` column and required deltas
+- **Sidecar Integration**: Plan for system prompt threading through OCR endpoints
+- **3-Step Pipeline**: Design for sequential execution (OCR → AI Extract → RAG Prep)
+- **Optimistic Locking**: Include version conflict handling in prompt activation flows
+
+For specialized OCR workflows, use `/speckit.ocr-prompt-management` instead.
@@ -17,3 +17,44 @@ description: Execute the implementation planning workflow using the plan templat

 4. **On Error**:
   - If `spec.md` is missing: Run `/speckit.specify` first to create the feature specification
+
+## OCR-Specific Planning Considerations
+
+When planning OCR & AI Extraction prompt management features (ADR-037), include:
+
+### Infrastructure Planning
+
+- **OCR Sidecar**: Verify Desk-5439 sidecar availability (port 8765)
+- **Endpoints**: Plan for `/ocr-upload`, `/embed`, and `/normalize` endpoints
+- **Environment Variables**: Document required env vars (OCR_SIDECAR_API_KEY, OCR_API_URL)
+- **Network**: Verify VLAN 10 connectivity between backend and Desk-5439
+
+### Database Planning
+
+- **Schema Changes**: Use SQL deltas per ADR-009 (no TypeORM migrations)
+- **Version Column**: Verify `ai_prompts` table has `version` column
+- **Entity Mapping**: Ensure `@VersionColumn()` in `ai-prompts.entity.ts`
+- **Seed Data**: Plan for default OCR system prompt seed
+
+### Service Architecture
+
+- **Validation Service**: Extend existing `ai-prompts.service.ts` for prompt validation
+- **Optimistic Locking**: Plan version conflict handling (409 Conflict responses)
+- **Prompt Resolution**: Design `resolveActive()` for template placeholder substitution
+- **BullMQ Integration**: Plan queue jobs for OCR, extraction, and RAG prep
+
+### 3-Step Pipeline Design
+
+- **Sequential Execution**: Design OCR → AI Extract → RAG Prep flow
+- **State Tracking**: Plan Redis-based pipeline status tracking
+- **Input/Output Contract**: Define data flow between pipeline steps
+- **Error Recovery**: Design rollback and retry mechanisms
+
+### Frontend Planning
+
+- **Tab Structure**: Plan separate tabs for OCR, AI Extraction, and Sandbox
+- **Version History**: Design version list display and activation UI
+- **Validation UI**: Plan inline validation error display
+- **Vector Preview**: Design chunk list and vector dimension display (5 dims)
+
+For specialized OCR workflows, use `/speckit.ocr-prompt-management` instead.
@@ -19,3 +19,51 @@ description: Execute the implementation plan by processing and executing all tas
   - If `tasks.md` is missing: Run `/speckit.tasks` first
   - If `plan.md` is missing: Run `/speckit.plan` first
   - If `spec.md` is missing: Run `/speckit.specify` first
+
+## OCR-Specific Implementation Considerations
+
+When implementing OCR & AI Extraction prompt management features (ADR-037), handle:
+
+### Sidecar Integration
+
+- **System Prompt Threading**: Append system prompt to `messages[0]["content"]` in sidecar (typhoon OCR single-message format)
+- **API Key Authentication**: Send `X-API-Key: $OCR_SIDECAR_API_KEY` header to sidecar endpoints
+- **Path Remapping**: Handle backend → sidecar path mapping (e.g., `/app/uploads/temp` → `/mnt/uploads/temp`)
+- **Error Handling**: Implement retry logic for sidecar connection failures
+
+### Database Implementation
+
+- **SQL Deltas**: Apply schema changes via SQL deltas per ADR-009 (no TypeORM migrations)
+- **Version Column**: Verify `ai_prompts.version` column exists and entity has `@VersionColumn()`
+- **Seed Data**: Apply delta for default OCR system prompt (INSERT with `prompt_type='ocr_system'`)
+
+### Service Implementation
+
+- **Optimistic Locking**: Modify `activate()` to accept `expectedVersion` parameter
+- **409 Conflict Handling**: Return proper HTTP 409 when version mismatch occurs
+- **Prompt Validation**: Extend `create()` to support `ocr_system` (free-form) and `ocr_extraction` ({{ocr_text}} required)
+- **Prompt Resolution**: Use `resolveActive()` for template placeholder substitution
+
+### BullMQ Integration
+
+- **Queue Jobs**: Implement handlers for `sandbox-ocr`, `sandbox-extract`, `sandbox-rag-prep`
+- **Sequential Execution**: Wire Step 2 output as Step 3 input
+- **State Tracking**: Store pipeline status in Redis
+- **Error Recovery**: Implement rollback mechanisms for failed pipeline steps
+
+### Frontend Implementation
+
+- **Service Layer**: Create `adminAiPromptService` with optimistic locking support
+- **Tab Components**: Implement `PromptManagementTabs`, `OcrPromptTab`, `AiExtractionPromptTab`
+- **Version History**: Display version list with activation status
+- **Validation UI**: Show inline errors for missing placeholders
+- **Vector Preview**: Display chunk list with first 5 dimensions
+- **Step Indicators**: Implement 3-step status display (pending/processing/completed/failed)
+
+### Testing Implementation
+
+- **Unit Tests**: Test prompt validation, optimistic locking, version conflict scenarios
+- **Integration Tests**: Test full 3-step pipeline end-to-end
+- **E2E Tests**: Test admin UI workflows (create prompt → activate → run sandbox)
+
+For specialized OCR workflows, use `/speckit.ocr-prompt-management` instead.
@@ -0,0 +1,147 @@
+---
+auto_execution_mode: 0
+description: Execute OCR & AI Extraction prompt management workflow following ADR-037 3-Step Pipeline (OCR → AI Extract → RAG Prep)
+---
+
+# Workflow: speckit.ocr-prompt-management
+
+This workflow orchestrates the **OCR & AI Extraction prompt management** feature implementation, following the 3-step pipeline pattern defined in ADR-037.
+
+## Phase 1: Database & Infrastructure Setup
+
+1. **Database Schema**:
+   - Verify `version` column exists in `ai_prompts` table (delta: `2026-06-15-fix-ai-prompts-columns.sql`)
+   - Seed default OCR system prompt (delta: `2026-06-17-seed-ocr-system-prompt.sql`)
+   - Verify entity has `@VersionColumn()` at `backend/src/modules/ai/prompts/ai-prompts.entity.ts`
+
+2. **Infrastructure Verification**:
+   - Verify OCR sidecar is running on Desk-5439 (port 8765)
+   - Verify `/embed` endpoint exists in sidecar
+   - Verify environment variables: `OCR_SIDECAR_API_KEY`, `OCR_API_URL`
+
+## Phase 2: Foundational Services
+
+1. **Validation Service**:
+   - Extend `ai-prompts.service.ts` `create()` to support `ocr_system` (free-form, no required placeholder)
+   - Verify `{{ocr_text}}` placeholder validation for `ocr_extraction`
+   - Use existing DTOs: `CreateAiPromptDto`, `UpdatePromptNoteDto`, `ContextConfigDto`
+
+2. **Optimistic Locking**:
+   - Modify `activate()` in `ai-prompts.service.ts` to accept `expectedVersion`
+   - Handle HTTP 409 Conflict when version mismatch occurs
+   - Add retry logic with exponential backoff in frontend
+
+## Phase 3: User Story 1 - OCR System Prompt Management
+
+### Backend
+- Verify `AiPromptsService.create()` supports `ocr_system` (version auto-increment)
+- Verify `getActive(promptType)` returns active ocr_system with Redis cache (60s)
+- Verify existing routes: GET `/api/ai/prompts/{promptType}`, POST `/api/ai/prompts/{promptType}`, POST `/api/ai/prompts/{promptType}/{versionNumber}/activate`
+
+### Frontend
+- Create `adminAiPromptService` in `frontend/lib/services/admin-ai-prompt.service.ts`
+- Implement `getPrompts()`, `createPrompt()`, `activatePrompt()` with optimistic locking
+- Create `PromptManagementTabs` component
+- Create `OcrPromptTab` component with text editor and version history
+- Implement "Save New Version" button with validation
+- Handle 409 Conflict error - show refresh dialog
+
+### Sidecar Integration
+- Update `/ocr-upload` endpoint in `specs/04-Infrastructure-OPS/04-00-docker-compose/Desk-5439/ocr-sidecar/app.py`:
+  - Add parameter: `systemPrompt: Optional[str] = Form(default=None)`
+  - Thread `systemPrompt` through `_process_pdf_doc()` → `process_ocr()`
+  - Append system prompt to `messages[0]["content"]` (typhoon OCR single-message format)
+- Update `sandbox-ocr-engine.service.ts` to fetch active `ocr_system` prompt and send to sidecar
+
+## Phase 4: User Story 2 - AI Extraction Prompt Management
+
+### Backend
+- Verify `ocr_extraction` validation in `create()` ({{ocr_text}} required)
+- Verify `resolveActive('ocr_extraction', ocrText)` exists
+- Verify `ai-batch.processor.ts` uses active `ocr_extraction` prompt
+
+### Frontend
+- Create `AiExtractionPromptTab` component
+- Add placeholder helper buttons ({{ocr_text}}, {{master_data_context}})
+- Show validation error inline if missing required placeholder
+- Add template preview with syntax highlighting
+
+## Phase 5: User Story 3 - Separate UI Tabs
+
+### Frontend UI Polish
+- Style `PromptManagementTabs` with clear tab indicators
+- Add tab icons (OCR: eye/scan icon, AI: brain/robot icon)
+- Show active status badge on each tab
+- Implement tab state persistence (URL hash or localStorage)
+- Add warning badge if no active prompt for a type
+
+## Phase 6: User Story 4 - Full 3-Step Sandbox with RAG Prep
+
+### Backend (RAG Prep Integration)
+- Verify `rag_prep_prompt` validates `{{text}}` placeholder
+- Verify `SandboxRagPrepDto` exists at `backend/src/modules/ai/dto/sandbox-rag-prep.dto.ts`
+- Extend `ai-batch.processor.ts` `sandbox-rag-prep` job handler
+- Implement semantic chunking using active `rag_prep_prompt`
+- Verify sidecar `/embed` endpoint exists
+- Verify POST `/api/ai/admin/sandbox/rag-prep` exists in AiController
+- Verify Redis storage for RAG Prep results
+- Verify GET sandbox job result endpoint (`/api/ai/admin/sandbox/job/:id`)
+
+### Frontend (3-Step Sandbox UI)
+- Create `SandboxStepIndicator` component showing 3 steps with status icons
+- Extend `PromptManagementTabs` with "Sandbox" tab containing 3-step workflow
+- Create `RagPrepResultPanel` component with chunk list + vector preview
+- Implement vector preview display (first 5 dimensions: `[0.234, -0.891, ...]`)
+- Add "Run Step 3 (RAG Prep)" button enabled after Step 2 completes
+- Display chunk count and embedding status for each chunk
+- Add "Activate This Version" button visible after all 3 steps complete successfully
+
+### Integration (Full Pipeline)
+- Wire Step 2 output (extracted metadata + text) as Step 3 input
+- Implement sequential step execution (Step 1 → Step 2 → Step 3)
+- Add pipeline status tracking in Redis
+
+## Phase 7: Testing & Validation
+
+### Error Handling (ADR-007)
+- Add user-friendly error messages for validation errors in frontend
+- Implement retry logic for 409 Conflict with exponential backoff
+- Add Toast notifications for success/error states
+
+### Testing
+- Write unit tests for `AiPromptValidationService`
+- Write integration test for optimistic locking conflict scenario
+- E2E test: Admin creates OCR prompt → activates → runs Sandbox Step 1
+- E2E test: Full 3-step pipeline - upload PDF → OCR → Extract → RAG Prep
+- E2E test: Vector preview displays correctly with 5 dimensions
+- E2E test: Step indicators show correct status for each step
+
+## Usage
+
+```
+/speckit.ocr-prompt-management
+```
+
+## Dependencies
+
+- **Phase 1**: None (infrastructure setup)
+- **Phase 2**: Phase 1
+- **Phase 3**: Phase 2
+- **Phase 4**: Phase 2 (can run in parallel with Phase 3)
+- **Phase 5**: Phase 3 + Phase 4
+- **Phase 6**: Phase 3 + Phase 4
+- **Phase 7**: Phase 6
+
+## On Error
+
+If any phase fails, stop and report:
+- Which phase failed
+- The specific task that failed
+- Suggested remediation (e.g., "Verify OCR sidecar is running before Phase 3")
+
+## Related ADRs
+
+- **ADR-009**: Database schema changes (SQL deltas, no TypeORM migrations)
+- **ADR-016**: Security authentication (RBAC for admin-only endpoints)
+- **ADR-023/023A**: AI architecture (BullMQ queues, Ollama isolation)
+- **ADR-037**: 3-Step Pipeline (OCR → AI Extract → RAG Prep)