lcbp3/specs/200-fullstacks/237-unified-prompt-management-ux-ui/validation-report.md

Validation Report: Unified Prompt Management UX/UI (ADR-037)

Date: 2026-06-15 Status: PARTIAL — 3 gaps require action before sign-off Feature: 237-unified-prompt-management-ux-ui Validated Against: spec.md, tasks.md, plan.md, ADR-037

---

Coverage Summary

Metric	Count	Percentage
Functional Requirements	25/29	86%
Acceptance Criteria Met	16/18	89%
Edge Cases Handled	9/12	75%
Tests Present	8/10	80%

---

✅ Verified — Implemented Correctly

Phase 1 — Database Setup

Task	File	Status
T001	deltas/2026-06-14-create-ai-execution-profiles.sql	✅
T002	deltas/2026-06-14-seed-execution-profiles.sql	✅
T003	deltas/2026-06-14-seed-additional-prompt-types.sql	✅

Phase 2 — Foundational

• AiExecutionProfile entity — ✅ correct columns, no @VersionColumn needed (not required)
• AiPrompt entity — ✅ @VersionColumn added (T066)
• ContextConfigDto, SandboxRagPrepDto, CreateExecutionProfileDto, UpdateExecutionProfileDto — ✅ all present
• AiModule registered AiExecutionProfile — ✅

Phase 3 — User Story 1 (Multi-Type Prompt Management)

• AiPromptsService.create() — ✅ validates placeholders for all 4 types; increments version per promptType
• PromptTypeDropdown component — ✅ exists
• VersionHistory component — ✅ showAllTypes prop, grouped view, pagination (20/page)
• PromptEditor component — ✅ live placeholder validation via PLACEHOLDER_REQUIREMENTS
• prompt-management/page.tsx — ✅ 2-column responsive layout (Tailwind lg:col-span-4/8)
• i18n keys for th and en — ✅ present in ai.json / common.json

Phase 4 — User Story 2 (Context Config Management)

• GET /api/ai/prompts/:type/:version/context-config — ✅ implemented with CASL guard
• PUT /api/ai/prompts/:type/:version/context-config — ✅ with Idempotency-Key + CASL + audit
• AiPromptsService.getContextConfig() / updateContextConfig() — ✅
• Context config validation: pageSize (1–1000), language required, project/contract UUID existence — ✅
• Optimistic locking (@VersionColumn) + error mapping to BusinessException — ✅
• ContextConfigEditor component — ✅

Phase 5 — User Story 3 (3-Step Sandbox)

• POST /api/ai/admin/sandbox/ocr — ✅ (Step 1)
• POST /api/ai/admin/sandbox/extract — ✅ (Step 2, maps to "sandbox-extract" job)
• POST /api/ai/admin/sandbox/rag-prep — ✅ added 2026-06-14 (Step 3)
• GET /api/ai/admin/sandbox/job/:id — ✅ with 300 req/min throttle
• SandboxTabs — ✅ 3-step sequential flow: OCR → Extract → RAG Prep with step guards
• "Activate This Version" button in sandbox results — ✅ (handleActivate wired to onActivateVersion)

Phase 6 — User Story 4 (Runtime Parameters Separation)

• AiExecutionProfilesService — ✅
• GET/POST/PUT/DELETE /api/ai/execution-profiles — ✅ with CASL guards
• RuntimeParametersPanel component — ✅ labelled "Runtime Parameters (Global - Applies to All AI Jobs)"
• Integrated into Sandbox tab (separate from Context Config) — ✅

Phase 7 — Polish

• ADR-007 layered error handling in page mutations — ✅ (toast with userMessage + recoveryAction)
• CASL guard on all mutation endpoints — ✅
• Redis cache invalidation on activation — ✅ (both active:type and versions:type keys deleted)
• Block deletion of active version — ✅ (CANNOT_DELETE_ACTIVE_PROMPT BusinessException)
• SELECT FOR UPDATE concurrent activation — ✅

Phase 8 — Grilling Session Resolutions

• "All Types" option in PromptTypeDropdown — ✅
• "All Types" grouped view in VersionHistory — ✅
• @VersionColumn on AiPrompt entity — ✅ (T066)
• Context config field validation backend — ✅ (T068)
• Responsive design breakpoints in page — ✅ (grid-cols-1 lg:grid-cols-12)
• "Runtime Parameters (Global...)" label — ✅
• ADR-007 layered Toast/Inline errors in page — ✅
• Redis cache (60s TTL) for version history — ✅ (setex(cacheKey, 60, ...))
• Pagination (20 versions/page) in VersionHistory — ✅
• Database SELECT FOR UPDATE for activation — ✅
• Block active version deletion — ✅
• Redis TTL (60m) for sandbox results — to be confirmed (see gap below)

---

⚠️ Gaps Identified

GAP-1: Placeholder Validation Mismatch — Backend vs Spec [MEDIUM]

FR-023 / FR-024 / FR-026 violation

Prompt Type	Spec Required Placeholders	Backend Checks	Frontend Checks
ocr_extraction	{{ocr_text}} (req), {{master_data_context}} (opt)	{{ocr_text}} only ✅	{{ocr_text}}, {{master_data_context}} both required ❌
rag_query_prompt	{{user_query}} (req), {{retrieved_chunks}} (req)	{{query}} + {{context}} ❌	{{query}} + {{context}} ❌
rag_prep_prompt	{{document_text}} (req)	{{text}} ❌	{{text}} ❌
classification_prompt	{{document_metadata}} (req), {{document_text}} (opt)	{{document_text}} only ❌	{{document_text}} only ❌

Spec FR-023–FR-026 defines exact placeholder names that differ from what was implemented. Additionally, {{master_data_context}} is marked "optional" in the spec but PLACEHOLDER_REQUIREMENTS requires it (making it a required validation that blocks save).

Impact: Incorrect placeholder names mean production prompts using spec-defined names ({{user_query}}, {{retrieved_chunks}}, {{document_text}} for rag_prep, {{document_metadata}}) will fail validation and cannot be saved.

Recommendation: Decide canonical placeholder names — align spec or align code. Suggested: update spec FR-023–FR-026 to reflect implemented names ({{query}}, {{context}}, {{text}}) since these are used in actual production seed data. Also remove {{master_data_context}} from required list in PLACEHOLDER_REQUIREMENTS (mark as optional per spec).

---

GAP-2: Mobile Collapsible Accordion (T071) — Not Implemented [LOW]

FR-021 / T071: Spec requires "collapsible Left Panel accordion for mobile". The VersionHistory component has no <Accordion> or collapse-on-mobile logic. It renders the same <Card> on all screen sizes.

Impact: On mobile (<768px) the Left Panel is not collapsible — it stacks vertically (technically responsive) but without the accordion UX defined in T071.

Recommendation: Wrap VersionHistory content in a shadcn/ui <Collapsible> or <Accordion> gated by a md:hidden toggle button.

---

GAP-3: Integration Test (T032) Marked describe.skip [LOW]

T032 (Integration test for 3-step sandbox workflow in backend/tests/integration/ai/sandbox-workflow.spec.ts) is implemented but marked describe.skip due to missing e2e infrastructure (UserModule, CacheModule, etc.).

Impact: The 3-step sandbox workflow is not covered by automated tests at integration level. Unit tests for individual steps exist.

Recommendation: Either un-skip with a proper test module setup, or document as a known deferred test requiring e2e infrastructure setup. Update tasks.md T032 status to reflect this.

---

Uncovered Requirements

Requirement	Status	Notes
FR-023	⚠️ Partial	Backend checks {{ocr_text}} only; spec also defines {{master_data_context}} as optional (frontend wrongly requires it)
FR-024	⚠️ Mismatch	Spec: {{user_query}}, {{retrieved_chunks}}; implemented: {{query}}, {{context}}
FR-025	⚠️ Mismatch	Spec: {{document_text}}; implemented: {{text}}
FR-026	⚠️ Partial	Spec: {{document_metadata}} required; implemented: checks {{document_text}} (wrong placeholder)
FR-021 (mobile accordion)	⚠️ Partial	Responsive breakpoints exist but Left Panel is not collapsible accordion
T032 integration test	⚠️ Skipped	Valid test structure but describe.skip — no CI coverage

---

ADR Compliance Check

ADR	Check	Status
ADR-019	No parseInt on UUID; publicId only	✅ Pass — controller uses ParseIntPipe on versionNumber (INT), not UUID
ADR-009	No TypeORM migrations; SQL deltas used	✅ Pass — 3 SQL deltas created
ADR-016	CASL guards on all mutations	✅ Pass — @RequirePermission('system.manage_all') on every mutation
ADR-016	Idempotency-Key on POST/PUT	✅ Pass — POST :type, POST activate, PUT context-config all require it
ADR-007	Layered error handling	✅ Pass — BusinessException/ValidationException + Toast/Inline in frontend
ADR-008	Sandbox jobs via BullMQ (no inline AI)	✅ Pass — all sandbox steps enqueue via aiQueueService.enqueueSandboxJob()
ADR-023/A	AI boundary — no direct Ollama access	✅ Pass — BullMQ queues used for all AI calls
ADR-029	Redis cache TTL 60s for active prompts	✅ Pass — setex(cacheKey, 60, ...)
ADR-037	Single page layout; 3-step sandbox	✅ Pass
TypeScript	Zero any, zero console.log	✅ Pass — reviewed ai-prompts.service.ts, controller, page.tsx
i18n	No hardcoded Thai/English strings	⚠️ Partial — SandboxTabs contains several hardcoded Thai strings (e.g., "กรุณาเลือกไฟล์ PDF", "ทำ OCR สำเร็จแล้ว")

---

Recommendations (Priority Order)

1. [HIGH — FR-023–FR-026] Align placeholder names between spec and code. Recommended approach: update spec to use implemented names ({{query}}, {{context}}, {{text}}). Fix PLACEHOLDER_REQUIREMENTS to mark {{master_data_context}} as optional (not blocking save).
2. [MEDIUM — i18n] Extract hardcoded Thai strings in SandboxTabs.tsx to i18n keys (pre-existing ai.json or common.json).
3. [LOW — T071] Add collapsible accordion to VersionHistory for mobile screens.
4. [LOW — T032] Un-skip integration test or create a tracking issue for e2e infrastructure setup.

---

Sign-off Readiness

Area	Ready?
Backend API endpoints	✅ Yes
Frontend page & components	✅ Yes
Database schema / seed data	✅ Yes
RBAC / Security (ADR-016)	✅ Yes
Error handling (ADR-007)	✅ Yes
Redis cache (ADR-029)	✅ Yes
AI boundary (ADR-023/A)	✅ Yes
Placeholder validation accuracy	❌ No (GAP-1)
Mobile UX (collapsible panel)	⚠️ Partial (GAP-2)
Test coverage (T032 skipped)	⚠️ Partial (GAP-3)
i18n completeness	⚠️ Partial

Conclusion: Core architecture and business logic are correctly implemented. The feature is functionally complete but requires a placeholder naming decision (GAP-1) before production sign-off.