Skip to main content

Technical Context — ValkyrAI Workflow Engine Architecture

Current Architecture Analysis​

Backend Stack​

  • Framework: Spring Boot with Quartz Scheduler
  • Security: Spring Security ACL with RBAC
  • Database: JPA/Hibernate with generated ThorAPI models
  • API: REST endpoints under /v1/vaiworkflow/
  • Real-time: WebSocket integration for execution monitoring
  • Module System: 20+ ExecModules with pluggable architecture

Core Backend Components​

Workflow Engine (ValkyrWorkflowService)​

- executeWorkflow() - Async/sync workflow execution
- loadWorkflowSchedules() - Quartz integration
- registerEventTrigger() - Event-driven execution
- Task chaining with branching logic
- Security context management

ExecModules (Production Ready)​

- EmailModule / MailtrapSendModule - Email automation
- RestApiModule - HTTP API calls
- FileSystemModule - File operations
- StripeCheckoutModule - Payment processing
- SocialMediaExecModule - Social posting
- AiChatModule - LLM integration
- BranchingModule - Conditional logic
- And 15+ more specialized modules

Data Models (ThorAPI Generated)​

- Workflow - Main workflow definition
- Task - Individual workflow steps
- ExecModule - Pluggable execution units
- WorkflowState - Runtime state management

Frontend Stack​

  • Framework: React 18 + TypeScript
  • Visual Designer: React Flow for drag-drop workflow design
  • State Management: Redux Toolkit Query (RTK Query)
  • API Integration: ThorAPI generated services
  • UI Components: Bootstrap + custom components
  • Real-time: WebSocket integration

Current Frontend Components​

WorkflowStudio (/components/WorkflowStudio/)​

- Visual drag-drop workflow designer
- Node palette with ExecModules
- Real-time execution monitoring
- Graph validation and persistence
- Authentication integration

Supporting Components​

- WorkflowCanvas - React Flow integration
- NodePalette - Draggable module palette
- InspectorPanel - Property editor
- ExecModuleAddModal - Module configuration
- WorkflowPickerModal - Workflow selection

Enhancement Requirements​

1. Production-Level UX​

  • Error Handling: Comprehensive error boundaries and user feedback
  • Performance: Lazy loading, virtualization, caching
  • Accessibility: ARIA labels, keyboard navigation
  • Responsiveness: Mobile-friendly design

2. ExecModule Parameter Mapping​

  • Visual Data Flow: Show data connections between modules
  • Parameter Editor: Rich form for module configuration
  • Variable Mapping: Map WorkflowState to module inputs
  • Type Safety: Validate data types and constraints

3. Built-in QBE ExecModule​

  • API Integration: Query any REST endpoint
  • Filter UI: Visual query builder
  • Data Preview: Show query results
  • Variable Injection: Use workflow inputs in queries

4. Async Branching System​

  • Parallel Execution: Multiple concurrent paths
  • Join Points: Synchronization mechanisms
  • Conditional Logic: Complex branching rules
  • Error Handling: Graceful failure recovery

5. Monitoring & Operations​

  • Real-time Status: Live execution tracking
  • Performance Metrics: Execution time, success rates
  • Error Logging: Detailed failure analysis
  • Resource Usage: Memory, CPU monitoring

Technical Debt & Improvements​

Backend Enhancements Needed​

  1. Better Error Handling: Structured error responses
  2. Async Improvements: Better CompletableFuture usage
  3. Caching Strategy: Redis integration for state
  4. Metrics Collection: Micrometer integration
  5. Testing Coverage: Comprehensive integration tests

Frontend Enhancements Needed​

  1. Component Architecture: Better separation of concerns
  2. State Management: Optimized Redux patterns
  3. Type Safety: Strict TypeScript configuration
  4. Performance: Bundle optimization, code splitting
  5. Testing: Jest + React Testing Library coverage

Security Considerations​

  • RBAC Integration: Workflow-level permissions
  • Secure Execution: Sandboxed module execution
  • Audit Logging: Complete execution trails
  • Secret Management: Encrypted configuration storage

Deployment Architecture​

  • Container Ready: Docker support
  • Scalability: Horizontal scaling capabilities
  • Monitoring: Health checks and metrics
  • CI/CD: Automated testing and deployment

Integration Points​

  • ThorAPI: Model generation and CRUD operations
  • Spring Security: Authentication and authorization
  • Quartz: Scheduled execution
  • WebSocket: Real-time communication
  • Database: Persistent state management

Performance Targets​

  • Workflow Load Time: < 2 seconds
  • Execution Start: < 1 second
  • Large Workflows: 100+ nodes supported
  • Concurrent Users: 50+ simultaneous editors
  • Memory Usage: < 512MB per workflow session