Skip to main content

ValkyrAI Workflow Engine — N8N KILLER Implementation Plan 🔥

Mission Critical: Build the n8n killer or we're dead CMO/CTO Level: Silicon Valley quality standards Date: 2025-10-23 Status: 🚨 EXECUTION PHASE 🚨

🎯 Mission Objectives​

Transform ValkyrAI Workflow Engine into the dominant workflow orchestration platform that makes n8n obsolete through:

  1. Deterministic execution with exactly-once semantics
  2. Typed payloads with schema validation
  3. True RBAC/ACL with field-level encryption
  4. Horizontal scaling to 10k concurrent workflows, 250k active tasks
  5. < 75ms P95 dispatch latency with linear scaling
  6. Delightful UX with React Flow Studio, time-travel debugging, hot-swap

📊 Current State Analysis​

✅ Strong Foundation (Keep & Enhance)​

  • ThorAPI-native: All models generated from OpenAPI
  • 20+ ExecModules: Email, REST, Stripe, Files, Social, AI, etc.
  • Quartz scheduling: CRON + event triggers working
  • Security: Spring Security RBAC + ACL on all operations
  • Async execution: CompletableFuture with proper auth propagation
  • WebSocket monitoring: Real-time execution updates
  • Transaction safety: REQUIRES_NEW for state persistence

🔧 Critical Gaps (Must Fix)​

  1. No WorkflowExecution separation — executions mixed with definitions
  2. No Run/attempt tracking — can't replay or track retries properly
  3. No idempotency enforcement — risk of duplicate effects
  4. No lease mechanism — can't recover from crashes reliably
  5. No DLQ + replay tooling — failed tasks go to black hole
  6. In-process execution only — no sandboxing, resource limits, or hot-swap
  7. Basic Studio UI — needs React Flow upgrade with mapper
  8. No distributed observability — needs OpenTelemetry + flame charts

🏗️ Architecture Enhancements​

Data Model Additions​

Current:                        Enhanced v2.0:
─────────                       ─────────────────

Workflow                        Workflow
  └─ Task                         └─ Task
       └─ ExecModule                   └─ ExecModule (enhanced with ioSchema)

WorkflowState                   WorkflowExecution
                                  ├─ state tracking
                                  ├─ trigger info
                                  ├─ metrics snapshot
                                  └─ Run (per task attempt)
                                       ├─ idempotency key
                                       ├─ lease tracking
                                       ├─ heartbeat
                                       ├─ circuit breaker
                                       └─ DLQ on failure

                                DeadLetterQueue
                                  ├─ failed Run reference
                                  ├─ replay capability
                                  └─ resolution tracking

                                CircuitBreakerState
                                  ├─ module failure tracking
                                  ├─ exponential backoff
                                  └─ auto-recovery

                                Budget & Quota
                                  ├─ per-principal limits
                                  ├─ cost tracking
                                  └─ kill switches

Execution Flow Enhancement​

Current Flow:                   Enhanced v2.0 Flow:
────────────                    ──────────────────

User → executeWorkflow()        User → /Workflow/{id}/execute
  ↓                               ↓
ValkyrWorkflowService           WorkflowExecutionService
  ↓                               ↓
For each task:                  Create WorkflowExecution
  - Get module                    ↓
  - Execute sync                For each task:
  - Update state                  ↓
  - Save workflow               Create Run with idempotency key
                                  ↓
                                Runner acquires lease
                                  ↓
                                Execute with heartbeat
                                  ├─ Resource limits
                                  ├─ Circuit breaker check
                                  ├─ Timeout enforcement
                                  └─ Idempotency guard
                                  ↓
                                On success: update Run → SUCCESS
                                  ↓
                                On failure:
                                  ├─ Transient → retry with backoff
                                  ├─ Circuit open → backoff
                                  └─ Permanent → DLQ
                                  ↓
                                Update WorkflowExecution state
                                  ↓
                                Emit OpenTelemetry spans

🚀 Implementation Roadmap​

Phase 1: Foundation (Week 1) — DATA MODEL​

Priority: CRITICAL ⚡

1.1 Enhanced OpenAPI Schemas​

File: valkyrai/src/main/resources/openapi/api.hbs.yaml

Add/enhance schemas:

  • WorkflowExecution (exists, needs enhancement)
  • Run (exists, needs enhancement)
  • DeadLetterQueue (exists, needs enhancement)
  • CircuitBreakerState (exists)
  • Budget (new)
  • Quota (new)
  • ExecutionMetrics (new)
  • RetryPolicy (new)
  • CompensationTask (new)

Enhancements needed:

  • Add compensationTaskId to Task
  • Add ioSchema (JSONSchema) to ExecModule
  • Add retryPolicy JSON to Task
  • Add resourceLimits to Task (CPU/mem caps)
  • Add budgetId FK to Workflow
  • Add cost tracking fields to Run

1.2 Regenerate ThorAPI Code​

cd /Users/johnmcmahon/workspace/2025/valkyr/ValkyrAI
mvn -pl thorapi clean install -DskipTests

Verify generated:

  • valkyrai/generated/spring/src/main/java/com/valkyrlabs/model/WorkflowExecution.java
  • valkyrai/generated/spring/src/main/java/com/valkyrlabs/model/Run.java
  • valkyrai/generated/spring/src/main/java/com/valkyrlabs/model/DeadLetterQueue.java
  • valkyrai/generated/spring/src/main/java/com/valkyrlabs/model/Budget.java
  • valkyrai/generated/spring/src/main/java/com/valkyrlabs/model/Quota.java

1.3 Database Migrations​

File: valkyrai/src/main/resources/db/migration/V2.1__n8n_killer_foundation.sql

-- WorkflowExecution table
CREATE TABLE workflow_execution (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    workflow_id UUID NOT NULL REFERENCES workflow(id),
    version INTEGER NOT NULL,
    state VARCHAR(20) NOT NULL,
    started_at TIMESTAMP WITH TIME ZONE NOT NULL,
    finished_at TIMESTAMP WITH TIME ZONE,
    parent_execution_id UUID REFERENCES workflow_execution(id),
    trigger_type VARCHAR(20) NOT NULL,
    trigger_payload TEXT,
    context_data TEXT,
    error_message TEXT,
    error_stack_trace TEXT,
    metrics_snapshot TEXT,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

CREATE INDEX idx_workflow_execution_workflow_id ON workflow_execution(workflow_id);
CREATE INDEX idx_workflow_execution_state ON workflow_execution(state);
CREATE INDEX idx_workflow_execution_started_at ON workflow_execution(started_at DESC);

-- Run table (enhanced)
CREATE TABLE run (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    execution_id UUID NOT NULL REFERENCES workflow_execution(id),
    task_id UUID NOT NULL REFERENCES task(id),
    exec_module_id UUID REFERENCES exec_module(id),
    attempt INTEGER NOT NULL DEFAULT 1,
    state VARCHAR(20) NOT NULL,
    lease_until TIMESTAMP WITH TIME ZONE,
    leased_by VARCHAR(255),
    runner_id VARCHAR(255),
    idempotency_key VARCHAR(64) NOT NULL,
    inputs_hash VARCHAR(64),
    config_hash VARCHAR(64),
    started_at TIMESTAMP WITH TIME ZONE,
    finished_at TIMESTAMP WITH TIME ZONE,
    heartbeat_at TIMESTAMP WITH TIME ZONE,
    outputs TEXT,
    error TEXT,
    error_type VARCHAR(20),
    retry_after TIMESTAMP WITH TIME ZONE,
    duration_ms BIGINT,
    cost_tokens NUMERIC(12, 4),
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

CREATE UNIQUE INDEX idx_run_idempotency ON run(execution_id, task_id, idempotency_key);
CREATE INDEX idx_run_execution_id ON run(execution_id);
CREATE INDEX idx_run_task_id ON run(task_id);
CREATE INDEX idx_run_state ON run(state);
CREATE INDEX idx_run_lease_until ON run(lease_until) WHERE state = 'PENDING' OR state = 'LEASED';
CREATE INDEX idx_run_heartbeat ON run(heartbeat_at) WHERE state = 'RUNNING';

-- Dead Letter Queue
CREATE TABLE dead_letter_queue (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    run_id UUID NOT NULL REFERENCES run(id),
    execution_id UUID NOT NULL REFERENCES workflow_execution(id),
    task_id UUID NOT NULL REFERENCES task(id),
    failure_reason TEXT NOT NULL,
    failure_type VARCHAR(30) NOT NULL,
    original_inputs TEXT,
    original_config TEXT,
    quarantined_at TIMESTAMP WITH TIME ZONE NOT NULL,
    reviewed_by VARCHAR(255),
    reviewed_at TIMESTAMP WITH TIME ZONE,
    resolution VARCHAR(20),
    resolution_notes TEXT,
    requeued_as_run_id UUID REFERENCES run(id),
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

CREATE INDEX idx_dlq_resolution ON dead_letter_queue(resolution);
CREATE INDEX idx_dlq_quarantined_at ON dead_letter_queue(quarantined_at DESC);

-- Circuit Breaker State
CREATE TABLE circuit_breaker_state (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    module_name VARCHAR(255) NOT NULL,
    target_endpoint VARCHAR(500),
    state VARCHAR(20) NOT NULL,
    failure_count INTEGER NOT NULL DEFAULT 0,
    last_failure_at TIMESTAMP WITH TIME ZONE,
    opened_at TIMESTAMP WITH TIME ZONE,
    half_open_at TIMESTAMP WITH TIME ZONE,
    success_count_since_half_open INTEGER DEFAULT 0,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    UNIQUE(module_name, target_endpoint)
);

CREATE INDEX idx_circuit_breaker_state ON circuit_breaker_state(state);

-- Budget & Quota
CREATE TABLE budget (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    principal_id UUID NOT NULL REFERENCES principal(id),
    period_start TIMESTAMP WITH TIME ZONE NOT NULL,
    period_end TIMESTAMP WITH TIME ZONE NOT NULL,
    max_cost_tokens NUMERIC(12, 4),
    max_executions INTEGER,
    current_cost_tokens NUMERIC(12, 4) DEFAULT 0,
    current_executions INTEGER DEFAULT 0,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

CREATE INDEX idx_budget_principal ON budget(principal_id);
CREATE INDEX idx_budget_period ON budget(period_start, period_end);

CREATE TABLE quota (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    principal_id UUID NOT NULL REFERENCES principal(id),
    resource_type VARCHAR(50) NOT NULL,
    max_concurrent INTEGER,
    max_per_hour INTEGER,
    current_concurrent INTEGER DEFAULT 0,
    current_hour_count INTEGER DEFAULT 0,
    hour_reset_at TIMESTAMP WITH TIME ZONE,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    UNIQUE(principal_id, resource_type)
);

CREATE INDEX idx_quota_principal ON quota(principal_id);

-- Add new columns to existing tables
ALTER TABLE task ADD COLUMN compensation_task_id UUID REFERENCES task(id);
ALTER TABLE task ADD COLUMN retry_policy TEXT;
ALTER TABLE task ADD COLUMN resource_limits TEXT;

ALTER TABLE exec_module ADD COLUMN io_schema TEXT;
ALTER TABLE exec_module ADD COLUMN rate_limit_config TEXT;

ALTER TABLE workflow ADD COLUMN budget_id UUID REFERENCES budget(id);

Phase 2: Core Services (Week 1-2) — BACKEND​

Priority: CRITICAL ⚡

2.1 RunService​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/service/RunService.java

Responsibilities:

  • Generate idempotency keys (SHA-256 of inputs + config)
  • Lease acquisition/release with DB row locking
  • Heartbeat tracking (extend lease on heartbeat)
  • Zombie reaper (reclaim expired leases)
  • DLQ promotion on permanent failure

Key methods:

public Run createRun(WorkflowExecution execution, Task task, ExecModule module, Map<String, Object> inputs);
public Optional<Run> acquireLease(UUID runId, String runnerId, Duration leaseDuration);
public void updateHeartbeat(UUID runId, String runnerId);
public void completeRun(UUID runId, Map<String, Object> outputs, long durationMs, double costTokens);
public void failRun(UUID runId, String error, ErrorType errorType);
public List<Run> findExpiredLeases();
public void promoteToDeadLetterQueue(UUID runId, String failureReason, FailureType failureType);

2.2 WorkflowExecutionService​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/service/WorkflowExecutionService.java

Responsibilities:

  • Create WorkflowExecution from Workflow
  • Orchestrate Run creation for each Task
  • Track execution state transitions
  • Handle pause/resume/cancel operations
  • Coordinate with existing ValkyrWorkflowService

Key methods:

public WorkflowExecution createExecution(Workflow workflow, TriggerType triggerType, String triggerPayload);
public CompletableFuture<WorkflowExecution> execute(UUID executionId);
public void pauseExecution(UUID executionId);
public void resumeExecution(UUID executionId);
public void cancelExecution(UUID executionId);
public WorkflowExecution getExecutionStatus(UUID executionId);

2.3 DLQService​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/service/DLQService.java

Responsibilities:

  • Query DLQ entries with filters
  • Requeue with input overrides
  • Discard with notes
  • Track resolution workflow

Key methods:

public DeadLetterQueue quarantine(Run run, String failureReason, FailureType failureType);
public Run requeue(UUID dlqId, Map<String, Object> inputOverrides, String notes);
public DeadLetterQueue discard(UUID dlqId, String notes);
public List<DeadLetterQueue> findUnresolved();

2.4 RunnerService​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/runner/RunnerService.java

Responsibilities:

  • Poll for pending Runs
  • Acquire lease before execution
  • Execute with heartbeat keepalive thread
  • Enforce resource limits (thread pool, timeout)
  • Check circuit breaker before execution
  • Update Run state on completion/failure

Key methods:

@Scheduled(fixedDelay = 100) // Poll every 100ms
public void pollAndExecute();

public void executeRun(Run run);
private void executeWithHeartbeat(Run run, VModule module, Map<String, Object> inputs);

2.5 CircuitBreakerService​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/service/CircuitBreakerService.java

Responsibilities:

  • Track module failure rates
  • Open circuit on threshold breach
  • Half-open after cooldown period
  • Close on success count threshold

Key methods:

public boolean isCircuitOpen(String moduleName, String targetEndpoint);
public void recordSuccess(String moduleName, String targetEndpoint);
public void recordFailure(String moduleName, String targetEndpoint);
public void openCircuit(String moduleName, String targetEndpoint);
public void closeCircuit(String moduleName, String targetEndpoint);

2.6 BudgetService & QuotaService​

Files:

  • valkyrai/src/main/java/com/valkyrlabs/workflow/service/BudgetService.java
  • valkyrai/src/main/java/com/valkyrlabs/workflow/service/QuotaService.java

Responsibilities:

  • Track per-principal budgets (cost tokens, execution counts)
  • Enforce quotas (concurrent, per-hour limits)
  • Kill switch on budget exceeded
  • Reset quota counters on period boundaries

Phase 3: Enhanced Controllers (Week 2) — REST API​

Priority: HIGH 🔥

3.1 WorkflowExecutionController​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/controller/WorkflowExecutionController.java

Implement custom operations (CRUD auto-generated):

  • POST /WorkflowExecution/{id}/cancel
  • POST /WorkflowExecution/{id}/pause
  • POST /WorkflowExecution/{id}/resume

3.2 RunController​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/controller/RunController.java

Implement custom operations:

  • POST /Run/{id}/heartbeat

3.3 DLQController​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/controller/DLQController.java

Implement custom operations:

  • POST /DeadLetterQueue/{id}/requeue
  • POST /DeadLetterQueue/{id}/discard

Phase 4: ExecModule Enhancements (Week 2-3) — MODULE SYSTEM​

Priority: HIGH 🔥

4.1 Add I/O Schema Validation​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/modules/VModule.java

Add methods:

public abstract String getInputSchema(); // JSONSchema
public abstract String getOutputSchema(); // JSONSchema
public void validateInputs(Map<String, Object> inputs) throws ValidationException;

4.2 Rate Limiting Annotations​

Add @RateLimit annotation:

@RateLimit(maxPerSecond = 10, maxPerMinute = 100)
public class StripeCheckoutModule extends VModule { ... }

4.3 Hot-Swap Support​

Track module versions in ExecModule:

  • Add version field
  • Add implementationClass field
  • Load modules dynamically via class name + version
  • Support traffic shifting (canary %, blue/green)

Phase 5: Observability (Week 3) — TELEMETRY​

Priority: HIGH 🔥

5.1 OpenTelemetry Integration​

Dependencies: Add to pom.xml

<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-api</artifactId>
</dependency>
<dependency>
    <groupId>io.opentelemetry.instrumentation</groupId>
    <artifactId>opentelemetry-spring-boot-starter</artifactId>
</dependency>

Instrumentation points:

  • Workflow execution start/end
  • Each Run start/end
  • Module execution with input/output size
  • Database queries
  • External API calls

Span attributes:

  • workflow.id
  • workflow.name
  • execution.id
  • run.id
  • task.id
  • module.name
  • run.attempt
  • run.duration_ms
  • run.cost_tokens

5.2 Metrics Collection​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/metrics/ExecutionMetricsCollector.java

Collect per-execution:

  • Total duration
  • Step durations (flame chart data)
  • Cost breakdown by module
  • Retry counts
  • Success/failure rates

Expose via:

  • Micrometer/Prometheus metrics
  • Custom /metrics endpoint with filters

Phase 6: Frontend Studio (Week 3-4) — UX​

Priority: HIGH 🔥

6.1 React Flow Canvas​

File: web/src/components/WorkflowStudio/FlowCanvas.tsx

Features:

  • Drag-drop from module palette
  • Snap-to-grid
  • Auto-layout (dagre)
  • Port compatibility highlighting (schema-based)
  • Mini-map
  • Searchable palette with fuzzy search

6.2 Data Mapper​

File: web/src/components/WorkflowStudio/DataMapper.tsx

Features:

  • Visual JSONPath selector
  • Formula editor (powered by GridHeim)
  • Type validation (red wiring on mismatch)
  • Live preview with sample data

6.3 Simulate Mode​

File: web/src/components/WorkflowStudio/Simulator.tsx

Features:

  • Input sample data
  • Dry-run execution (no side effects)
  • Step-by-step debugger
  • State inspector at each step

6.4 Execution Monitor​

File: web/src/components/ExecutionMonitor/ExecutionMonitor.tsx

Features:

  • Real-time execution view (WebSocket)
  • Flame chart for step durations
  • Retry timeline
  • Live logs with filtering
  • Time-travel: replay with input overrides

6.5 DLQ Browser​

File: web/src/components/DLQBrowser/DLQBrowser.tsx

Features:

  • Table view of DLQ entries
  • Filters: failure type, date range, workflow
  • Requeue modal with input override editor
  • Discard modal with notes
  • Resolution history timeline

Phase 7: Advanced Features (Week 4-5) — KILLER FEATURES​

Priority: MEDIUM 📊

7.1 Saga/Compensation​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/service/CompensationService.java

Features:

  • Track compensation tasks per Task
  • Execute undo workflow on failure
  • Support nested compensation chains
  • Store compensation execution state

7.2 Webhook System​

File: valkyrai/src/main/java/com/valkyrlabs/workflow/controller/WebhookController.java

Features:

  • POST /webhook/{workflowId}/{secretToken} endpoint
  • Signature verification (HMAC-SHA256)
  • Schema validation against registered event types
  • Rate limiting per webhook URL
  • DLQ for malformed webhooks

7.3 Queue Adapters​

Files:

  • valkyrai/src/main/java/com/valkyrlabs/workflow/adapters/NatsAdapter.java
  • valkyrai/src/main/java/com/valkyrlabs/workflow/adapters/KafkaAdapter.java

Features:

  • Subscribe to NATS/Kafka topics
  • Trigger workflows on message arrival
  • Backpressure: pause subscription when queue depth > threshold
  • At-least-once delivery with idempotency

7.4 Multi-Region Active/Active​

Configuration: application.yml

valkyr:
  workflow:
    region: us-east-1
    scheduler:
      leaderElection: true
      lockTable: scheduler_lock
    runner:
      shardId: ${RUNNER_SHARD_ID:auto}

Features:

  • Database-backed scheduler lock
  • Runner sharding by consistent hash
  • Cross-region execution visibility
  • Conflict-free state merging (CRDTs)

📈 Performance Targets​

Latency (P95)​

  • Dispatch: ≤ 75ms
  • Cold start: ≤ 400ms
  • Resume from snapshot: ≤ 120ms
  • Webhook → first step: ≤ 150ms

Throughput​

  • 10k concurrent executions
  • 250k active tasks
  • Runner CPU ≤ 70% under load

Durability​

  • Exactly-once step execution (idempotency)
  • Zero data loss on 1/3 pod crash
  • Auto-recovery ≤ 5s

🧪 Testing Strategy​

Unit Tests​

  • All services with MockMvc/MockBean
  • Idempotency key generation
  • Lease acquisition logic
  • Circuit breaker state machine
  • DLQ quarantine/requeue

Integration Tests​

  • End-to-end workflow execution
  • Crash recovery (kill runner mid-step)
  • DLQ replay with overrides
  • Budget enforcement (kill switch)
  • Webhook signature validation

Chaos Tests​

  • Kill 1/3 runners → verify recovery
  • Kill scheduler leader → verify failover
  • Network partition → verify state convergence
  • Database connection loss → verify retry

Load Tests​

  • Ramp to 10k concurrent workflows
  • Verify P95 latency targets
  • Monitor runner CPU/memory
  • Verify linear scaling

📋 Acceptance Criteria​

Must Have (GA Blocker)​

  • WorkflowExecution model with state tracking
  • Run model with idempotency + leasing
  • RunService with lease acquisition + heartbeat
  • WorkflowExecutionService with pause/resume/cancel
  • DLQService with requeue/discard
  • RunnerService with polling + heartbeat keepalive
  • CircuitBreakerService with auto-recovery
  • All custom controllers implemented
  • Database migrations applied
  • OpenTelemetry spans on all operations
  • React Flow Studio with mapper
  • Execution Monitor with flame charts
  • DLQ Browser with replay UI
  • Crash recovery tests pass
  • Load tests pass (10k workflows, P95 < 75ms)

Nice to Have (Post-GA)​

  • Saga/compensation support
  • NATS/Kafka adapters
  • Multi-region active/active
  • Canary deploys for modules
  • Marketplace packaging

🚨 Risk Mitigation​

Risk: Thundering herd on requeue​

Mitigation: Exponential backoff + shard locks + priority queue

Risk: User modules misbehave​

Mitigation: Sandbox with CPU/mem limits + network policies + timeout enforcement

Risk: Webhook storms​

Mitigation: Global rate limiters + token buckets per principal + circuit breakers

Risk: Database connection pool exhaustion​

Mitigation: Connection pooling tuning + read replicas for queries + write batching

Risk: Memory leaks in long-running runners​

Mitigation: Pod rotation every N executions + memory profiling + heap dump on OOM

📚 Documentation Deliverables​

  • API documentation (Swagger UI)
  • Architecture diagrams (Mermaid)
  • Runbooks (crash recovery, scaling, DLQ management)
  • Developer guide (writing ExecModules)
  • Operations guide (monitoring, alerts, troubleshooting)
  • Migration guide (v1.x → v2.0)

🎖️ Definition of Done​

Code Complete​

  • All services implemented with unit tests
  • All controllers with integration tests
  • All UI components with Storybook stories
  • Code coverage ≥ 80%

Quality Gate​

  • Zero critical security vulnerabilities
  • Zero P0/P1 bugs
  • Load tests pass at 10k workflows
  • Chaos tests pass (1/3 pod crash)

Documentation​

  • All runbooks complete
  • API docs published
  • Migration guide reviewed
  • Developer guide reviewed

Deployment​

  • Staging deployment successful
  • Production canary at 10% traffic
  • Monitoring dashboards deployed
  • On-call runbooks in place

🎯 Success Metrics (30 days post-GA)​

  • Adoption: ≥ 1000 workflows created
  • Reliability: ≥ 99.9% execution success rate
  • Performance: P95 dispatch < 75ms sustained
  • NPS: ≥ 70 for Studio UX
  • Conversion: ≥ 4.5% publish-to-run for new flows

🔥 CALL TO ACTION​

ValorIDE Agent: Execute this plan NOW. We are building the n8n killer or we're dead. Quality is non-negotiable. Velocity is critical. Ship Phase 1 (data model) TODAY. Ship Phase 2 (core services) by end of Week 1. Ship Phase 3-6 by end of Week 4. This is MISSION CRITICAL.

LET'S BUILD. 🚀