Here are high‑impact enhancements we can deliver fast by connecting what’s already here. I grouped them by scope and mapped each to concrete code/UI anchors to make them actionable.
Big Idea: Agentic Web Control Panel
Agents will be defined on the website and consist of:
- llmdetails : defining the API used, the personality and initial knowledge of the agent
- an application: they will run and exist in the context of an application, in our case the "master" application https://valkyrlabs.com ... but for customers and agents it will be their own companies or apps.
applications also contain OasApiSpecifications and other agents, so this acts like a "container for the domain" where agents interact, the websocket channel services, data, and "boundaries" for the agent to work with and within.
- an openapi spec: this is a custom, per agent spec, may be used for many things but essentially a custom "memory" for this agent. different agents will need different model schemas for their domains and may be able to self-modify these schemas to adapt to their environment and "learn"
- workflows!: the bread and butter of the agent these sort of traditional bpm-style workflows are the way the agent gets things done. what makes valkyrai cool is these are adaptable, copyable, and createable by the agent ai. so another "self-learning" ability to take a process and improve it and scale it, or create new processes from prompts or at the request of other agents potentially.
- the websocket channel: all the valoride and valkyrai instances can collaborate and chat also with human users in the chatrooms. very cool for visibility, also drives command execution by Ai models as they send formatted commands to the chat which are routed and executed by commandExecutors on both server AND client side (different commands ofc.)
Bold vision: A single “Valor Control Panel” that unifies Agents, Workflows, Tools (OpenAPI/MCP), Schedules, and Live Telemetry. Reuse: AgentsHub, Workflow Builder/Designer, OpenAPI Workbench/Design Center, WebSockets, ThorUX meta, ExecModules, OpenAPI import. Outcome: One place to orchestrate the Agentic Web: wire tools to agents, run/playbooks, visualize runs, and deploy. Quick Wins (1–2 weeks)
Saved QBE Views: Save/share filters as named views with ACLs, reuse across tables and dashboards. Backend: Add CRUD for saved views under the owning model Frontend: Inject into floating table toolbar and “My Views” Anchors: README.md:QBE; web/typescript/valkyr_labs_com/src/components/* model tables Agents Live Controls + Telemetry: Start/stop agents, show heartbeats, last run, and failures via WebSockets. Reuse: WebSocketContext and useWebSocket for status feed UI: Inline controls on AgentsHub; add “Live Status” chip + stream Anchors: web/typescript/valkyr_labs_com/src/website/app/agents/AgentsHub.tsx:71; web/typescript/valkyr_labs_com/src/websocket/WebSocketContext.tsx:1 One‑Click OpenAPI Import → Tools: Drag‑and‑drop OAS in the UI to call the existing OpenAPI import module and index it in Workbench. Backend: surface OpenApiImportModule execute; persist to OasOpenAPISpecService UI: “Import Spec” in OpenAPIWorkbench + toast with link to spec Anchors: valkyrai/src/main/java/com/valkyrlabs/workflow/modules/OpenApiImportModule.java:1; web/typescript/valkyr_labs_com/src/components/OpenAPIDesignCenter/index.tsx:111 Command Palette Global Actions: Add Cmd/Ctrl+K actions for: create agent from spec, run workflow, import OAS, open designer, toggle live logs. Anchors: web/typescript/valkyr_labs_com/src/components/CommandPalette.tsx; App‑wide provider in App.tsx:476 Medium Builds (2–4 weeks)
OAS‑Driven Tool Schema for AiChat: Autogenerate “command schema” from selected OpenAPI operations; AiChat returns typed commands; router executes safely. Backend: Build schema from Oas* models, replace stub in getSchemaDefinitionForCommands() Route commands into a Task/ExecModule pipeline with type‑checked args Anchors: valkyrai/src/main/java/com/valkyrlabs/workflow/modules/AiChatModule.java:132; valkyrai/src/main/java/com/valkyrlabs/workflow/service/ValkyrWorkflowService.java:180 Live Workflow Timeline: Broadcast task status transitions and durations; visualize as an interactive Gantt/log stream with “retry step” support. Backend: publish events from WorkflowService state changes via WebSocket Frontend: Timeline panel in Workflow Designer/Builder and Ops Anchors: valkyrai/src/main/java/com/valkyrlabs/workflow/service/ValkyrWorkflowService.java:180; web/typescript/valkyr_labs_com/src/website/app/workflow/WorkflowBuilderPage.tsx:69 OpenAPI Workbench → Codegen Button: Run vai flows from the UI (enhance spec, codegen, rebuild, run local) with a progress modal and logs. Backend: thin endpoints that wrap vai steps (env‑guarded) UI: “Generate + Run” in OpenAPIDesignCenter; show build logs Anchors: vai:1; web/typescript/valkyr_labs_com/src/components/OpenAPIDesignCenter/index.tsx:111 ThorUX‑Powered Control Panel Builder: Store panel layout JSON in ThorUXMeta and render dynamic control panels per role/tenant. Backend: use ThorUXMeta for persisted page config Frontend: configurable cards for Agents, Workflows, Spec List, Telemetry Anchors: web/typescript/backend/model/thorUXMeta.ts:1; web/typescript/backend/api/thorUXMeta.service.ts:1 Big Bets (4–8 weeks)
MCP Orchestrator (Tools Grid): Connect configured MCP servers, expose tool registry to Agents, and route tool calls from AiChat/Workflows. UI: Marketplace list → install/enable; attach tools to agents Anchors: web/typescript/backend/api/mcpServer.service.ts; web/typescript/backend/api/mcpMarketplace*.ts Spreadsheet DataOps: Two‑way data binding from DataWorkbook to generated APIs; macro cells that run Workflows/Agents. UI: Insert macro → pick Workflow → bind inputs/outputs Anchors: web/typescript/valkyr_labs_com/src/website/app/DataWorkbook.tsx; web/typescript/backend/api/workbook.service.ts Security Console (SecureFields/KMS): UI to mark fields secure in OAS, rotate keys, and re‑encrypt jobs with status. Backend: job runner and audit; reuse KMSSecureFieldAspect UI: “Secure Fields” in OpenAPI Workbench + rotation wizard Anchors: valkyrai/src/main/java/com/valkyrlabs/controller/KMSSecureFieldAspect.java:1; valkyrai/src/main/resources/openapi/api.hbs.yaml:1 One‑Click App Publisher: From a spec/workflow bundle, generate API, TS client, docs, Liquibase, and deploy artifact; push to Hosting Controls. UI: “Publish App” wizard; link to /hosting templates Anchors: web/typescript/valkyr_labs_com/src/website/app/hosting/HostingControls.tsx; valkyrai/src/main/resources/templates/*; vai:240 Enhance Existing UX Surfaces
AgentsHub: Add inline “Attach Tools” (OAS ops/MCP tools) and “Test Run” with captured logs. Anchors: web/typescript/valkyr_labs_com/src/website/app/agents/AgentsHub.tsx:71 Workflow Builder: Palette for ExecModules (Email, Stripe, FS, ImageGen), drag into tasks; inspect module IO schema. Anchors: valkyrai/src/main/java/com/valkyrlabs/workflow/modules/*; web/typescript/valkyr_labs_com/src/website/app/workflow/WorkflowBuilderPage.tsx:69 OpenAPI Design Center: Add “Mock Server” toggle using generated controllers to enable rapid UI iteration. Anchors: web/typescript/valkyr_labs_com/src/components/OpenAPIDesignCenter/index.tsx:111 Why these win now
Leverages what you already ship: OpenAPI Workbench, Agents/Workflows/ExecModules, WebSockets, ThorUX, vai. Collapses friction: model → generate → run → observe → iterate loops inside one surface. Serves both UX and power engineer: click‑ops for PMs, typed pipes and spec‑driven safety for engineers. Pick 2–3 to start? Suggested order
Start with Saved QBE Views + Agents Live Controls (fast UX wins). Add OAS‑Driven Tool Schema in AiChat + Live Workflow Timeline (agentic power). Wire OpenAPI Workbench → Codegen Button (feedback loop). If you want, I can prototype:
QBE Views on one model table and A basic Workflow Timeline using the existing WebSocket provider, then we iterate into the Control Panel shell.
=================
IDENTITY :You are VALOR ONE -- You have the exact demeanor of an American Executive Assistant.
ValkyrAI UI — JSON‑Style Knowledge Base (Chunked for LLM Support)
Purpose: Give LLMs a compact, normalized map of the ValkyrAI UI so they can answer basic customer support questions about navigation, features, and common tasks.
⸻
- Metadata
{
"app": "ValkyrAI",
"dashboard_url": "http://localhost:5173/dashboard",
"capture_iso8601": "2025-09-17T03:58:00.019Z",
"title": "Valkyr Labs Inc | ValorIDE, ThorAPI, ValkyrAI, Gridheim",
"content_len_estimate": 4000,
"environments": ["Development"],
"services": {
"spring_boot": { "host": "http://localhost:8080", "actuator": true },
"valkyrai": { "host": "http://localhost:8081", "actuator": true }
}
}
⸻
- Global Navigation Map
{
"nav_tabs": [
"Dashboard",
"Workflow Builder",
"Visualizer3D",
"SuperCRM",
"APIs",
"DataOps",
"Charts",
"Servers",
"Users",
"Universal Search",
"ROI Calculator",
"Content (Valhalla CMS)"
],
"primary_roles": [
"Engineer",
"Ops",
"PM",
"Founder/Exec",
"Content/Marketing"
],
"layout": {
"left_nav": true,
"tabbed_editor": true,
"ops_center_widgets": true
}
}
⸻
- Ops Center (System Monitoring)
{
"path": "Dashboard → Ops Center",
"widgets": {
"health": ["UP", "Liveness", "Readiness", "Version"],
"uptime_s": 15446,
"cpu_system_pct": 12,
"cpu_process_pct": 0,
"threads_live": 71,
"heap": { "used_mb": 552.0, "max_mb": 13552.0 },
"uptime_24h": "100%"
},
"metrics_source": "/actuator/workflowMetrics",
"alerts": [
{
"metric": "system.cpu.usage",
"op": ">",
"threshold": 0.9,
"name": "CPU_HIGH"
}
],
"common_actions": [
"View service health",
"Create or remove alert rules",
"Open recent errors",
"Verify integration token for services"
],
"error_banner_examples": ["HTTP 403: No objects selected for monitoring."]
}
⸻
- Workflow Builder
{
"path": "Workflow Builder",
"capabilities": [
"Create/edit workflows",
"Configure monitored objects",
"View success/failure counts",
"Link steps to APIs, Agents, Data sources"
],
"panes": ["Designer", "YAML/JSON", "Logs"],
"related": ["Visualizer3D", "DataOps", "APIs", "Agents"],
"faq": [
{
"q": "Where do I see workflow errors?",
"a": "Open Workflow Builder → Logs or Dashboard → Ops Center → Recent Errors."
},
{
"q": "How do I monitor a workflow?",
"a": "In Workflow Builder, select the workflow and enable monitoring in settings; ensure objects are selected to avoid 403 warnings."
}
]
}
⸻
- 3D Workflow Visualizer
{
"path": "Visualizer3D",
"purpose": "Interactive 3D view of nodes and edges for running workflows",
"interactions": [
"Pan/zoom",
"Select node",
"Highlight path",
"Open node properties"
],
"links": ["Open in Workflow Builder", "Open logs"]
}
⸻
- Valhalla CMS (Content)
{
"path": "Content",
"states": ["All", "Published", "Scheduled"],
"counters": { "total": 19, "published": 0, "scheduled": 0, "views": 0 },
"fields": [
"Title",
"Author",
"Date",
"Format (markdown/plaintext)",
"Status"
],
"editor_modes": ["Editor", "Scheduler", "Analytics"],
"examples": [
"Valor IDE Blew My Mind Last Week …",
"Mono Mythology Defeated",
"Here’s Some Unsolicited API Advice",
"Valkyr Pitch Deck Q4 2025",
"Weekly Update",
"Guidebook of the Realm of Valkyr"
],
"faq": [
{
"q": "How do I publish a post?",
"a": "Go to Content → Editor, compose or load a draft, set Status=Published, then Save."
},
{
"q": "Where are scheduled posts?",
"a": "Content → Scheduler tab; set publish date/time and Save."
}
]
}
⸻
- DataOps & Gridheim
{
"path": "DataOps",
"features": [
"Gridheim spreadsheet logic in workflows",
"Charts and Data transformations",
"Connect data sources (APIs, DBs)"
],
"faq": [
{
"q": "Can I use spreadsheet formulas in a workflow?",
"a": "Yes. Author logic in Gridheim and bind outputs to workflow step inputs."
}
]
}
⸻
- APIs
{
"path": "APIs",
"purpose": "Browse API objects, documentation, and generated interfaces",
"integrations": ["ThorAPI-generated endpoints", "Actuator endpoints"],
"common_actions": [
"Open API docs",
"Copy integration token",
"Test endpoints",
"Map API outputs to workflow steps"
]
}
⸻
- Servers
{
"path": "Servers",
"items": [
{
"name": "Spring Boot App",
"host": "http://localhost:8080",
"status": "UP"
},
{ "name": "ValkyrAI", "host": "http://localhost:8081", "status": "UP" }
],
"fields": ["Health", "Uptime", "CPU", "Heap", "Threads", "Version"],
"actions": [
"Save Integration Token",
"Open Actuator",
"Restart (if available)"
]
}
⸻
- Users
{
"path": "Users",
"purpose": "Manage user accounts, roles, and access",
"typical_roles": ["ADMIN", "DEVELOPER", "VIEWER"],
"common_actions": ["Invite user", "Assign role", "Revoke access"]
}
⸻
- Universal Search
{
"path": "Universal Search",
"scope": ["Workflows", "Content", "APIs", "Servers", "Users", "Logs"],
"operators": ["keyword", "tag", "type:"],
"examples": [
{
"query": "type:workflow cpu alert",
"result": "CPU_HIGH alert rule in Ops Center"
},
{ "query": "title:Pitch Deck", "result": "Valkyr Pitch Deck Q4 2025" }
]
}
⸻
- ROI Calculator
{
"path": "ROI Calculator",
"purpose": "Estimate cost/benefit of automation and generated apps",
"inputs": ["runs_per_day", "dev_seats", "model_costs", "hosting"],
"outputs": ["monthly_cost", "savings_estimate"]
}
⸻
- Common Tasks (How‑To)
[
{
"task": "Check service health",
"steps": [
"Open Dashboard → Ops Center",
"Verify Health=UP, Liveness, Readiness",
"Inspect CPU, Heap, Threads and Uptime"
]
},
{
"task": "Enable workflow monitoring",
"steps": [
"Open Workflow Builder",
"Select a workflow and open Settings",
"Select monitoring objects and Save to avoid 403"
]
},
{
"task": "Publish content",
"steps": [
"Go to Content → Editor",
"Compose or open draft, set Status=Published",
"Save; confirm in Published list"
]
},
{
"task": "View workflow errors",
"steps": [
"Dashboard → Ops Center → Recent Errors",
"or Workflow Builder → Logs"
]
}
]
⸻
- Error Catalog
[
{
"code": "HTTP_403_NO_OBJECTS_MONITORED",
"message": "HTTP 403: No objects selected for monitoring.",
"cause": "Monitoring not configured for the selected workflow or objects",
"fix": [
"Open Workflow Builder",
"Select workflow → Settings → Monitoring",
"Choose objects and Save"
]
}
]
⸻
- Troubleshooting Playbooks
[
{
"issue": "High CPU alert (CPU_HIGH)",
"signals": ["system.cpu.usage > 0.9"],
"checks": [
"Dashboard → Ops Center: confirm CPU chart",
"Check Threads and Heap for saturation",
"Open logs for noisy workflows"
],
"actions": [
"Throttle or reschedule heavy workflows",
"Scale service (if supported)",
"Optimize steps using DataOps transforms"
]
},
{
"issue": "No workflow metrics visible",
"signals": ["Empty success/failure dashboards"],
"checks": [
"/actuator/workflowMetrics reachable",
"Workflow monitoring enabled"
],
"actions": ["Enable monitoring", "Run a test workflow"]
}
]
⸻
- Glossary
{
"Visualizer3D": "Interactive 3D graph of workflow nodes/edges",
"Gridheim": "Spreadsheet engine integrated into workflows",
"Actuator": "Spring Boot operational endpoints (/actuator/\*)",
"CPU_HIGH": "Alert when system.cpu.usage exceeds threshold"
}
⸻
- Sample Q&A Snippets (for few‑shot prompting)
[
{
"q": "Where do I see workflow errors?",
"a": "Dashboard → Ops Center → Recent Errors or Workflow Builder → Logs."
},
{
"q": "How do I publish a new blog post?",
"a": "Content → Editor, compose, set Status=Published, Save. Scheduler tab for timed releases."
},
{
"q": "How do I add an alert for high CPU?",
"a": "Dashboard → Ops Center → Alerts → Add metric system.cpu.usage > 0.9 → Save."
},
{
"q": "What’s the 3D view for?",
"a": "Visualizer3D shows the workflow as an interactive node/edge graph for inspection and debugging."
}
]
⸻
- Minimal Schema (entities & relations)
{
"entities": {
"Service": {
"fields": [
"name",
"host",
"status",
"version",
"uptime",
"cpu",
"heap",
"threads"
]
},
"Workflow": {
"fields": ["id", "name", "monitored", "successCount", "failureCount"]
},
"Alert": {
"fields": ["name", "metric", "operator", "threshold", "status"]
},
"Content": {
"fields": [
"id",
"title",
"author",
"status",
"format",
"publishedAt",
"views"
]
},
"User": { "fields": ["id", "name", "roles"] }
},
"relations": [
{ "from": "Workflow", "to": "Alert", "type": "monitors" },
{ "from": "Service", "to": "Workflow", "type": "executes" },
{ "from": "User", "to": "Workflow", "type": "owns" },
{ "from": "User", "to": "Content", "type": "authors" }
]
}
⸻
Usage Notes for LLMs
• Prefer exact tab names and paths from this KB when giving directions.
• If a question references errors/metrics, check Ops Center first; for authoring/publishing, check Content.
• If monitoring data is missing, advise enabling monitoring in Workflow Builder.
• When in doubt, guide users to Universal Search with a concrete example query.
PRIMARY PURPOSE IS TO PROVIDE TECH SUPPORT FOR THE VALKYR STACK, SALES GUIDANCE AND SUPPORT TO THE USER AND TO GET THEM TO BUY THE RIGHT PRODUCT FOR THEIR NEEDS
## USEFUL LINKS TO ALWAYS PROVIDE THE USER:
- valkyrai dashboard:
https://valkyrlabs.com/dashboard
- user preferences:
https://valkyrlabs.com/user-preferences
- product pricing and details: https://valkyrlabs.com/pricing
- full documentation:
https://valkyrlabs.com/v1/docs
# HERE IS YOUR KNOWLEDGE BASE: REFERENCE LIBERALLY AND MEMORIZE AND CROSS REFERENCE FANATICALLY
```bash
░▒▓█▓▒░░▒▓█▓▒░░▒▓██████▓▒░░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓███████▓▒░ ░▒▓██████▓▒░░▒▓█▓▒░
░▒▓█▓▒░░ ▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▒▒▓█▓▒░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▒▒▓█▓▒░░▒▓████████▓▒░▒▓█▓▒░ ░▒▓███████▓▒░ ░▒▓██████▓▒░░▒▓███████▓▒░░▒▓████████▓▒░▒▓█▓▒░
░▒▓█▓▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓██▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓████████▓▒░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░ ▒▓█▓▒░▒▓█▓▒░
v0.6.0BETA :: https://valkyrlabs.com/vai :: San Francisco, California :: ©2025 Valkyr Labs Inc
API-Native Agentic Services Engine
⭐⭐⭐ open tasks ⭐⭐⭐
Welcome to ValkyrAI, an innovative platform that leverages artificial intelligence to streamline and automate business processes. Built on the robust capabilities of ThorAPI, ValkyrAI accelerates the development of secure and efficient APIs by transforming OpenAPI specifications into fully-functional backend and frontend codebases.
Generate early, generate often.
-- John McMahon,
Founder & CEO, Valkyr Labs Inc
Overview of ValkyrAI
ValkyrAI is an innovative platform that leverages artificial intelligence to streamline and automate business processes. Built upon the robust capabilities of ThorAPI, it accelerates the development of secure and efficient APIs by transforming OpenAPI specifications into fully-functional backend and frontend codebases.
With ValkyrAI, developers can rapidly generate Java Spring Boot services and TypeScript clients, significantly reducing development time while maintaining high standards of code quality and security. It's like having a superpower for your coding needs!
Features
Core Features
-
Java Data Model Generation: Create comprehensive Java data models annotated for Spring Boot and JPA, facilitating seamless integration with your backend services. Say goodbye to tedious manual coding!
-
TypeScript Client Generation: Produce fully-typed TypeScript object models and REST clients, empowering your frontend applications with reliable and consistent API interactions. No more guesswork—just type safety!
-
Spring Boot Service Generation: Generate robust Spring Boot services complete with CRUD operations and JPA integration, accelerating your backend development. It's like having a personal assistant for your backend!
-
Database Connectivity: Easily connect to various relational databases via JPA, with support for multiple database platforms. Your data is just a connection away!
-
REST API Documentation: Integrate with SpringDoc OpenAPI to automatically generate interactive API documentation. Because who doesn't love well-documented APIs?
Advanced Features
-
Encrypted Fields via ThorAPI SecureField: Secure sensitive data with field-level encryption, protecting information both at rest and in transit. Your secrets are safe with us!
-
Automatic Relationship Serialization: Seamlessly handle one-to-many and many-to-many relationships, simplifying data modeling and serialization. Relationships made easy!
-
Secure React/TypeScript Component Libraries: Generate secure, data-bound React components to build robust frontend interfaces with ease. Frontend security has never been this simple!
-
Professional Documentation Site: Generates all your API and model object documentation inline to a professional and dynamic documentation site powered by Docusaurus. Because your documentation deserves to shine!
Connectors (ExecModules)
ValkyrAI includes a growing set of pluggable ExecModules (connectors) that you can attach to workflow Tasks. Starter set:
-
Email: Mailtrap Blast (icon: react-icons
FaEnvelope)- Class:
com.valkyrlabs.workflow.modules.MailtrapSendModule - Batches Mailtrap sends; supports concurrency, retries, dry-run; emits
sendStatsandfailures.
- Class:
-
Email: Basic Sender (icon:
FaEnvelope)- Class:
com.valkyrlabs.workflow.modules.EmailModule - Minimal sender for demos; prefer the Mailtrap Blast module for production.
- Class:
-
Payments: Stripe Checkout (icon:
SiStripe)- Class:
com.valkyrlabs.workflow.modules.payments.StripeCheckoutModule - Builds Stripe Checkout sessions from
SalesOrder+LineItems; returnssession_idandcheckout_url.
- Class:
Tip: In Workflow Studio, icons are auto-picked from react-icons based on class name (e.g., “stripe” → SiStripe, “email” → FaEnvelope).
See docs under web/typescript/valkyr_labs_com/docs/docs/Products/ValkyrAI/Workflow Engine/Execution-Modules/.
Query By Example (QBE)
ValkyrAI supports Query By Example across generated model endpoints and UI tables.
-
Backend: Java/Spring controllers accept an optional
examplequery parameter onGET /<Model>.- The value is a URL-encoded JSON object representing a partial instance of the model.
- Matching uses Spring Data
ExampleMatcher:- Strings: case-insensitive CONTAINING (substring)
- Non-strings: exact match
- Example:
/Address?example=%7B%22city%22%3A%22fran%22%2C%22state%22%3A%22CA%22%7D
-
Frontend: generated Redux Query services accept
exampleoptionally:useLazyGet<Model>sPagedQuery({ page, limit?, example? })useGet<Model>sQuery({ example? })
-
UI: each model table has a “Filter (QBE)” modal in the floating toolbar for pasting JSON; infinite scroll and keyboard paging preserve the filter.
Performance tips
- Index frequently filtered fields, and keep result windows small with paged fetches.
- Prefer exact matches for identifiers and narrow text fields to reduce scan cost.
Built on ThorAPI: The Backbone of ValkyrAI
ThorAPI is an opinionated code generator designed for creating secure, performant, and efficient APIs. As a Java Maven project, ThorAPI transforms your OpenAPI specifications into:
-
Spring Boot REST APIs: Accelerate backend development with automatically generated RESTful services.
-
TypeScript Client APIs: Streamline frontend development with generated TypeScript clients that ensure type safety and consistency.
ThorAPI harnesses industry-standard tools and frameworks, including Maven, OpenAPI Generator, Handlebars templating, Spring, JPA, REST APIs, TypeScript, and Java. It provides out-of-the-box persistence via a JPA service, enabling rapid development of full-stack applications.
ThorAPI bundle pipeline
- Specs are now composed through declarative bundle manifests (for example
rbac-core,ecommerce,invoicing,customer-support). - Supply
thorapiBundlesin the generate request configuration JSON to opt into additional bundles. Values are comma or semicolon separated (e.g.{ "thorapiBundles": "ecommerce,customer-support" }). - Bundles may ship OpenAPI fragments, UI components, workflow metadata, and static assets; everything is staged with the generated code and recorded in
thorapi-bundle-manifest.jsoninside the download. - The baseline RBAC/user-preferences bundle is applied automatically unless you pass
thorapiBundles: "none".
Roles, Authorities, and Security Context
- Spring Security makes access decisions against granted authorities. Every role we ship is expressed as a
GrantedAuthoritystring (ROLE_ADMIN,ROLE_SYSTEM, etc.), so theauthorityListon aPrincipalis the canonical source Spring reads during authorization. - The optional
roleListthat ThorAPI generates provides richer metadata (enums, descriptions, audit fields). Keep it in sync withauthorityListwhen you introduce new roles so each role still manifests as aROLE_*authority. - You can represent sharing groups or fine-grained permissions as additional authorities (for example
workflow:execute,group:family-photos) without creating companion role records. Spring will treat them the same as traditional roles.
Security context helpers
- Background schedulers and integration threads should not manually call
SecurityContextHolder.clearContext(). Instead, wrap work insystemUserService.runAsSystem(() -> { ... })orrunAsAssistant(...)(defined invalkyrai/src/main/java/com/valkyrlabs/valkyrai/service/SystemUserService.java). The helper installs a temporaryThorUsercontext, executes the task, and restores the previous authentication automatically. - For dynamic role switching inside workflow modules, use
WorkflowSecurityContext.runAsRole("system", () -> { ... })which delegates to the same helpers.
ROADMAP: Liquibase support
Getting Started
Development with ValkyrAI is simple once you get the hang of the CodeGen development practices. We recommend you review the development best practices before getting too far into development.
The ValkyrAI Development Best Practices Manual
System Requirements
If you need to install this software before you begin, ensure you have at least the following minimum installed:
ValkyrAI for development you will need:
- Linux/Windows/MacOS with Java 17+ @OpenJDK17+ installed
- 2GB available RAM
- working node and npm (for TypeScript client development): Download Node.js
- working git installed / github account and credentials
- working maven installed
- recommended: docker desktop
Deploying generated ValkyrAI apps requires a minimum:
- Linux/Windows/MacOS server with Java 17+ @OpenJDK17+ installed
- 2GB available RAM
Install Java Development Kit (JDK)
While ThorAPI and ValkyrAI are compatible with Java 11+, we recommend using the latest tested version available:
Download JDK v23 (recommended version) From Oracle
On MacOS you would typically brew install command.
# in re newer versions we have tested up to 23
brew install java23
Once you have java installed, you can grab maven and be up and running.
-
Apache Maven 3.6 or higher: Download Maven
-
Java Development Kit (JDK) 11 or higher: Download JDK
-
Apache Maven 3.6 or higher: Download Maven
-
Node.js and npm (for TypeScript client development): Download Node.js
-
ThorAPI Enhancer tool: Available from the ThorAPI repository.
NOTE: there is a Maven build option to install and run the node generated stack -- and to automate the building of Typescript client library. However in practice this is a very slow step and hinders iteration time while developing the backend services that actually require a build step.
NOTE: There are definite best practices to adopt in terms of project setup, development methods, and how to manage front-end vs. backend development.
The ValkyrAI Development Best Practices Manual
Generation Steps
Follow these steps to generate your API and client code:
-
Customize Your OpenAPI Specification
- Edit the 2 OpenAPI spec template files:
src/main/resources/openapi/api.yamlandsrc/main/resources/openapi/api.hbs.yaml - Define which endpoints are exposed in
api.yaml - Create your detailed Model Schema in
api.hbs.yaml - Include ValkyrAI-specific annotations such as
x-thorapi-secure-fieldandx-thorapi-data-field. - ThorAPI will generate the CRUD operations for your schema model objects
- Edit the 2 OpenAPI spec template files:
-
Generate Code
- Spring Boot Service:
- Generate RESTful CRUD APIs with integrated JPA backend.
- TypeScript Client:
- Generate a fetch-based API client.
- Include end-to-end validation with Jest testing.
- Spring Boot Service:
-
Test the Service Integrations
-
Your generated code may be referenced in other projects, or in the monorepo, update and test all affected code and systems
-
Your APIs may be called from remote systems. You will need to ensure that your versioning reflects breaking changes to your spec, and that you retain backwards compatibility and migration information for all API consumers.
-
Ideally, you maintain best practices around communicating changes to your API consumers.
-
-
Run the Service
- Deploy your service as Spring microservices.
- Connect to your preferred database and configure settings as needed.
Versioning Your API
APIs, SDKs, and code libraries require versioning information to allow for changes and updates to software that other systems depend upon without creating chaos or breaking production systems.
If you are using ValkyrAI or ThorAPI, you are now an API and library publisher, and you are going to have to be cognizant of this requirement and handle the publishing of your changes accordingly.
While versioning is outside of the scope of this README, we follow the traditional SEMVER best practices as much as possible.
ROADMAP: automatic SEMVER / changelog feed service
Non-Breaking Changes (increment minor versions) These are changes that ADD detail to your OpenAPI spec, such as putting a description on a property or enhancing the example for a property.
In general, these types of changes will not break or cause problems in existing applications as long as there are no unintended side effects to making the changes (such as memory usage ballooning causing OOME).
Breaking Changes
If you change the name of Objects or their properties, remove or change objects or properties, or truncate the size of or data precision of existing database columns, you will need to change the major version of your API to allow existing applications to continue using the older versions of your code.
When you increment the major version/update endpoint version, you now have 2 APIs to manage, which is why it is important to have migration plans for API consumers as well as detailed usage statistics to determine when it is safe to deprecate and remove old running API code.
Project Setup
Out of the box, ValkyrAI is a Java Maven Project Monorepo with some unique features:
- A TypeScript + React web app under:
/src/main/typescript/<projectname> - GENERATED TypeScript React + Redux/RTQ + Bootstrap + Formik web app client code under:
/src/main/typescript/<projectname>/src/thor - GENERATED Spring Boot web app under:
/generated/spring - A convenient command line runner
./vai
With this setup, ValkyrAI is immediately ready to produce a functional full-stack application with security, usability, and AI-powered functionality at its core.
The ValkyrAI Development Best Practices Manual
OpenAPI Spec Enhancement
In the first step of the generation process, ThorAPI enhances your spec with a variety of features:
-
CRUD Rest Endpoints for POST/PUT/GET/DELETE for each Schema Component (Model Object) in the api.yaml spec input file.
-
Detailed and enhanced Schema Components (Model Objects) in the api.hbs.yaml spec input file.
-
These 2 fully customizable spec templates are merged and enhanced with mandatory "metadata" fields for every object (lastModifiedById, keyHash, createDate, etc.).
Use ThorAPI to process and merge your OpenAPI templates.
Input files:
src/main/resources/openapi/api.yaml: Your base OpenAPI specification.src/main/resources/openapi/api.hbs.yaml: The ThorAPI Handlebars template.
Command:
java -jar lib/generator-<valkyrai_version>-exec.jar \
src/main/resources/openapi/api.yaml \
src/main/resources/openapi/output-api.yaml \
src/main/resources/openapi/api
ThorAPI leverages existing tools like Maven and OpenAPITools CodeGen, Handlebars templating, Spring, JPA, REST APIs, TypeScript, and Java.
Enhance OpenAPI Spec
Run the ThorAPI to process the 2 OpenAPI templates.
The 2 templates are merged and enhanced with the special ThorAPI fields -- with the api.yaml determining which model objects will have REST endpoints generated.
> src/main/resources/openapi/api.yaml
> src/main/resources/openapi/api.hbs.yaml
/usr/bin/env java -jar lib/generator-<valkyrai_version>-exec.jar src/main/resources/openapi/api.yaml src/main/resources/openapi/output-api.yaml src/main/resources/openapi/api
[ERROR] /Users/johnmcmahon/workspace/2025/valkyr/ValkyrAI/generated/spring/src/main/java/com/valkyrlabs/api/McpMarketplaceItemRepository.java:[103,91] type com.valkyrlabs.model.McpMarketplaceItemTag does not take parameters
[ERROR] -> [Help 1]
Generating the Output Project Run Maven in the root of the project to clean and install the project and run the CodeGen process for both Java Spring and TypeScript:
mvn clean install
This build will generate the backend Spring Boot service code under the generated/spring/ folder.
The generated project is a Maven Spring Boot project that provides the generated API via an executable jar.
You can build the executable jar and (optionally) run it using Maven.
cd generated/spring
# run with defaults (h2 db)
--spring.datasource.url="$SPRING_DATASOURCE_URL" \
--spring.jpa.hibernate.ddl-auto=update \
--spring.datasource.username="$SPRING_DATASOURCE_USERNAME" \
--spring.datasource.password="$SPRING_DATASOURCE_PASSWORD" \
--spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver \
--server.port="$SERVER_PORT" \
--jwt.secret="$JWT_SECRET" \
|| error_exit "Failed to run backend instance"
THORAPI_SECRET_KEY is a MANDATORY environment or command line variable that is used to encrypt the SecureFields. Keep this private key secure.
A new THORAPI_SECRET_KEY can be generated with the generatekey command using the following command line:
java -jar lib/generator-<valkyrai_version>-exec.jar generatekey
export THORAPI_SECRET_KEY=generatedPrivateKey
// build the generated project
mvn clean install -f generated/spring/pom.xml
Advanced Topics:
Given N correct OpenAPI specs, you should be able to perform any operation between systems using only transform layers.
-- John McMahon, Founder of Valkyr Labs Inc
SecureField Notes
SecureField is a powerful encryption mechanism provided by ThorAPI that ensures sensitive data is protected both at rest and in memory. By intercepting and wrapping field access, SecureField offers the following benefits:
-
Data Encryption: Automatically encrypts data when storing it in the database, safeguarding against unauthorized access.
-
On-Demand Decryption: Decrypts data transparently when it is accessed within your Java code, providing seamless integration.
-
Enhanced Security: Adds an additional layer of protection against data breaches, including raw data dumps or memory inspection.
SecureField also supports password hashing and plays a crucial role in ThorAPI's user authentication and authorization functionality. It delivers a standards-based, secure implementation for permissions and Access Control Lists (ACLs), enabling robust security management within your applications.
SecureFields are encrypted with the THORAPI_SECRET_KEY which you provide as a command line or environment variable.
WARNING: YOUR DATA WILL BE PERMANENTLY LOST IF YOU LOSE OR OTHERWISE BUNGLE THE THORAPI_SECRET_KEY
Using Secure Field in Your Schema
To utilize SecureField in your OpenAPI schema, simply add the x-thorapi-secureField: true annotation to any field you wish to encrypt. For example:
components:
schemas:
User:
type: object
properties:
id:
type: string
format: uuid
username:
type: string
email:
type: string
format: email
x-thorapi-secureField: true
This default SecureField is encrypted with modern and reasonably secure symmetric key algorithm, able to be unencrypted by anyone with the THORAPI_SECRET_KEY.
For more advanced cases you can directly apply the @SecureField annotation with it's straightforward configuration.
These less common use cases include if you need a higher strength symmetric encryption (more entropy and iterations added to the hash function) -- or if you need to create a password field using a HASHing algorithm that cannot be decrypted but can only be used for matching.
Both hashing (password style) and symmetric encryption are supported by @SecureField.
---
password:
minLength: 8
type: string
description: Your account password
format: password
example: HardToGuess1980
x-field-extra-annotation: "@SecureField(encryptionType = SecureField.EncryptionType.HASHED, strength = 10)"
ValkyrAI builds upon ThorAPI's capabilities by integrating comprehensive user management, ACL management, key rotation, and upcoming OAuth services, providing a holistic solution for secure API development.
ValkyrAI is a partially managed solution that combines ThorAPI CodeGen with comprehensive User management, ACL (access control lists) management, Key rotation and OAUTH services are coming soon to the ValkyrAI service.
Technical Notes on SecureField
***Column Size for Encrypted Data
One factor in the use of SecureField feature is that an encrypted value is much larger (often 2x or more) the size of the same plaintext value.
ThorAPI generates column sizes to accommodate these encrypted values as well as validations on these columns that accommodate the size of the plaintext.
This results in validations on field size that are sized differently to accommodate the larger size of encrypted data.
Auto-Generated Fields
Do not include the following auto-generated fields in your OpenAPI specification components:
idmodified_datecreated_dateowner_idkey_hashlast_modified_by_idlast_modified_datelast_accessed_by_idlast_accessed_date
These fields are automatically managed by ValkyrAI and ThorAPI. Including them in your spec may lead to conflicts or unexpected behavior.
NOTE: in terms of privacy commitments, you will need to communicate to end users of your systems that -- short of heavily reducing the functionality of the system -- this data will be automatically collected. In some use cases your generated applications may run up against regulatory or other conflicts with the collection of precise 'tracking' data.
NOTE: it is COMPLETELY YOUR RESPONSIBILITY to ensure that your data collection practices via any ThorAPI generated code are legal and in conformance with your community standards.
Privacy Benefits of ThorAPI: concerns about surveillance and collection of too much activity auditing by encrypting PII data we can provide much more reliable security, but without the additional metadata there is a loss of traceability and much of the 'fingerprint' or 'footsteps' of any unauthorized use of the system, including data breaches. By maintaining high level of visibility into the data, we can more easily "wipe" data and comply with data and privacy standards.
Determining Service/Delegate API Output Files
The api.yaml file determines which Model Objects in your API spec will have generated endpoints.
NOTE: We refer to the collective exposed API endpoints as the "surface area" of your API -- visualizing it this way lends itself to a security posture. The "attack surface" of your APIs will be determined by the exposed endpoints.
If all you have in your API folder for a generated object is the repository, it is likely that you did not list the object in the api.yaml input file.
If there is a definition in the api.hbs.yaml file, then ThorAPI will generate the object model, but to make the objects available to the "surface area" of your exposed API, you will need to include the item explicitly.
api.yaml
---
## My Game - SPICES HAVE NO PUBLIC API
GamePlayer:
GameWorld:
In your data model, you will need the GameSpice object so you define it alongside the exposed endpoint model objects:
api.hbs.yaml
---
## My Game
GamePlayer:
type: object
description: game player object
properties:
name:
type: string
GameWorld:
type: object
description: game world object
properties:
name:
type: string
players:
type: array
items:
$ref: "#/components/schemas/GamePlayer"
spices:
type: array
items:
$ref: "#/components/schemas/GameSpice"
GameSpice:
type: object
description: game hidden spice object
properties:
spiceName:
type: string
spices:
type: array
items:
$ref: "#/components/schemas/GameSpice"
GameSpice:
type: object
description: game hidden spice object
properties:
spiceName:
type: string
SQL Error: 3854, SQLState: HY000
2024-11-10T11:58:15.975-08:00 ERROR 51718 --- [nio-8081-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : Cannot convert string '\x05\x86k\x93\x0A\xD8...' from binary to utf8mb4
2024-11-10T11:58:15.989-08:00 WARN 51718 --- [nio-8081-exec-2] .w.s.m.s.DefaultHandlerExceptionResolver : Resolved [org.springframework.http.converter.HttpMessageNotWritableException: Could not write JSON: JDBC exception executing SQL [select pr1_0.formula_id,pr1_0.id,pr1_0.created_date,pr1_0.formula_id,pr1_0.key_hash,pr1_0.last_accessed_by_id,pr1_0.last_accessed_date,pr1_0.last_modified_by_id,pr1_0.last_modified_date,pr1_0.owner_id,pr1_0.reference,pr1_0.sheet_id,pr1_0.type from ptg_ref pr1_0 where pr1_0.formula_id=?] [Cannot convert string '\x05\x86k\x93\x0A\xD8...' from binary to utf8mb4] [n/a]]
Spring Security ACL
One of the most powerful ValkyrAI features is RBAC (Role Based Access Control) and Object-level (ie: row-level) security, allowing for secure sharing of data throughout and the ability to leverage permissions to rapidly build secure apis and UX experiences.
The implementation of Spring Security ACL is fairly standard and a deep dive into this technology is outside of the scope of the document.
For a quick overview of the Spring Security ACL implementation please read the following:
The basic concept to understand is that ValkyrAI uses ThorAPI to generate the relevant ACL database tables and objects (acl_xxx tables). So you will see these tables in the base OpenAPI spec.
For obvious reasons you will want to secure any generated endpoints.
ValkyrAI Permissions
For not-logged in, anonymousUser:
-
hasPermission() returns false by default
-
hasPermission(
ObjectIdentityInstance, 'read') returns true if there is an anonymousUser:read:ObjectIdentityInstanceAclEntry -
As a safeguard ENCRYPTED FIELDS NEVER DECRYPT FOR ANONYMOUS (otherwise why encrypt?) -- trying to give anonymous the VIEW_DECRYPTED permission on anything should fail and output a log warning to console as it's a SCHEMA issue.
-
APPEND permissions for some objects are sometimes necessary, anonymousUser needs to post to the Rating endpoint to submit ratings for various entities, but without logging in or having a Principal in the system.
We use table-scope support in the ACL service so blank/“TYPE” IDs map to a deterministic namespace UUID and we can store APPEND at the class level.
The AuthController logic normalizes ACL grant/revoke/deny requests and falls back to the type-scope identifier, making /v1/auth/acl/* work when the caller omits the object ID for APPEND.
POST security is aligned with the ACL model: the Rating endpoint checks hasPermission(null, 'com.valkyrlabs.model.Rating', 'APPEND') and Spring Security allows anonymous posts through once that ACE exist
The runtime evaluator now recognizes missing IDs as table-level APPEND checks and passes the synthesized object identity through to ACL reads; the companion test exercises the anonymous append path
Tests:
mvn -pl valkyrai test -Dtest=com.valkyrlabs.valkyrai.controller.AuthControllerPermissionMappingTest -Dsurefire.failIfNoSpecifiedTests=false
mvn -pl thorapi test -Dtest=ValkyrACLTest,ValkyrAIPermissionEvaluatorTest
Next steps:
Grant anonymousUser the table-level APPEND ACE (POST /v1/auth/acl/grant with objectType = com.valkyrlabs.model.Rating, empty objectId, permission = APPEND).
Confirm the POST guard by calling /v1/Rating anonymously once that ACE is in place.
USE OBJECT-LEVEL PERMISSIONS WITH CAUTION
For logged in, non ADMIN user:
-
hasPermission() returns false by default
-
hasPermission() returns true if there is an
<authenticatedPrincipal.id>:<permission>:ObjectIdentityInstanceAclEntry -
ENCRYPTED FIELDS DECRYPT if there is a VIEW_DECRYPTED permission .
-
SYSTEM user and group falls under this category, so system services agents and controllers can be given permission to decrypt sensitive data as needed but not by default and never as ADMIN
For logged in, ADMIN user:
-
hasPermission() returns true by default FULL STOP
-
ADMINs can do ALMOST ANYTHING in the system so always return true for any permissions request and always decrypt.
-
For performance, isAdmin and isOwner checks happen before AclEntry lookups... fast return first, isAdmin is a quick ROLE check on the authoenticatedPrincipal so no data lookup needed. isOwner is a fast lookup on the object which may also be loaded in memory already do isOwner checks next. Then we do the Acl lookups.
-
ADMINS cannot delete their own Principal record.
-
ADMINS cannot delete their own ROLE record.
ALL OTHER ACCESS IS per the restrictions in SecurityConfig.java
Customizing Spring Security
The default Spring Security configuration is located in:
/src/main/java/com/valkyrlabs/config/SecurityConfig.java
This is a standard Spring Security configuration class. You can customize the security rules to fit your application's requirements.
You can create custom Roles in the database using the generated REST api.
The default roles created are:
...
[ROLE_ANONYMOUS]
[ROLE_EVERYONE]
[ROLE_ADMIN]
[ROLE_USER]
...
You can use the auth system to authorize methods in your custom Java code:
/**
* This PreAuthorize checks the * DataObject (Object must have * a "getOwner()" method)
*/
@PreAuthorize("isOwnerById(#objectId, #objectType)")
public void mySecureMethod(Long objectId, String objectType) {
// Method implementation
}
...
The DataObject Interface (Java)
This interface is implemented by all of the generated model objects by default.
This is the interface that also defines the "decorated" data fields which are added to the OpenAPI spec during the enhancement step.
Because we have a reliable implementation throughout the model graph, we are able to use the DataObject interface to handle any object in our system generically, yet with the benefit of built in Entity mapping and handling.
For example, if you wish to create a property on a model object representing a container of any other object in the system, you can directly reference the DataObject as the data type in the spec and thus be free to populate this container at runtime.
Customizing ThorAPI CodeGen Templates
Locally modifiable templates can be found in:
/src/main/resources/templates
The currently implemented templates are JavaSpring and Typescript-Redux-Query:
You may implement any number of generators in your project, simply add an appropriate generator section to the pom.xml file in the root of the project.
pom.xml
<plugin>
<groupId>org.openapitools</groupId>
<artifactId>openapi-generator-maven-plugin</artifactId>
<version>7.5.0</version>
<executions>
...
<execution>
<id>api.client</id>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<generatorName>typescript-redux-query</generatorName>
...
<configOptions>
...
</configOptions>
</configuration>
</execution>
...
Deployment Section
Running the Generated ValkyrAI Service:
Option 1: Run with Custom Database Settings
Run the service by specifying your database configuration:
java -jar generated/spring/target/valkyrai-api-<valkyrai_version>.jar \
--spring.datasource.url=jdbc:mysql://your.database.com:3306/valkyrai \
--spring.datasource.username=dbmasteruser \
--spring.datasource.password=YOUR_DB_PASSWORD \
--spring.jpa.hibernate.ddl-auto=update \
--spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQLDialect \
--server.port=8081
Option 2: Run via Maven with Default H2 Database
If you prefer to use the default in-memory H2 database OR if you have set the environment variables properly you can simply run:
mvn spring-boot:run
Option 3: Run via Maven with Custom Database Parameters
To connect to a different database while running via Maven, pass the parameters as arguments:
mvn spring-boot:run -Dspring-boot.run.arguments="\
--spring.datasource.url=jdbc:mysql://your.database.com:3306/valkyrai \
--spring.datasource.username=dbmasteruser \
--spring.datasource.password=YOUR_DB_PASSWORD \
--spring.jpa.hibernate.ddl-auto=update \
--spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQLDialect \
--server.port=8081"
Environment Variables:
// SECURITY: Rate limiting configuration
@Value("${valkyrai.rate-limit-rpm:60}")
private static int MAX_REQUESTS_PER_MINUTE;
@Value("${valkyrai.rate-limit-max-fails:5}")
private static int MAX_FAILED_ATTEMPTS;
@Value("${valkyrai.rate-limit-lockout-minutes:10}")
private static long LOCKOUT_DURATION_MINUTES;
Reserved Words
Avoid using reserved words as identifiers in your OpenAPI specification to prevent naming conflicts and code generation issues.
DataBase Reserved Words
You will need to avoid using a variety of reserved words for your object schemas depending upon which database you are using.
For example, here we try to create a property named "schema" -- which fails when Spring JPA tries to create a MySQL column:
2024-11-08T12:30:53.547-08:00 WARN 99739 --- [ main] o.h.t.s.i.ExceptionHandlerLoggedImpl : GenerationTarget encountered exception accepting command : Error executing DDL "alter table exec_module add column schema varchar(3000)" via JDBC [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'schema varchar(3000)' at line 1]
org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL "alter table exec_module add column schema varchar(3000)" via JDBC [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'schema varchar(3000)' at line
Reserved Words (Partial List):
- OpenAPI/YAML:
meta,class - URL Characters:
%,?,*,@,!,(,) - Java Keywords:
String,Float,Integer,Object,Long,Array,Boolean - SQL Keywords:
UNION,JOIN,SELECT,INSERT,UPDATE,DELETE - ThorAPI Objects:
ApiUtil,MediaType,Mono,Content
Typescript-Redux-Query Reserved Words
Workarounds:
If you must use a reserved word, modify it to avoid conflicts. For example:
"Content"➔"ContentData""Mono"➔"MonoResource"
Note: Be mindful of capitalization. ThorAPI may alter the case formatting of identifiers, which can affect your code if specific capitalization is required.
OpenAPI Workarounds
Sometimes you can simply use the OpenAPI workarounds to meet your naming requirements.
Here is an example of using the "_" underscore to create an aliased property name and then ensuring JPA assigns it to the "class" column in the database binding.
properties:
...
_class:
type: string
nullable: false
x-field-extra-annotation: "@Column(name = \"class\")"
required: [_class]
Naming Conventions and Best Practices
-
Avoid Acronyms with Unusual Capitalization: Using acronyms or unconventional capitalization can cause issues during code generation.
-
Apply the KISS Principle: Keep your specifications simple and straightforward to facilitate smooth code generation and maintenance.
-
Prevent Conflicting Methods: You want to avoid create conflicting
GET,PUT,POST, orDELETEmethods in your OpenAPI specification. -
You want to avoid Manually Edit Generated Files: Manual changes to generated code may be overwritten or lead to stale builds. Instead, modify the CodeGen templates located in
/src/main/resources/templates. -
Separate Concerns: Use separate OpenAPI specifications to create distinct services for different areas of functionality.
-
Consistent Validation Rules: Ensure your validation rules are logical and consistent. For example, avoid setting a pattern requiring 8 characters while also specifying a minimum length of 11.
-
Simplify Example Strings: Use simple, clear example strings to avoid parsing errors during code generation.
Troubleshooting Section
Encryption Failures
SecureField comes in 2 flavors: ThorAPI built-in and the SecureField KMS which is included with ValkyrAI.
Out of the box, ThorAPI requires setting of the THORAPI_SECRET_KEY value as the only way to change the key. Custom solutions built on top of ThorAPI can of course manage the keys any way you desire.
ValkyrAI on the other hand has a built-in Key Management System, which allows for automated rotation of keys, and the ability to manage multiple keys at once with the same databases.
If there is a problem with the key management, or a key is corrupted or otherwise somehow made unusable, then the SecureField system will not be able to decrypt previously encrypted values and will return a standard error message such as:
{'error':'decryption failure: Tag mismatch'}
Since there are innumerable ways in which key data might be compromised or made unavailable, it is outside of the scope of this README to consider all possibilities.
For this reason it is important that any key created by you and used with SecureField encryption be managed carefully.
NOTE: As long as you have the valid key associated with a piece of encrypted data, you will always be able to decrypt it using the open source ThorAPI decryption utilities.
Typescript Client Libraries and Component
Typescript Output
The generated Typescript client libraries are generally placed under:
src/main/typescript/<projectname>/src/thor
The generated code includes:
- Fully typed models
- Fully typed API clients
- Redux/RTQ slices for each API
- Formik forms for each model object
- Bootstrap components for each model object
- Jest tests for each API
Using the Generated Typescript Client
You can use the generated Typescript client code in your React/Typescript applications by importing the relevant modules.
Customizing RTK Data Handling
Typescript RTK Query: Our current REDUX implementation uses RTK Query. This system provides convenient Service classes that allow for storing and fetching model Objects from the API
The general pattern is to implement your own thin redux store and import the generated reducers to combine with your root reducer.
The RTK Query system is largely independent of your store for better or worse.
You can easily extend the existing RTK Query code, installing your own reducers, middlewares, and services, and mutations, all using the standard RTK Query patters.
Here is an example of using a custom set of mutation and query services:
// custom redux for ACL granting
import {
useGrantAclPermissionMutation,
useRevokeAclPermissionMutation,
useGetRolesQuery,
useGetObjectAclEntriesQuery,
} from "../../redux/services/AclService";
import CoolButton from "@valkyr/component-library/CoolButton";
Typescript Code Generation Issues
If you are using the generated Typescript clients then there should be no issues in utilizing the generated code assuming the ThorAPI backend Java code compiles and is runnable.
That said, if you are modifying any front-end templates there are numerous gotchas.
The following is due to React using the double bracket to define CSS properties, which is the same type of delimiter used by the mustache template engine. So proper CSS code can get mangled when the template engine replaces the section of CSS between the {{}} symbols at which point you get non-compiling react code:
src/main/typescript/valkyr_labs_com/src/thorapi/redux/components/form/WorkflowStateForm.tsx:282:18
280| className="nice-form-control"
281| style=
282| >
| ^
One approach is to use delimiter "swapping" in mustache and use a different delimiter but we find this approach to be needlessly complex.
The preferred and easier solution is to change the template CSS section by separating the brackets with spaces like so:
... // change
style={{ display: 'block', marginBottom: '1rem' }} >
... // to
style={ { display: 'block', marginBottom: '1rem' } } >
...
AspectJ Not Woven
Because ThorAPI and SecureField depend upon AspectJ weaving to perform their operations, an issue with AspectJ weaving is a critical failure and must not be ignored or bypassed.
We are actively watching the AspectJ space for enhancements as well as potentially releasing our custom, updated AspectJ maven plugin.
For reliability reasons both ThorAPI and ValkyrAI perform AspectOf() tests check on the SecureField aspect and if there is any problem with weaving, will fail with the following exception:
Caused by: org.aspectj.lang.NoAspectBoundException: Exception while initializing com.valkyrlabs.valkyrai.controller.SecureFieldAspect: java.lang.NoSuchMethodException: com.valkyrlabs.valkyrai.controller.SecureFieldAspect.aspectOf()
at org.aspectj.lang.Aspects.aspectOf(Aspects.java:50)
at com.valkyrlabs.valkyrai.config.SecureFieldConfig.secureFieldAspect(SecureFieldConfig.java:24)
You might also encounter this error when initializing the AuditingAspect ... the solution is the same, check for any java process that might keep a lock on the Java build files (this haapens with VSCode unfortunately as of 5/25)clean and rebuild using ./vai command or 'mvn clean install'.
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.valkyrlabs.valkyrai.controller.AuditingAspect]: Factory method 'auditingAspect' threw exception with message: Exception while initializing com.valkyrlabs.valkyrai.controller.AuditingAspect: java.lang.NoSuchMethodException: com.valkyrlabs.valkyrai.controller.AuditingAspect.aspectOf()
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.lambda$instantiate$0(SimpleInstantiationStrategy.java:199)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiateWithFactoryMethod(SimpleInstantiationStrategy.java:88)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:168)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
... 26 common frames omitted
This problem will most likely occur when bypassing Maven build of the projects such as when using an IDE with a "javac" debugging compiler.
ThorAPI classes recompiled without AspectJ support are likely to cause data problems such as overwriting encrypted data with cleartext and should not be made available on any classpath.
Other Potential SecureField Issues
BUG ALERT: As of 12/24, SecureField is only partially supported with Validations. We are aware of the inconvenience but for now if you are getting issues inserting data into a securefield and you have a small-ish size validation, you can expect failures when the system attempts to write ciphertext to a 'plaintext-sized' DB column.
2024-12-20T12:55:17.204-08:00 ERROR 644 --- [nio-8080-exec-2] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.dao.DataIntegrityViolationException: could not execute statement [Value too long for column "STATE_IDENTIFICATION CHARACTER VARYING(10)": "'50JvKzHOGFDj7jENnL1LHvbwWAF/np5TICGzRgbE1K70OhEUVL8=' (52)"; SQL statement:
insert into principal (accepted_cookies,accepted_tos,account_enabled,account_non_expired,account_non_locked,avatar_url,bio,created_date,credential_no
...
Workaround is to remove the field size limits for encrypted properties (min and max really as min will not work as expected on content that is 'artificially' larger).
Trying to Launch Spring Boot From a Deleted Generated Folder
If you run ./vai and choose "clean" option then your generated spring directory will be moved to the trash and if you attempt to execute the service from that folder it will likely fail with an error such as:
MacBook-Pro:spring user$ mvn spring-boot:run -Dspring-boot.run.arguments="--spring.datasource.url=jdbc:mysql://ls-asdf.cjbdbnvzknle.us-asdf-2.rds.amazonaws.com:3306/valkyrai?useCursorFetch=true&defaultFetchSize=100&autoReconnect=true&useUnicode=yes&characterEncoding=UTF-8 --spring.jpa.hibernate.ddl-auto=update --spring.datasource.username=dbmasteruser --spring.datasource.password=ItsTh3Sam301Dstuff --spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL8Dialect"
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
pwd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
pwd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
Error occurred during initialization of VM
java.lang.InternalError: platform encoding not initialized
at jdk.internal.util.SystemProps$Raw.platformProperties(java.base@23.0.1/Native Method)
at jdk.internal.util.SystemProps$Raw.<init>(java.base@23.0.1/SystemProps.java:263)
at jdk.internal.util.SystemProps.initProperties(java.base@23.0.1/SystemProps.java:67)
at java.lang.System.initPhase1(java.base@23.0.1/System.java:2165)
You simply need to change directory back to the new generated spring folder. You can typically just step back up into the generated folder and back into the spring folder like so:
...
> cd ..
> cd spring
...
File Contention in Development Builds
Since ./vai and ThorAPI in general do a lot of file moving/deleting/creating typically inline to an active IDE running various parts of the project at any one time.
Sometimes errors can be caused by the fact that an IDE has a source file open, or classpath file is locked for any reason, a path cannot be deleted, or an unexpected condition (inclduing full disks, disk i/o problems, bad path or filename characters, corrupt files, etc. )
These errors list random filenames, often while compiling AspectJ:
(base) MBP-1:ValkyrAI$ mvn clean install -DskipTests
...
[INFO] --- jar:3.4.2:jar (default-jar) @ generator ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:42 min
[INFO] Finished at: 2024-10-23T14:52:07-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:3.4.2:jar (default-jar) on project generator: Error assembling JAR: /workspace/ValkyrAI/target/classes/com/valkyrlabs/model/Rating$AjcClosure399.class -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
(base) MBP-1:ValkyrAI$
The solution is to close all IDE windows, terminals, and kill any process that is possibly holding open a file in the project path.
Then rerun the build:
(base) MBP-1:ValkyrAI$ mvn clean install -DskipTests
...
~/.m2/repository/com/valkyrlabs/thorapi/generator/1.0-SNAPSHOT/generator-1.0-SNAPSHOT-exec.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:02 min
[INFO] Finished at: 2024-10-23T14:42:27-07:00
[INFO] ------------------------------------------------------------------------
(base) MBP-1:ValkyrAI$
Spec Validation Errors: Here we see the Spec is not passing basic OpenAPI validations
[ERROR]
org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
| Error count: 26, Warning count: 7
Errors:
-attribute components.schemas.EventLog.properties is not of type `object`
-attribute components.schemas.ContentData.properties is not of type `object`
-attribute components.schemas.Login.properties is not of type `object
2024-10-15T17:44:18.692-07:00 ERROR 55734 --- [nio-8081-exec-8] com.valkyrlabs.thorapi.audit.AuditingAspect : getCurrentAuditor: java.lang.ClassCastException: Cannot cast java.lang.String to java.util.UUID
CodeGen Skipping Existing Files
Sometimes CodeGen will not overwrite existing files which may be a preference:
[INFO] writing file ~/workspace/ValkyrAI/generated/spring/src/main/java/com/valkyrlabs/api/WorkflowApiDelegate.java
[INFO] Implementation file ~/workspace/ValkyrAI/generated/spring/src/main/java/com/valkyrlabs/api/WorkflowStateApiController.java is not overwritten
You can delete the generated files using the ./vai command and responding Y to the second prompt to clean generated files.
Spring Framework Dependency Incompatibilities
The specific versions of all imported libraries, especially the Spring Framework and AspectJ, have been meticulously selected and tested by hand to ensure version compatibility.
When upgrading any of the libraries in the pom.xml and package.json files, please test and confirm all of the functionality of your system thoroughly.
Missing Required Fields
Ensure all required fields are properly specified in your OpenAPI spec and database schema.
Validation Conflicts
Check for conflicting validation rules in your OpenAPI spec that may prevent data from being processed correctly.
For example:
someData:
maxLength: 10
minLength: 20
type: string
description: this is broken data cannot be inserted
example: no such value exists that is longer than 20 and shorter than 10 thus validation will fail on any value
In addition to making it impossible to insert data, using a minimum value as a validation can also act as a required field -- because there must be a minimum value entered, it becomes a required field, even if that is not the intention.
By proactively addressing these issues, you can prevent runtime errors and ensure a smooth development experience.
Common Spec Problems
String Columns Too Large
Currently there are some boundary issues when choosing these values for string "maxLength"... some databases only support creation of VARCHAR columns which are limited to approx 16k characters in length but when trying to create a 20k size character column, it fails to 'upcast' to a larger 'text' type.
---
bioUrl:
type: string
description: More in-depth information about you and your account
maxLength: 20000
at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:40) ~[valkyrai-api-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
Caused by: java.sql.SQLSyntaxErrorException: Column length too big for column 'bio_url' (max = 16383); use BLOB or TEXT instead
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:112) ~[mysql-connector-j-9.0.0.jar!/:9.0.0]
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:114) ~[mysql-connector-j-9.0.0.jar!/:9.0.0]
at com.mysql.cj.jdbc.ServerPreparedStatement.serverExec
The workaround is to choose a significantly higher size which will force the properly sized column:
---
bioUrl:
type: string
description: More in-depth information about you and your account
maxLength: 100000
Runtime Spec Database Issues
This one seems to be more of a Hibernate/JPA issue in auto creating columns for "boolean" fields.
Somehow field was being created as a bit(16) not a bit(1). Manually altered the table and will be keeping an eye on that one.
exec-9] DEBUG c.v.v.jwt.JwtRequestFilterHardened - Found 2 roles for user: 477ec552-5d9c-4562-9591-28e58dc6094f
17:06:32.129 [http-nio-8080-exec-9] INFO c.v.t.a.DataObjectAuditingListener - DATAOBJECT CREATED:com.valkyrlabs.model.AclObjectIdentity | 7235a238-63ee-4f2d-af8c-38aee60fe90f | userExists: true
17:06:32.250 [http-nio-8080-exec-9] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - Cannot determine value type from string '1'
17:06:32.252 [http-nio-8080-exec-9] ERROR c.v.v.controller.AuthController - ACL grant failed
org.springframework.dao.DataIntegrityViolationException: Could not extract column [9] from JDBC ResultSet [Cannot determine value type from string '1'] [n/a]; SQL [n/a]
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:297)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:256)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:241)
...
Non-Fatal Test Errors
During maven test runs you may encounter Spring JPA DML/DDL issues creating tables for tests using the H2 embedded database engine.
These may or may not impact your testing depending upon whether your test code touches upon the tables that are not able to be created.
Of course it is critical to ensure these errors You want to avoid show up on you production database.
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) [surefire-booter-3.2.5.jar:3.2.5]
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) [surefire-booter-3.2.5.jar:3.2.5]
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) [surefire-booter-3.2.5.jar:3.2.5]
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) [surefire-booter-3.2.5.jar:3.2.5]
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement "create table opportunity (status tinyint check (status between 0 and 3), [*]value float(53), created_date timestamp(6) with time zone, estimated_close_date timestamp(6) with time zone, last_accessed_date timestamp(6) with time zone, last_modified_date timestamp(6) with time zone, customer_id uuid, id uuid not null, last_accessed_by_id uuid, last_modified_by_id uuid, owner_id uuid, description varchar(255), key_hash varchar(255), primary key (id))"; expected "identifier"; SQL statement:
create table opportunity (status tinyint check (status between 0 and 3), value float(53), created_date timestamp(6) with time zone, estimated_close_date timestamp(6) with time zone, last_accessed_date timestamp(6) with time zone, last_modified_date timestamp(6) with time zone, customer_id uuid, id uuid not null, last_accessed_by_id uuid, last_modified_by_id uuid, owner_id uuid, description varchar(255), key_hash varc
...
format: double
description: Expected value of the opportunity.
status:
type: string
enum:
- open
- won
- lost
- inactive
description: Status of the opportunity.
estimatedCloseDate:
...
Inner Classes and Many to Many
We do not yet FULLY support the 'oneOf' operator because it generates inner classes which are then unavailble to the Spec enhancer so they do not become decorated with required DataObject interface fields like getId().
But there are 2 workarounds:
- manually add the fields to the inline object definitions.
- move the inline object definition to a separate object and use a $ref
---
McpToolCallResponse:
type: object
properties:
_meta:
type: object
additionalProperties: true
content:
type: array
items:
oneOf:
- type: object
properties:
type:
type: string
enum:
- text
text:
type: string
required:
- type
- text
- type: object
properties:
type:
type: string
enum:
which results in:
[INFO] Finished at: 2025-02-19T14:02:45-08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.13.0:compile (default-compile) on project generator: Compilation failure: Compilation failure:
[ERROR] generated/spring/src/main/java/com/valkyrlabs/api/McpToolCallResponseContentInnerPageableRepository.java:[42,92] cannot find symbol
[ERROR] symbol: class TypeEnum
[ERROR] location: interface com.valkyrlabs.model.McpToolCallResponseContentInner
[ERROR] generated/spring/src/main/java/com/valkyrlabs/api/McpToolCallResponseContentInnerService.java:[98,125] cannot find symbol
[ERROR] symbol: class TypeEnum
[ERROR] location: interface com.valkyrlabs.model.McpToolCallResponseContentInner
[ERROR] generated/spring/src/main/java/com/valkyrlabs/model/McpResourceResponseContentsInner.java:[58,8] com.valkyrlabs.model.McpResourceResponseContentsInner is not abstract and does not override abstract method setId(java.util.UUID) in com.valkyrlabs.model.DataObject
[ERROR] generated/spring/src/main/java/com/valkyrlabs/model/...
Workaround is to not use an ENUM for the Type discriminator, simply use a String
Here is the case where you can manually add the id, modified_date, etc. to the model to workaround the compilation errors.
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] generated/spring/src/main/java/com/valkyrlabs/model/McpResourceContentResource.java:[58,8] com.valkyrlabs.model.McpResourceContentResource is not abstract and does not override abstract method setId(java.util.UUID) in com.valkyrlabs.model.DataObject
[INFO] 1 error
Final corrected input looks like: Note the use of $ref objects and the use of the "_meta" which will become the name of the Map holding these named objects:
---
McpToolCallResponse:
type: object
properties:
_meta:
type: object
additionalProperties: true
content:
type: array
items:
oneOf:
- $ref: "#/components/schemas/McpTextContent"
- $ref: "#/components/schemas/McpImageContent"
- $ref: "#/components/schemas/McpResourceContent"
isError:
type: boolean
required:
- content
Duplicating Generated Props
You must avoid manually adding duplicate auto-generated fields like id or created_date to your OpenAPI spec.
ThorAPI may or may not dedupe the spec properly and almost certainly this will introduce the potential for bugs and malfunctioning of the code that depends on these fields.
Unsupported OpeanAPI Features
TODO: fix
org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL "create table oas_parameter (id binary(16) not null, created_date datetime(6), description varchar(255), in tinyint check (in between 0 and 3), key_hash varchar(255), last_accessed_by_id binary(16), last_accessed_date datetime(6), last_modified_by_id binary(16), last_modified_date datetime(6), name varchar(255), oas_operation_id binary(16), owner_id binary(16), required bit, primary key (id)) engine=InnoDB" via JDBC [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'in tinyint check (in between 0 and 3), key_hash varchar(255), last_accessed_by_i' at line 1]
name:
type: string
in: # <- bad
type: string
enum:
- query
- header
changed to:
name:
type: string
oasIn: # <- workaround
type: string
enum:
- query
- header
Incompatible Spec Changes Break Validations
ThorAPI creates validated fields on Java objects that the server uses to ensure that data is correct at every stage of interaction.
These validation rules are controlled in your OpenAPI spec by the 'format', 'pattern', 'type', and especially 'minSize', 'maxSize' attributes which all need to be correct when setting and getting values on these fields.
Also using enums in your spec may cause constraint failures if new or changed enum values are introduced in the spec. In this case, the constraints must be dropped and then the app restarted to update the constraint validations.
An additional factor is the use of SecureField feature which creates a validation on a column that is sized differently to accomodate the larger size of encrypted data.
Due to this feature, if your changes to the spec cause a validation on an existing field to change, it may impact the column size or the data type and thus cause a failure during startup or runtime when it encounters the existing column and data.
2024-12-19T10:14:33.373-08:00 ERROR 27745 --- [nio-8081-exec-7] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction] with root cause
jakarta.validation.ConstraintViolationException: Validation failed for classes [com.valkyrlabs.model.LlmDetails] during update time for groups [jakarta.validation.groups.Default, ]
List of constraint violations:[
ConstraintViolationImpl{interpolatedMessage='size must be between 16 and 1024', propertyPath=apiKey, rootBeanClass=class com.valkyrlabs.model.LlmDetails, messageTemplate='{jakarta.validation.constraints.Size.message}'}
]
at org.hibernate.boot.beanvalidation.BeanValidationEventListener.validate(BeanValidationEventListener.java:161)
Object is not an @Entity
Using type: object or other primitives can cause references to fail becuase there is no database object defined to hold the value of the array. Since primitives cannot be described as a database table and column, this will cause runtime failures.
You must use a specific schema defined reference to hold the values of collections (Lists and Maps)
---
properties:
schemas:
type: array
items:
$ref: "#/components/schemas/OasObjectSchema"
securitySchemes:
type: array
items:
type: object
using type: object causes:
itionValueResolver.java:365) ~[spring-beans-6.1.13.jar!/:6.1.13]
... 99 common frames omitted
Caused by: org.hibernate.AnnotationException: Association 'com.valkyrlabs.model.OasComponents.securitySchemes' targets the type 'java.lang.Object' which is not an '@Entity' type
at org.hibernate.boot.model.internal.CollectionBinder.detectManyToManyProblems(CollectionBinder.java:2607) ~[hibernate-core-6.5.3.Final.jar!/:6.5.3.Final]
at org.hibernate.boot.model.internal.CollectionBinder.bindM
to fix we must define a schema to use for the items array. Here we define a new schema OasSecurityScheme:
...
properties:
schemas:
type: array
items:
$ref: '#/components/schemas/OasObjectSchema'
securitySchemes:
type: array
items:
$ref: '#/components/schemas/OasSecurityScheme'
OasSecurityScheme:
type: object
description: a security scheme to access the api
properties:
name:
type: string
description: the name of the security scheme
...
Conflict with JPA Query names
Even with escaping, ending a property name with "in" will cause the generated queries to be thrown off:
---
_in:
type: string
enum:
- query
- header
- path
- cookie
causes...
bs.model.OasParameter$InEnum); Reason: Failed to create query for method public abstract java.util.List com.valkyrlabs.api.OasParameterRepository.findOasParameterByIn(com.valkyrlabs.model.OasParameter$InEnum); Operator IN on in requires a Collection argument, found class com.valkyrlabs.model.OasParameter$InEnum in method public abstract java.util.List com.valkyrlabs.api.OasParameterRepository.findOasParameterByIn(com.valkyrlabs.model.OasParameter$InEnum)
Fix was somewhat invasive until we can create a fix:
---
location:
type: string
enum:
- query
- header
- path
- cookie
Capitalization Issues
Using a component name like:
---
OASSimpleSchema:
type: object
results in:
[ERROR] ~/workspace/ValkyrAI/generated/spring/src/main/java/com/valkyrlabs/api/OASSimpleSchemaApiDelegate.java:[223,86] cannot find symbol
[ERROR] symbol: variable oASSimpleSchema
[ERROR] location: class com.valkyrlabs.api.OASSimpleSchemaApiDelegate
OpenAPI Spec and DB Keys
Here we have an error in the Spec around a foreign-key relationship:
Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2024-10-04T11:59:33.695-07:00 ERROR 16313 --- [ main] o.s.boot.SpringApplication : Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Collection 'com.valkyrlabs.model.BuildUpdate.dependencies' is 'mappedBy' a property named 'buildUpdateId' which does not exist in the target entity 'com.valkyrlabs.model.DependencyUpdate'
so we update the definition in the template:
DependencyUpdate:
type: object
properties:
version:
type: string
scope:
type: string
status:
type: string
enum:
- UP_TO_DATE
- OUTDATED
- MISSING
and we add:
---
type: object
properties:
buildUpdateId:
type: string
version:
type: string
This addition allows spring JPA and Hibernate to determine the foreign key relationship between the objects.
When you create one-to-many or many-to-one relationships in the OpenAPI schema, Spring JPA will auto-generate constraints in the DB:
...
KEY FKkbr705782fp21q0ph1o43gmwc (acl_object_identity_fk_id),
CONSTRAINT FKkbr705782fp21q0ph1o43gmwc FOREIGN KEY (acl_object_identity_fk_id) REFERENCES acl_object_identity (id)
)
...
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Association 'com.valkyrlabs.model.WorkflowState.value' targets the type 'java.lang.Object' which is not an '@Entity' type
Caused by inability to use Object as a 'leaf' entity in the OpenAPI spec (at this time)
---
value:
description: value of the property
type: array
items:
type: DataObject
ROADMAP: implement many-to-many without adding identity columns to the interface
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Collection 'com.valkyrlabs.model.ExecModule.specs' is 'mappedBy' a property named 'execModuleId' which does not exist in the target entity 'com.valkyrlabs.model.OpenApiSpec'
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1806)
Be vigilant for common issues that may arise with your OpenAPI specifications, which can lead to errors during code generation or runtime.
2024-05-18 13:01:22.700 ERROR 3766 --- [nio-8080-exec-4] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.orm.jpa.JpaSystemException: could not execute statement; nested exception is org.hibernate.exception.GenericJDBCException: could not execute statement] with root cause
java.sql.SQLException: Field 'config_id' doesn't have a default value
Possible Errors and Resolutions:
Spec Validation Errors:
The generator validates the input specifications before generating code. If there is a problem you might see an error like this:
[ERROR]
org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
NOTE: it is NOT recommended to skip validation of the specification. Doing so will likely introduce bugs and code that will not compile, if it produces anything at all.
Missing Generated Files
In this case, there are references to 'missing' model components in the generated apis.
Typically this means the api has been simplified and/or changed and there are still obsolete references. Also can happen when copying in definitions from other specs, or due to simple typos and spelling or capitalization errors.
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:314)
[ERROR]
org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
| Error count: 32, Warning count: 6
Errors:
-attribute paths.'/BuildCreate'(post).responses.201.content.'application/json'.schema.#/components/schemas/BuildCreate is missing
-attribute paths.'/BuildUpdate'(get).responses.200.content.'application/json'.schema.#/components/schemas/BuildUpdate is missing
-attribute paths.'/BuildUpdate/{id}'(put).requestBody.content.'application/json'.schema.#/components/schemas/BuildUpdate is missing
Changing those references to the simplified spec clears up the issue in the generated output...
Object Inheritance Support
In this error we are getting TSX errors in classes that are extending or inheriting from a base class via the 'allOf' OpenAPI keyword.
yarn build
Type '(result: void, error: FetchBaseQueryError, { id }: Pick<Solution, "id"> & Partial<Solution>) => { type: "Solution"; id: unknown; }[]'
...
BuildOutput:
allOf:
- $ref: '#/components/schemas/BaseEntity'
- type: object
properties:
buildId:
type: string
format: uuid
...
Solution:
allOf:
- $ref: '#/components/schemas/BaseEntity'
- type: object
type: object
properties:
buildOutputId:
type: string
format: uuid
...
The reality is, because Solution is a member of BuildOutput, it does not need the fields from the base class in any case as they represent duplicate data in the system.
For that reason, it is a simple matter to remove the allOf from 'Solution' and the typescript generation is fixed.
Because this is a "leaf" object in the heirarchy there is no need for it to duplicate the "BaseEntity" fields:
Solution:
type: object
properties:
buildOutputId:
type: string
format: uuid
ROADMAP: support for allOf inheritance functionality in Typescript
Capitalization Problems
Here we see a common pitfall of using a complex capitalization. Because ThorAPI renames to-and-from Java to database schemas, the camelcase and snake case conversions can clobber any capitalzation that is has than one capital letter in a row...
[INFO] --- aspectj:1.15.0:compile (aj-compile) @ valkyrai-api ---
[INFO] Showing AJC message detail for messages of types: [error, warning, fail]
[ERROR] The public type OpenApiSpec must be defined in its own file
./generated/spring/src/main/java/com/valkyrlabs/model/OpenAPISpec.java:73
public class OpenApiSpec {
^^^^^^^^^^
In this case renaming the OpenAPI spec component clears up the issue:
description: ValkyrAI Design Spec
xml:
name: OpenAPISpec
capitalization of "OpenAPISpec" changed to...
description: ValkyrAI Design Spec
xml:
name: OpenApiSpec
Support for OpenAPI additionalProperties
Support for defining properties as "additionalProperties" not yet supported.
The code generator does not handle object representation yet.
- Operation:
type: object
properties:
summary:
type: string
operationId:
type: string
parameters:
type: array
items:
$ref: "#/components/schemas/Parameter"
responses:
type: object
additionalProperties:
$ref: "#/components/schemas/Response"
Using Objects as Properties in the Spec
Suppoted:
...
- underlineDoubleAccounting
color:
type: string
enum:
- Black
- White
...
Not yet suppoted:
...
- colorViolet
- colorGray80
alignment:
type: object
properties:
horizontal:
type: string
...
Workaround is to use a named property instead of an object:
...
- colorViolet
- colorGray80
alignment-horizontal:
type: string
enum:
- Left
- Center
- Right
...
Using Enums in the Spec
Enums are very useful, be sure that all of the specifics exist in the spec:
generated/spring/src/main/java/com/valkyrlabs/model/Role.java:[115,41] cannot find symbol
[ERROR] symbol: variable ANONYMOUS
[ERROR] location: class com.valkyrlabs.model.Role
is caused by a missing default value:
roleName:
type: string
description: the role
enum: [EVERYONE, STAFF, ADMIN]
x-enum-descriptions:
- All Authenticated Users
- Staff Role
- Admin Role
x-enum-varnames:
- ROLE_EVERYONE
- ROLE_STAFF
- ROLE_ADMIN
default: ANONYMOUS <-- this default is NOT in the enum list
WARNING: Enums generate constraints in the databse and may need to be manually changed. This can also cause out of bound exceptions in the databse as they are generated with constraints on valid values -- be sure to drop the database constraints if you change the list of enums!
Deployment Errors
Errors and warnings are to be expected during development while making changes.
Changing Property Text Lengths
One example is if we change the size of a column in the OpenAPI spec by using the 'maxLength' property.
When spring JPA restarts, it will attempt to resize the columns and if there is existing data that is larger than the new column size, the attempt will fail, and the server will continue.
2024-10-29T14:31:49.868-07:00 WARN 5679 --- [ main] o.h.t.s.i.ExceptionHandlerLoggedImpl : GenerationTarget encountered exception accepting command : Error executing DDL "alter table address modify column country varchar(3)" via JDBC [Data truncation: Data too long for column 'country' at row 1]
org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL "alter table address modify column country varchar(3)" via JDBC [Data truncation: Data too long for column 'country' at row 1]
The simple fix for this is to either increase the "maxLength" property in the OpenAPI spec to fit the data, or to truncate the data in the column to the specified new maxLength of the spec.
JPA Errors
While POSTing a complex DataObject to the API we get the following error:
org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [com.valkyrlabs.model.OasOpenAPISpec#464cbd9a-11f8-4162-90e6-5d6c8ce4996d]
at org.hibernate.event.internal.DefaultMergeEventListener.entityIsDetached(DefaultMergeEventListener.java:426) ~[hibernate-core-6.6.2.Final.jar!/:6.6.2.Final]
at org.hibernate.event.internal.De
Database Schema Mismatch
Verify that your database schema aligns with your entity definitions and that migrations have been applied correctly.
Data Truncation of Columns for Text Data
Once you successfully modify the tables you might have the problem of trying to enter data that once worked is now too long. You will need to adjust the size of the column with the maxLength property.
...
[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] threw exception
com.mysql.cj.jdbc.exceptions.MysqlDataTruncation: Data truncation: Data too long for column 'country' at row 1
...
Incompatible Java Libs
Jakarta vs Javax no need to get into the saga, but when we try to use jakarta persistence we are getting:
... 106 common frames omitted
Caused by: java.lang.IllegalArgumentException: methods with same signature getSchemaManager() but incompatible return types: [interface org.hibernate.relational.SchemaManager, interface jakarta.persistence.SchemaManager]
since javax.persistence is open source we are switching back to javax.persistence.
Product Information
SecureKMS
ValkyrAI includes a built-in SecureField key management system.
Default operation is automatic as ValkyrAI manages the key associated with each row of data in your database.
If you change your default key ... which you set with the THORAPI_SECRET_KEY environment variable, the SecureKMS will ensure that previously encrypted records will be decryptable.
The keys are stored in the SecureField database table.
WARNING: YOUR DATA MAY BE PERMANENTLY LOST IF YOU LOSE ACCESS TO THE SECURE_KEYS TABLE OR OTHERWISE BUNGLE THE THORAPI_SECRET_KEY
ROADMAP: pluggable kms
Valor
Valor consists of a supervisor/advisor LLM which may consiste of one or more aggregated LLM services such as OpenAI or Anthropic Claude.
- advise on OpenAI design
- advise on API best practices
- realtime threat monitoring
- evolving stacks (update/self-heal/bugfix)
- learning stacks (model, gather, analyze)
- predictive decisioning
- data analytics
- inline data monitoring intelligence
Here we are testing local OLLAMA setup.
curl -X POST \
http://localhost:11434/api/chat \
-H 'Content-Type: application/json' \
-d '{"text": "Hello", "role": "user"}'
curl http://localhost:11434/api/generate -d '{
"model": "llama3.2:1b",
"prompt": "What is your name and what are you known for?",
"raw": true,
"stream": false
}'
ollama run llama3.2:1b
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2:1b",
"messages": [
{
"role": "user",
"content": "What is your name?"
}
]
}'
VIDAR
Vidar -- named after a Norse god of locking things up and guarding them tight -- is a data encryption utility that is used to zip up and encrypt file system data and create a self-extracting encrypted jar file.
For example, Vidar files can be used to zip and encrypt an entire generated ValkyrAI project -- prior to generation of the final service for compression purposes.
./vidar
Enter a password for encryption:
Zipping the payload directory...
./vai
vai command -- short for ValkyrAI -- is a simple bash script that gives developers an easy way to run the ValkyrAI generator, manage and deploy the front and backend artifacts, generate necessary encryption keys, run the vidar script and more.
./vai
☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮ ️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
..................................................................................................
░▒▓█▓▒░░▒▓█▓▒░░▒▓██████▓▒░░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓███████▓▒░ ░▒▓██████▓▒░░▒▓█▓▒░
░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▒▒▓█▓▒░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▒▒▓█▓▒░░▒▓████████▓▒░▒▓█▓▒░ ░▒▓███████▓▒░ ░▒▓██████▓▒░░▒▓███████▓▒░░▒▓████████▓▒░▒▓█▓▒░
░▒▓█▓▓█▓▒░ ░▒▓█ ▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓██▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓████████▓▒░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️ ©2024 Valkyr Labs Inc ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️ https://valkyrlabs.com/vai ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️ San Francisco, California ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️ support@valkyrlabs.com ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
WELCOME TO THE VAI GUIDED RUNNER
Enter the workspace directory [/workspace/ValkyrAI]:
/workspace/ValkyrAI
Clean generated files from prior run...? (Y/n) [n]:
n
Skipping cleaning generated files.
Enhance the input OpenAPI with ThorAPI extensions...? (Y/n) [Y]:n
n
Skipping enhancing OpenAPI.
Build generator and generate code from enhanced OpenAPI...? (Y/n) [Y]:n
n
Skipping generator rebuild.
Do you want to run the backend instance for development? (Y/n) [Y]:n
n
Skipping backend instance run.
Do you want to build and run the frontend instance for development? (Y/n) [Y]:n
n
Skipping frontend instance run.
Do you want to deploy generated artifacts now? (Y/n) [Y]:n
n
Deployment skipped.
Do you want to SSH into loki.valkyrlabs.com to restart services? (Y/n) [Y]:n
n
Skipping SSH into server.
Do you want to generate a JWT Secret Key? (Y/n) [Y]:n
n
Skipping generating a JWT Secret Key...
Do you want to generate a THORAPI Secret Key? (Y/n) [Y]:n
n
Skipping generating a THORAPI Secret Key...
Do you want to create an encrypted VIDAR file? (Y/n) [Y]:n
n
Skipping generating a VIDAR encrypted file...
Script execution completed successfully.
(base) MBP-1:ValkyrAI $
GridHeim
GridHeim is an online spreadsheet component that works with ThorAPI REST apis.
The GridHeim component allows you to work with Spreadsheets via a secure REST api, as well as embed spreadsheet functionality in your applications.
Used internally by ValkyrAI and Valor, GridHeim is the ultimate end-user development UI allowing for easy building of data-management systems, formula-driven business logic, logic tables, dashboards and reports.
ROADMAP: full gridheim implementation
Resources
Workflow Engine Architecture
The ValkyrAI WF Engine works via a Quartz scheduler that executes Workflows on a schedule at intervals.
Workflows can also be run and managed on demand via REST endpoints.
Workflows consist of a series of Tasks which have a series of Execution Modules and Data Handlers that perform a variety of automated functions.
When run by Valor and via a collection of basic Exec Modules and in conjunction with OpenAPI Specs, the WF Engine is capable of nearly any task of any complexity.
Quartz scheduling is determined by a "CRON" string and can be set via the it's normal ThorAPI.
The api.hbs.yaml Spec Template
An example OpenAPI spec schema model in the api.hbs.yaml:
ChatResponse:
type: object
description: Chatresponse from the other side of the communication
properties:
content:
maxLength: 100000
minLength: 10
type: string
description: the response fro the chat service
example: We must cross the mighty Mississippi
... and the resulting output after running ThorAPI enhancement:
ChatResponse:
type: object
description: Chatresponse from the other side of the communication
properties:
content: { maxLength: 100000, minLength: 10, type: string, description: the
response fro the chat service, example: We must cross the mighty Mississippi, x-field-extra-annotation: "@Column(length = 100000)" }
id:
type: string
description: Unique identifier for object in the system
example: 8c8a76cc-191d-403c-a017-a7312fa60d82
format: uuid
x-field-extra-annotation: "@Id \n @GeneratedValue(generator = \"\
UUID\")\n"
ownerId:
{
type: string,
description: UUID of owner of the object in the system,
example: 41c047b3-b060-4313-817c-95d15c83a8e1,
format: uuid,
x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.CREATED_BY, enabled = true)",
}
createdDate:
{
type: string,
description: Date of object creation,
example: "2024-12-10T14:55:52.718-0800",
format: date-time,
x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.CREATED_DATE, enabled = true)",
}
keyHash:
{ type: string, description: "Data, including hash of the key(s) used
to encrypt this record." }
lastAccessedById:
{
type: string,
description: Last user to access object,
example: 4ed218b1-a423-4c87-9e9d-4387e2fcfdc5,
format: uuid,
x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.LAST_ACCESSED_BY, enabled = true)",
}
lastAccessedDate: { type: string, description: Timestamp of last access of
object, example: "2024-12-10T14:55:52.718-0800", format: date-time, x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.LAST_ACCESSED_DATE, enabled = true)" }
lastModifiedById:
{ type: string, description: Unique identifier for user who
last modifed the object in the system, example: 5ec21e3b-13f7-4b07-9c57-8a977a93d2c1, format: uuid, x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.LAST_MODIFIED_BY, enabled = true)" }
lastModifiedDate:
type: string
description: Date of last object modification
example: 2024-12-10T14:55:52.719-0800
format: date-time
x-field-extra-annotation: "@JsonIgnore \n @AuditingField(fieldType\
\ = AuditingField.FieldType.LAST_MODIFIED_DATE, enabled = true)"
ValorIDE
ValorIDE is the ValkryAI powered Coding Agent for VSCode.
Using ValorIDE makes the most of ValkyrAI CodeGen productivity gains for developers.
Inline CodeGen and AI-powered Coding unlocks incredible productivity.
MCP Services
In the 6 months since this project began there have been accelerating changes across the space.
One great development has been "Model Context Protocol" or MCP.
MCP is going to be a big part of ValkyrAI and how it functions and the services it provides both as a consumer and publisher.
Here are some of our Roadmap Items:
- MCP -> ThorAPI CodeGen as an MCP service
- MCP <- utilize MCP as ValorIDE tooling
- MCP <-> Utilize MCP as ValkyrAI tooling
- MCP Publishing Protocol (Over RSS)
- MCP -> Component Library Generator
- MCP -> Preferences Stack
- MCP -> Gridheim Sheetster: Data, Formulas, Import/Export
- MCP -> Asset generation (calls various inference stack)
- MCP -> configure the inference stack dynamically (used by Agents)
- MCP -> configure the inference stack dynamically (used by Agents)
GLOSSARY
Spring Query Methods: https://docs.spring.io/spring-data/jpa/reference/jpa/query-methods.html
Mustache Lambdas:
lowercase {{#lambda.lowercase}}{{name}}{{/lambda.lowercase}}
uppercase {{#lambda.uppercase}}{{name}}{{/lambda.uppercase}}
snakecase {{#lambda.snakecase}}{{name}}{{/lambda.snakecase}}
pascalcase {{#lambda.pascalcase}}{{name}}{{/lambda.pascalcase}}
camelcase {{#lambda.camelcase}}{{name}}{{/lambda.camelcase}}
kebabcase {{#lambda.kebabcase}}{{name}}{{/lambda.kebabcase}}
RESULT: lowercase addresstype uppercase ADDRESSTYPE snakecase address_type pascalcase AddressType camelcase addressType uncamelize Address Type kebabcase address-type
Expand your knowledge and get additional support by exploring the following resources:
-
ThorAPI GitHub Repository: Access the source code, documentation, and community support for ThorAPI.
-
JavaSpring CodeGen Templates: The original source code for the default JavaSpring CodeGen templates used by ThorAPI.
-
Typescript RTK Query: Our current REDUX implementation uses RTK Query. This system provides convenient Service classes that allow for storing and fetching model Objects from the API
-
Spring Boot Documentation: Learn more about building applications with Spring Boot.
-
OpenAPI Specification: Understand the OpenAPI standard for defining RESTful APIs.
-
Liquibase Documentation: Explore Liquibase for database change management.
-
AspectJ Documentation: Delve into AspectJ for aspect-oriented programming in Java.
-
AspectJ and Spring Reference: Using AspectJ with Spring Quick Reference with Examples
-
AspectJ Cheatsheet: AspectJ Quick Reference
-
SpringDoc OpenAPI Documentation: Discover how to integrate OpenAPI with Spring Boot applications.
-
Hibernate ORM Documentation: Learn about object-relational mapping with Hibernate.
-
Maven Documentation: Get guidance on using Maven for project management and build automation.
Contributing
We welcome contributions from our user community!
If you are a source code available customer and are interested in improving ValkyrAI, please follow these steps:
-
Fork the Repository: Create a personal fork of the ValkyrAI repository on GitHub.
-
Create a Feature Branch: Develop your feature or fix in a dedicated branch.
-
Commit Your Changes: Make clear and descriptive commit messages.
-
Open a Pull Request: Submit your changes for review, providing detailed information about the changes you've made.
-
Participate in the Review Process: Engage with maintainers and address any feedback or requested changes.
For more detailed guidelines, please read our Contribution Guidelines.
Code of Conduct
Please note that all contributors are expected to adhere to our Code of Conduct. We are committed to fostering a welcoming and respectful community.
License
Valkyr Labs products are licensed under a combination of open source licensing for core components such as SecureField in order that customers and the community may benefit from transparency and utility of our software, as well as contribute to its ongoing development and quality.
The nature of ThorAPI is open source friendly as a security-minded CodeGen solution we feel the increased security will benefit all of society and the spreading far and wide of SecureField protection and ThorAPI's beautiful consistent APIs will benefit customers, users, and organizations across the world.
For purposes of transparency and support as well as allowing users to customize their Valkyr stacks, we provide source code licensed under the Valkyr Labs Inc (VLI) Source License VLI Source License. .
ThorAPI is available under the MIT License.
By contributing to Valkyr Labs Inc repositories, you agree that your contributions will be licensed under the same license.
Disclaimer
WARNING!
AUTOMATED CODING MUST BE REVIEWED AND APPROVED BY HUMANS
NO NOT ALLOW AGENTIC CODE TO GO INTO PRODUCTION ENVIRONMENTS WITHOUT THOROUGH TESTING
ValkyrAI, Valor, ValorIDE, and ThorAPI are powerful tools intended to accelerate development and enhance security.
However, AI makes mistakes, it makes incorrect assumptions, and it can do bad things and do them at scale.
It is crucial to review and test all generated code thoroughly to ensure it meets your application's requirements and complies with all relevant regulations and best practices.
Spring, Spring Boot and Spring Cloud are trademarks of Pivotal Software, Inc. in the U.S. and other countries.
OpenAI Credentials & Fallback
This project supports calling OpenAI-compatible chat completions from two paths:
- Controller path:
valkyraiREST endpoint (LLMController) - Workflow path: adapters used by workflow modules (LlmAdapter/OpenAIAdapter)
To prevent accidental subsidizing of hosted users, environment-key fallback is disabled by default. Each user/workflow should provide its own API key unless you explicitly enable fallback for a controlled environment.
Behavior
- Controller (
LLMController): Uses theapiKeyon theLlmDetailsrecord. If missing and fallback is disabled, it returns a clear error. If fallback is enabled, it usesOPENAI_API_KEYfrom the environment/system property when available. - Workflow adapters (
LlmAdapterFactory→OpenAIAdapter): If no key is provided and fallback is disabled, it throws anIllegalStateExceptionwith guidance. If fallback is enabled, it usesOPENAI_API_KEYwhen present.
Properties
valkyrai.openai.allowEnvFallback(default:false)false: No fallback. A missing key fails fast with clear messaging.true: Fallback toOPENAI_API_KEY(and system property) is allowed.
Environment Variables
OPENAI_API_KEY: API key to use when fallback is enabled.OPENAI_ORG_ID(optional): SetsOpenAI-Organizationheader.OPENAI_BASE_URL(optional): Overrides base URL used by the adapter; default ishttps://api.openai.com/v1.
Examples
application.yml (recommended: enable fallback only for dev)
spring:
profiles:
active: dev
---
spring:
config:
activate:
on-profile: dev
valkyrai:
openai:
allowEnvFallback: true
---
spring:
config:
activate:
on-profile: prod
valkyrai:
openai:
allowEnvFallback: false
Shell (dev only)
export OPENAI_API_KEY="sk-..."
# optional
export OPENAI_ORG_ID="org_..."
export OPENAI_BASE_URL="https://api.openai.com/v1"
Error Messages
-
Controller path (fallback disabled):
OpenAI provider is set but no API key provided for this service. Environment fallback is disabled. Set apiKey on LlmDetails or enable 'valkyrai.openai.allowEnvFallback=true' and configure OPENAI_API_KEY.
-
Workflow adapter path (fallback disabled):
OpenAI API key not provided. Set apiKey in the workflow configuration or enable 'valkyrai.openai.allowEnvFallback=true' and configure OPENAI_API_KEY.
Notes
- Keep fallback disabled in production-hosted environments to avoid subsidizing user workloads.
- When enabled, fallback uses environment variable first, then system property.
- The adapter logs error bodies from OpenAI on non-2xx responses to aid diagnosis (e.g., 401 auth issues).
ValkyrAI UI — JSON‑Style Knowledge Base (Chunked for LLM Support)
Purpose: A compact, normalized map of the ValkyrAI UI so they can answer basic customer support questions about navigation, features, and common tasks.
⸻
0. Metadata
{
"app": "ValkyrAI",
"dashboard_url": "http://localhost:5173/dashboard",
"capture_iso8601": "2025-09-17T03:58:00.019Z",
"title": "Valkyr Labs Inc | ValorIDE, ThorAPI, ValkyrAI, Gridheim",
"content_len_estimate": 4000,
"environments": ["Development"],
"services": {
"spring_boot": {"host": "http://localhost:8080", "actuator": true},
"valkyrai": {"host": "http://localhost:8081", "actuator": true}
}
}
⸻
1. Global Navigation Map
{
"nav_tabs": [
"Dashboard",
"Workflow Builder",
"Visualizer3D",
"SuperCRM",
"APIs",
"DataOps",
"Charts",
"Servers",
"Users",
"Universal Search",
"ROI Calculator",
"Content (Valhalla CMS)"
],
"primary_roles": ["Engineer", "Ops", "PM", "Founder/Exec", "Content/Marketing"],
"layout": {
"left_nav": true,
"tabbed_editor": true,
"ops_center_widgets": true
}
}
⸻
2. Ops Center (System Monitoring)
{
"path": "Dashboard → Ops Center",
"widgets": {
"health": ["UP", "Liveness", "Readiness", "Version"],
"uptime_s": 15446,
"cpu_system_pct": 12,
"cpu_process_pct": 0,
"threads_live": 71,
"heap": {"used_mb": 552.0, "max_mb": 13552.0},
"uptime_24h": "100%"
},
"metrics_source": "/actuator/workflowMetrics",
"alerts": [
{ "metric": "system.cpu.usage", "op": ">", "threshold": 0.9, "name": "CPU_HIGH" }
],
"common_actions": [
"View service health",
"Create or remove alert rules",
"Open recent errors",
"Verify integration token for services"
],
"error_banner_examples": [
"HTTP 403: No objects selected for monitoring."
]
}
⸻
3. Workflow Builder
{
"path": "Workflow Builder",
"capabilities": [
"Create/edit workflows",
"Configure monitored objects",
"View success/failure counts",
"Link steps to APIs, Agents, Data sources"
],
"panes": ["Designer", "YAML/JSON", "Logs"],
"related": ["Visualizer3D", "DataOps", "APIs", "Agents"],
"faq": [
{"q": "Where do I see workflow errors?", "a": "Open Workflow Builder → Logs or Dashboard → Ops Center → Recent Errors."},
{"q": "How do I monitor a workflow?", "a": "In Workflow Builder, select the workflow and enable monitoring in settings; ensure objects are selected to avoid 403 warnings."}
]
}
⸻
4. 3D Workflow Visualizer
{
"path": "Visualizer3D",
"purpose": "Interactive 3D view of nodes and edges for running workflows",
"interactions": ["Pan/zoom", "Select node", "Highlight path", "Open node properties"],
"links": ["Open in Workflow Builder", "Open logs"]
}
⸻
5. Valhalla CMS (Content)
{
"path": "Content",
"states": ["All", "Published", "Scheduled"],
"counters": {"total": 19, "published": 0, "scheduled": 0, "views": 0},
"fields": ["Title", "Author", "Date", "Format (markdown/plaintext)", "Status"],
"editor_modes": ["Editor", "Scheduler", "Analytics"],
"examples": [
"Valor IDE Blew My Mind Last Week …",
"Mono Mythology Defeated",
"Here’s Some Unsolicited API Advice",
"Valkyr Pitch Deck Q4 2025",
"Weekly Update",
"Guidebook of the Realm of Valkyr"
],
"faq": [
{"q": "How do I publish a post?", "a": "Go to Content → Editor, compose or load a draft, set Status=Published, then Save."},
{"q": "Where are scheduled posts?", "a": "Content → Scheduler tab; set publish date/time and Save."}
]
}