Skip to main content
░▒▓█▓▒░░▒▓█▓▒░░▒▓██████▓▒░░▒▓█▓▒░      ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓███████▓▒░ ░▒▓██████▓▒░░▒▓█▓▒░
░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▒▒▓█▓▒░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▒▒▓█▓▒░░▒▓████████▓▒░▒▓█▓▒░ ░▒▓███████▓▒░ ░▒▓██████▓▒░░▒▓███████▓▒░░▒▓████████▓▒░▒▓█▓▒░
░▒▓█▓▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓██▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓████████▓▒░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░

v1.0.1-SNAPSHOT :: https://valkyrlabs.com/vai :: San Francisco, California :: ©2025 Valkyr Labs Inc

API-Native Agentic Services Engine

Java CI with Maven

Stable releases in Maven Central

open tasks

🔥 N8N Killer Workflow Designer Initiative

Building the most intuitive, powerful workflow automation engine!

Key Features:

  • ✨ Unified ExecModule configuration editor (all 20+ module types)
  • 🔗 Module chaining visualization (snap together like LEGO)
  • 📊 REST API lookups & auto-complete
  • 💾 Rock-solid create/edit/execute workflow flows
  • 🚀 Production-ready, enterprise-grade UX

Welcome to Valhalla Suite - the complete AI-native development platform that revolutionizes how you build secure, scalable applications. Powered by ValkyrAI and built on the robust capabilities of ThorAPI, this suite accelerates development by transforming OpenAPI specifications into fully-functional backend and frontend codebases with enterprise-grade security and AI-powered automation.


Generate early, generate often.

-- John McMahon,

Founder & CEO, Valkyr Labs Inc

Overview of Valhalla Suite

Valhalla Suite is the complete AI-native development ecosystem consisting of:

  • ValkyrAI - Core AI services engine with workflow automation
  • ThorAPI - Enterprise code generator for secure APIs
  • ValorIDE - AI-powered development agent for VSCode
  • GridHeim - Advanced spreadsheet engine with OpenXLS backend
  • Web Platform - Modern React/TypeScript application suite

With Valhalla Suite, developers can rapidly generate Java Spring Boot services, TypeScript clients, React components, and complete applications - all while maintaining enterprise-grade security and AI-powered automation.

SWARM integration

ValkyrAI includes a lightweight SWARM (agentic) coordination layer used by ValorIDE for remote-control and presence features. See README_SWARM.md for design notes, message formats, and quick-start instructions for the SWARM registry and mothership websocket integration.

What's New in v1.0.1

🚀 Latest Enhancements

  • Advanced Workflow Engine - Production-quality workflow automation with 20+ ExecModules, real-time monitoring, and visual designer
  • GridHeim Spreadsheet Engine - Enterprise-grade spreadsheet functionality with OpenXLS backend integration
  • Enhanced Spring Security - Updated to Spring Boot 3.4.6 with Spring Security 6.4.5 for advanced RBAC and ACL
  • Modern Dependency Stack - Latest OpenAPI tools (7.10.0), AspectJ (1.9.23), and AWS SDK (2.29.39)
  • ValorIDE Integration - Deep integration with VSCode AI development agent
  • Component Library Expansion - Production-ready React components with comprehensive testing

🎯 Workflow Engine Highlights

  • Visual workflow designer with React Flow
  • 20+ built-in ExecModules (Email, REST, Files, Social, Payments)
  • Query By Example (QBE) data selection
  • Async branching with eventual consistency
  • Real-time WebSocket monitoring
  • RBAC-aware permissions and sharing

📊 GridHeim Features

  • Excel-like cell selection and navigation
  • Professional clipboard operations (cut/copy/paste)
  • Advanced formula editing with OpenXLS backend
  • Real-time synchronization with ThorAPI
  • Enterprise-grade performance optimization

Features

Core Features

  • Java Data Model Generation: Create comprehensive Java data models annotated for Spring Boot and JPA, facilitating seamless integration with your backend services. Say goodbye to tedious manual coding!

  • TypeScript Client Generation: Produce fully-typed TypeScript object models and REST clients, empowering your frontend applications with reliable and consistent API interactions. No more guesswork—just type safety!

  • Spring Boot Service Generation: Generate robust Spring Boot services complete with CRUD operations and JPA integration, accelerating your backend development. It's like having a personal assistant for your backend!

  • Database Connectivity: Easily connect to various relational databases via JPA, with support for multiple database platforms. Your data is just a connection away!

  • REST API Documentation: Integrate with SpringDoc OpenAPI to automatically generate interactive API documentation. Because who doesn't love well-documented APIs?

Advanced Features

  • Encrypted Fields via ThorAPI SecureField: Secure sensitive data with field-level encryption, protecting information both at rest and in transit. Your secrets are safe with us!

  • Automatic Relationship Serialization: Seamlessly handle one-to-many and many-to-many relationships, simplifying data modeling and serialization. Relationships made easy!

  • Secure React/TypeScript Component Libraries: Generate secure, data-bound React components to build robust frontend interfaces with ease. Frontend security has never been this simple!

  • Professional Documentation Site: Generates all your API and model object documentation inline to a professional and dynamic documentation site powered by Docusaurus. Because your documentation deserves to shine!

Connectors (ExecModules)

ValkyrAI includes a growing set of pluggable ExecModules (connectors) that you can attach to workflow Tasks. Starter set:

  • Email: Mailtrap Blast (icon: react-icons FaEnvelope)

    • Class: com.valkyrlabs.workflow.modules.MailtrapSendModule
    • Batches Mailtrap sends; supports concurrency, retries, dry-run; emits sendStats and failures.
  • Email: Basic Sender (icon: FaEnvelope)

    • Class: com.valkyrlabs.workflow.modules.EmailModule
    • Minimal sender for demos; prefer the Mailtrap Blast module for production.
  • Payments: Stripe Checkout (icon: SiStripe)

    • Class: com.valkyrlabs.workflow.modules.payments.StripeCheckoutModule
    • Builds Stripe Checkout sessions from SalesOrder + LineItems; returns session_id and checkout_url.

Tip: In Workflow Studio, icons are auto-picked from react-icons based on class name (e.g., “stripe” → SiStripe, “email” → FaEnvelope).

See docs under web/typescript/valkyr_labs_com/docs/docs/Products/ValkyrAI/Workflow Engine/Execution-Modules/.

Query By Example (QBE)

ValkyrAI supports Query By Example across generated model endpoints and UI tables.

  • Backend: Java/Spring controllers accept an optional example query parameter on GET /<Model>.

    • The value is a URL-encoded JSON object representing a partial instance of the model.
    • Matching uses Spring Data ExampleMatcher:
      • Strings: case-insensitive CONTAINING (substring)
      • Non-strings: exact match
    • Example: /Address?example=%7B%22city%22%3A%22fran%22%2C%22state%22%3A%22CA%22%7D
  • Frontend: generated Redux Query services accept example optionally:

    • useLazyGet<Model>sPagedQuery({ page, limit?, example? })
    • useGet<Model>sQuery({ example? })
  • UI: each model table has a “Filter (QBE)” modal in the floating toolbar for pasting JSON; infinite scroll and keyboard paging preserve the filter.

Performance tips

  • Index frequently filtered fields, and keep result windows small with paged fetches.
  • Prefer exact matches for identifiers and narrow text fields to reduce scan cost.

Built on ThorAPI: The Backbone of ValkyrAI

ThorAPI is an opinionated code generator designed for creating secure, performant, and efficient APIs. As a Java Maven project, ThorAPI transforms your OpenAPI specifications into:

  • Spring Boot REST APIs: Accelerate backend development with automatically generated RESTful services.

  • TypeScript Client APIs: Streamline frontend development with generated TypeScript clients that ensure type safety and consistency.

ThorAPI harnesses industry-standard tools and frameworks, including Maven, OpenAPI Generator, Handlebars templating, Spring, JPA, REST APIs, TypeScript, and Java. It provides out-of-the-box persistence via a JPA service, enabling rapid development of full-stack applications.

ThorAPI bundle pipeline

  • Specs are now composed through declarative bundle manifests (for example rbac-core, ecommerce, invoicing, customer-support).
  • Supply thorapiBundles in the generate request configuration JSON to opt into additional bundles. Values are comma or semicolon separated (e.g. { "thorapiBundles": "ecommerce,customer-support" }).
  • Bundles may ship OpenAPI fragments, UI components, workflow metadata, and static assets; everything is staged with the generated code and recorded in thorapi-bundle-manifest.json inside the download.
  • The baseline RBAC/user-preferences bundle is applied automatically unless you pass thorapiBundles: "none".

Roles, Authorities, and Security Context

  • Spring Security makes access decisions against granted authorities. Every role we ship is expressed as a GrantedAuthority string (ROLE_ADMIN, ROLE_SYSTEM, etc.), so the authorityList on a Principal is the canonical source Spring reads during authorization.
  • The optional roleList that ThorAPI generates provides richer metadata (enums, descriptions, audit fields). Keep it in sync with authorityList when you introduce new roles so each role still manifests as a ROLE_* authority.
  • You can represent sharing groups or fine-grained permissions as additional authorities (for example workflow:execute, group:family-photos) without creating companion role records. Spring will treat them the same as traditional roles.

Security context helpers

  • Background schedulers and integration threads should not manually call SecurityContextHolder.clearContext(). Instead, wrap work in systemUserService.runAsSystem(() -> { ... }) or runAsAssistant(...) (defined in valkyrai/src/main/java/com/valkyrlabs/valkyrai/service/SystemUserService.java). The helper installs a temporary ThorUser context, executes the task, and restores the previous authentication automatically.
  • For dynamic role switching inside workflow modules, use WorkflowSecurityContext.runAsRole("system", () -> { ... }) which delegates to the same helpers.

ROADMAP: Liquibase support


Getting Started

Development with ValkyrAI is simple once you get the hang of the CodeGen development practices.

We recommend you review the development best practices before getting too far into development.
The ValkyrAI Development Best Practices Manual

System Requirements

If you need to install this software before you begin, ensure you have at least the following minimum installed:

ValkyrAI for development you will need:

  • Linux/Windows/MacOS with Java 17+ @OpenJDK17+ installed
    • Important: Java 17+ requires the JVM argument --add-opens=java.base/java.lang=ALL-UNNAMED for CGLIB proxy support. This is already configured in the startup scripts. See JAVA_MODULE_CGLIB_FIX.md for details.
  • 2GB available RAM
  • working node and npm (for TypeScript client development): Download Node.js
  • working git installed / github account and credentials
  • working maven installed
  • recommended: docker desktop

Deploying generated ValkyrAI apps requires a minimum:

  • Linux/Windows/MacOS server with Java 17+ @OpenJDK17+ installed
  • 2GB available RAM

Install Java Development Kit (JDK)

ValkyrAI requires Java 17+ and is fully tested with the latest versions. We recommend:

Download JDK v17+ From Oracle

On macOS you would typically use brew:

# Current recommended version for ValkyrAI
brew install openjdk@17
# or for latest
brew install openjdk@23

Once you have java installed, you can grab maven and be up and running.

NOTE: there is a Maven build option to install and run the node generated stack -- and to automate the building of Typescript client library. However in practice this is a very slow step and hinders iteration time while developing the backend services that actually require a build step.

NOTE: There are definite best practices to adopt in terms of project setup, development methods, and how to manage front-end vs. backend development.

The ValkyrAI Development Best Practices Manual

Generation Steps

Follow these steps to generate your API and client code:

  1. Customize Your OpenAPI Specification

    • Edit the 2 OpenAPI spec template files: src/main/resources/openapi/api.yaml and src/main/resources/openapi/api.hbs.yaml
    • Define which endpoints are exposed in api.yaml
    • Create your detailed Model Schema in api.hbs.yaml
    • Include ValkyrAI-specific annotations such as x-thorapi-secure-field and x-thorapi-data-field.
    • ThorAPI will generate the CRUD operations for your schema model objects
  2. Generate Code

    • Spring Boot Service:
      • Generate RESTful CRUD APIs with integrated JPA backend.
    • TypeScript Client:
      • Generate a fetch-based API client.
      • Include end-to-end validation with Jest testing.
  3. Test the Service Integrations

    • Your generated code may be referenced in other projects, or in the monorepo, update and test all affected code and systems

    • Your APIs may be called from remote systems. You will need to ensure that your versioning reflects breaking changes to your spec, and that you retain backwards compatibility and migration information for all API consumers.

    • Ideally, you maintain best practices around communicating changes to your API consumers.

  4. Run the Service

    • Deploy your service as Spring microservices.
    • Connect to your preferred database and configure settings as needed.

Versioning Your API

APIs, SDKs, and code libraries require versioning information to allow for changes and updates to software that other systems depend upon without creating chaos or breaking production systems.

If you are using ValkyrAI or ThorAPI, you are now an API and library publisher, and you are going to have to be cognizant of this requirement and handle the publishing of your changes accordingly.

While versioning is outside of the scope of this README, we follow the traditional SEMVER best practices as much as possible.

ROADMAP: automatic SEMVER / changelog feed service

Non-Breaking Changes (increment minor versions) These are changes that ADD detail to your OpenAPI spec, such as putting a description on a property or enhancing the example for a property.

In general, these types of changes will not break or cause problems in existing applications as long as there are no unintended side effects to making the changes (such as memory usage ballooning causing OOME).

Breaking Changes

If you change the name of Objects or their properties, remove or change objects or properties, or truncate the size of or data precision of existing database columns, you will need to change the major version of your API to allow existing applications to continue using the older versions of your code.

When you increment the major version/update endpoint version, you now have 2 APIs to manage, which is why it is important to have migration plans for API consumers as well as detailed usage statistics to determine when it is safe to deprecate and remove old running API code.


Project Setup

Out of the box, ValkyrAI is a Java Maven Project Monorepo with some unique features:

  • A TypeScript + React web app under: /src/main/typescript/<projectname>
  • GENERATED TypeScript React + Redux/RTQ + Bootstrap + Formik web app client code under: /src/main/typescript/<projectname>/src/thor
  • GENERATED Spring Boot web app under: /generated/spring
  • A convenient command line runner ./vai

With this setup, ValkyrAI is immediately ready to produce a functional full-stack application with security, usability, and AI-powered functionality at its core.

The ValkyrAI Development Best Practices Manual

OpenAPI Spec Enhancement

In the first step of the generation process, ThorAPI enhances your spec with a variety of features:

  • CRUD Rest Endpoints for POST/PUT/GET/DELETE for each Schema Component (Model Object) in the api.yaml spec input file.

  • Detailed and enhanced Schema Components (Model Objects) in the api.hbs.yaml spec input file.

  • These 2 fully customizable spec templates are merged and enhanced with mandatory "metadata" fields for every object (lastModifiedById, keyHash, createDate, etc.).

Use ThorAPI to process and merge your OpenAPI templates.

Input files:

  • src/main/resources/openapi/api.yaml: Your base OpenAPI specification.
  • src/main/resources/openapi/api.hbs.yaml: The ThorAPI Handlebars template.

Command:

java -jar lib/generator-<valkyrai_version>-exec.jar \
src/main/resources/openapi/api.yaml \
src/main/resources/openapi/output-api.yaml \
src/main/resources/openapi/api

ThorAPI leverages existing tools like Maven and OpenAPITools CodeGen, Handlebars templating, Spring, JPA, REST APIs, TypeScript, and Java.

Enhance OpenAPI Spec

Run the ThorAPI to process the 2 OpenAPI templates.

The 2 templates are merged and enhanced with the special ThorAPI fields -- with the api.yaml determining which model objects will have REST endpoints generated.

> src/main/resources/openapi/api.yaml
> src/main/resources/openapi/api.hbs.yaml
/usr/bin/env java -jar lib/generator-<valkyrai_version>-exec.jar src/main/resources/openapi/api.yaml src/main/resources/openapi/output-api.yaml src/main/resources/openapi/api
[ERROR] /Users/johnmcmahon/workspace/2025/valkyr/ValkyrAI/generated/spring/src/main/java/com/valkyrlabs/api/McpMarketplaceItemRepository.java:[103,91] type com.valkyrlabs.model.McpMarketplaceItemTag does not take parameters
[ERROR] -> [Help 1]

Generating the Output Project Run Maven in the root of the project to clean and install the project and run the CodeGen process for both Java Spring and TypeScript:

mvn clean install

This build will generate the backend Spring Boot service code under the generated/spring/ folder.

The generated project is a Maven Spring Boot project that provides the generated API via an executable jar.

You can build the executable jar and (optionally) run it using Maven.

cd generated/spring
# run with defaults (h2 db)
--spring.datasource.url="$SPRING_DATASOURCE_URL" \
--spring.jpa.hibernate.ddl-auto=update \
--spring.datasource.username="$SPRING_DATASOURCE_USERNAME" \
--spring.datasource.password="$SPRING_DATASOURCE_PASSWORD" \
--spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver \
--server.port="$SERVER_PORT" \
--jwt.secret="$JWT_SECRET" \
|| error_exit "Failed to run backend instance"

THORAPI_SECRET_KEY is a MANDATORY environment or command line variable that is used to encrypt the SecureFields. Keep this private key secure.

A new THORAPI_SECRET_KEY can be generated with the generatekey command using the following command line:

java -jar lib/generator-<valkyrai_version>-exec.jar generatekey
export THORAPI_SECRET_KEY=generatedPrivateKey
// build the generated project
mvn clean install -f generated/spring/pom.xml

Advanced Topics:


Given N correct OpenAPI specs, you should be able to perform any operation between systems using only transform layers.

-- John McMahon, Founder of Valkyr Labs Inc

SecureField Notes

SecureField is a powerful encryption mechanism provided by ThorAPI that ensures sensitive data is protected both at rest and in memory. By intercepting and wrapping field access, SecureField offers the following benefits:

  • Data Encryption: Automatically encrypts data when storing it in the database, safeguarding against unauthorized access.

  • On-Demand Decryption: Decrypts data transparently when it is accessed within your Java code, providing seamless integration.

  • Enhanced Security: Adds an additional layer of protection against data breaches, including raw data dumps or memory inspection.

SecureField also supports password hashing and plays a crucial role in ThorAPI's user authentication and authorization functionality. It delivers a standards-based, secure implementation for permissions and Access Control Lists (ACLs), enabling robust security management within your applications.

SecureFields are encrypted with the THORAPI_SECRET_KEY which you provide as a command line or environment variable.

WARNING: YOUR DATA WILL BE PERMANENTLY LOST IF YOU LOSE OR OTHERWISE BUNGLE THE THORAPI_SECRET_KEY

Using Secure Field in Your Schema

To utilize SecureField in your OpenAPI schema, simply add the x-thorapi-secureField: true annotation to any field you wish to encrypt. For example:

components:
schemas:
User:
type: object
properties:
id:
type: string
format: uuid
username:
type: string
email:
type: string
format: email
x-thorapi-secureField: true

This default SecureField is encrypted with modern and reasonably secure symmetric key algorithm, able to be unencrypted by anyone with the THORAPI_SECRET_KEY.

For more advanced cases you can directly apply the @SecureField annotation with it's straightforward configuration.

These less common use cases include if you need a higher strength symmetric encryption (more entropy and iterations added to the hash function) -- or if you need to create a password field using a HASHing algorithm that cannot be decrypted but can only be used for matching.

Both hashing (password style) and symmetric encryption are supported by @SecureField.

---
password:
minLength: 8
type: string
description: Your account password
format: password
example: HardToGuess1980
x-field-extra-annotation: "@SecureField(encryptionType = SecureField.EncryptionType.HASHED, strength = 10)"

ValkyrAI builds upon ThorAPI's capabilities by integrating comprehensive user management, ACL management, key rotation, and upcoming OAuth services, providing a holistic solution for secure API development.

ValkyrAI is a partially managed solution that combines ThorAPI CodeGen with comprehensive User management, ACL (access control lists) management, Key rotation and OAUTH services are coming soon to the ValkyrAI service.

Technical Notes on SecureField

***Column Size for Encrypted Data

One factor in the use of SecureField feature is that an encrypted value is much larger (often 2x or more) the size of the same plaintext value.

ThorAPI generates column sizes to accommodate these encrypted values as well as validations on these columns that accommodate the size of the plaintext.

This results in validations on field size that are sized differently to accommodate the larger size of encrypted data.


Auto-Generated Fields

Do not include the following auto-generated fields in your OpenAPI specification components:

  • id
  • modified_date
  • created_date
  • owner_id
  • key_hash
  • last_modified_by_id
  • last_modified_date
  • last_accessed_by_id
  • last_accessed_date

These fields are automatically managed by ValkyrAI and ThorAPI. Including them in your spec may lead to conflicts or unexpected behavior.

NOTE: in terms of privacy commitments, you will need to communicate to end users of your systems that -- short of heavily reducing the functionality of the system -- this data will be automatically collected. In some use cases your generated applications may run up against regulatory or other conflicts with the collection of precise 'tracking' data.

NOTE: it is COMPLETELY YOUR RESPONSIBILITY to ensure that your data collection practices via any ThorAPI generated code are legal and in conformance with your community standards.

Privacy Benefits of ThorAPI: concerns about surveillance and collection of too much activity auditing by encrypting PII data we can provide much more reliable security, but without the additional metadata there is a loss of traceability and much of the 'fingerprint' or 'footsteps' of any unauthorized use of the system, including data breaches. By maintaining high level of visibility into the data, we can more easily "wipe" data and comply with data and privacy standards.

Determining Service/Delegate API Output Files

The api.yaml file determines which Model Objects in your API spec will have generated endpoints.

NOTE: We refer to the collective exposed API endpoints as the "surface area" of your API -- visualizing it this way lends itself to a security posture. The "attack surface" of your APIs will be determined by the exposed endpoints.

If all you have in your API folder for a generated object is the repository, it is likely that you did not list the object in the api.yaml input file.

If there is a definition in the api.hbs.yaml file, then ThorAPI will generate the object model, but to make the objects available to the "surface area" of your exposed API, you will need to include the item explicitly.

api.yaml

---
## My Game - SPICES HAVE NO PUBLIC API
GamePlayer:
GameWorld:

In your data model, you will need the GameSpice object so you define it alongside the exposed endpoint model objects:

api.hbs.yaml

---
## My Game
GamePlayer:
type: object
description: game player object
properties:
name:
type: string

GameWorld:
type: object
description: game world object
properties:
name:
type: string
players:
type: array
items:
$ref: "#/components/schemas/GamePlayer"
spices:
type: array
items:
$ref: "#/components/schemas/GameSpice"

GameSpice:
type: object
description: game hidden spice object
properties:
spiceName:
type: string
spices:
type: array
items:
$ref: "#/components/schemas/GameSpice"

GameSpice:
type: object
description: game hidden spice object
properties:
spiceName:
type: string
SQL Error: 3854, SQLState: HY000
2024-11-10T11:58:15.975-08:00 ERROR 51718 --- [nio-8081-exec-2] o.h.engine.jdbc.spi.SqlExceptionHelper : Cannot convert string '\x05\x86k\x93\x0A\xD8...' from binary to utf8mb4
2024-11-10T11:58:15.989-08:00 WARN 51718 --- [nio-8081-exec-2] .w.s.m.s.DefaultHandlerExceptionResolver : Resolved [org.springframework.http.converter.HttpMessageNotWritableException: Could not write JSON: JDBC exception executing SQL [select pr1_0.formula_id,pr1_0.id,pr1_0.created_date,pr1_0.formula_id,pr1_0.key_hash,pr1_0.last_accessed_by_id,pr1_0.last_accessed_date,pr1_0.last_modified_by_id,pr1_0.last_modified_date,pr1_0.owner_id,pr1_0.reference,pr1_0.sheet_id,pr1_0.type from ptg_ref pr1_0 where pr1_0.formula_id=?] [Cannot convert string '\x05\x86k\x93\x0A\xD8...' from binary to utf8mb4] [n/a]]

Spring Security ACL

One of the most powerful ValkyrAI features is RBAC (Role Based Access Control) and Object-level (ie: row-level) security, allowing for secure sharing of data throughout and the ability to leverage permissions to rapidly build secure apis and UX experiences.

The implementation of Spring Security ACL is fairly standard and a deep dive into this technology is outside of the scope of the document.

For a quick overview of the Spring Security ACL implementation please read the following:

https://docs.spring.io/spring-security/site/docs/5.0.x/reference/html/domain-acls.html#domain-acls-key-concepts

The basic concept to understand is that ValkyrAI uses ThorAPI to generate the relevant ACL database tables and objects (acl_xxx tables). So you will see these tables in the base OpenAPI spec.

For obvious reasons you will want to secure any generated endpoints.

ValkyrAI Permissions

For not-logged in, anonymousUser:

  • hasPermission() returns false by default

  • hasPermission(ObjectIdentityInstance, 'read') returns true if there is an anonymousUser:read:ObjectIdentityInstance AclEntry

  • As a safeguard ENCRYPTED FIELDS NEVER DECRYPT FOR ANONYMOUS (otherwise why encrypt?) -- trying to give anonymous the VIEW_DECRYPTED permission on anything should fail and output a log warning to console as it's a SCHEMA issue.

  • APPEND permissions for some objects are sometimes necessary, anonymousUser needs to post to the Rating endpoint to submit ratings for various entities, but without logging in or having a Principal in the system.

We use table-scope support in the ACL service so blank/“TYPE” IDs map to a deterministic namespace UUID and we can store APPEND at the class level.

The AuthController logic normalizes ACL grant/revoke/deny requests and falls back to the type-scope identifier, making /v1/auth/acl/* work when the caller omits the object ID for APPEND.

POST security is aligned with the ACL model: the Rating endpoint checks hasPermission(null, 'com.valkyrlabs.model.Rating', 'APPEND') and Spring Security allows anonymous posts through once that ACE exist

The runtime evaluator now recognizes missing IDs as table-level APPEND checks and passes the synthesized object identity through to ACL reads; the companion test exercises the anonymous append path

Tests:

mvn -pl valkyrai test -Dtest=com.valkyrlabs.valkyrai.controller.AuthControllerPermissionMappingTest -Dsurefire.failIfNoSpecifiedTests=false
mvn -pl thorapi test -Dtest=ValkyrACLTest,ValkyrAIPermissionEvaluatorTest
Next steps:

Grant anonymousUser the table-level APPEND ACE (POST /v1/auth/acl/grant with objectType = com.valkyrlabs.model.Rating, empty objectId, permission = APPEND).

Confirm the POST guard by calling /v1/Rating anonymously once that ACE is in place.

USE OBJECT-LEVEL PERMISSIONS WITH CAUTION

For logged in, non ADMIN user:

  • hasPermission() returns false by default

  • hasPermission() returns true if there is an &lt;authenticatedPrincipal.id&gt;:&lt;permission&gt;:ObjectIdentityInstance AclEntry

  • ENCRYPTED FIELDS DECRYPT if there is a VIEW_DECRYPTED permission .

  • SYSTEM user and group falls under this category, so system services agents and controllers can be given permission to decrypt sensitive data as needed but not by default and never as ADMIN

For logged in, ADMIN user:

  • hasPermission() returns true by default FULL STOP

  • ADMINs can do ALMOST ANYTHING in the system so always return true for any permissions request and always decrypt.

  • For performance, isAdmin and isOwner checks happen before AclEntry lookups... fast return first, isAdmin is a quick ROLE check on the authoenticatedPrincipal so no data lookup needed. isOwner is a fast lookup on the object which may also be loaded in memory already do isOwner checks next. Then we do the Acl lookups.

  • ADMINS cannot delete their own Principal record.

  • ADMINS cannot delete their own ROLE record.

ALL OTHER ACCESS IS per the restrictions in SecurityConfig.java

Customizing Spring Security

The default Spring Security configuration is located in: /src/main/java/com/valkyrlabs/config/SecurityConfig.java

This is a standard Spring Security configuration class. You can customize the security rules to fit your application's requirements.

You can create custom Roles in the database using the generated REST api.

The default roles created are:

...
[ROLE_ANONYMOUS]
[ROLE_EVERYONE]
[ROLE_ADMIN]
[ROLE_USER]
...

You can use the auth system to authorize methods in your custom Java code:


/**
* This PreAuthorize checks the * DataObject (Object must have * a "getOwner()" method)
*/
@PreAuthorize("isOwnerById(#objectId, #objectType)")
public void mySecureMethod(Long objectId, String objectType) {
// Method implementation
}
...

The DataObject Interface (Java)

This interface is implemented by all of the generated model objects by default.

This is the interface that also defines the "decorated" data fields which are added to the OpenAPI spec during the enhancement step.

Because we have a reliable implementation throughout the model graph, we are able to use the DataObject interface to handle any object in our system generically, yet with the benefit of built in Entity mapping and handling.

For example, if you wish to create a property on a model object representing a container of any other object in the system, you can directly reference the DataObject as the data type in the spec and thus be free to populate this container at runtime.

Customizing ThorAPI CodeGen Templates

Locally modifiable templates can be found in:

/src/main/resources/templates

The currently implemented templates are JavaSpring and Typescript-Redux-Query:

JavaSpring

Typescript-Redux-Query

You may implement any number of generators in your project, simply add an appropriate generator section to the pom.xml file in the root of the project.

pom.xml

            <plugin>
<groupId>org.openapitools</groupId>
<artifactId>openapi-generator-maven-plugin</artifactId>
<version>7.5.0</version>
<executions>
...
<execution>
<id>api.client</id>
<goals>
<goal>generate</goal>
</goals>
<configuration>

<generatorName>typescript-redux-query</generatorName>
...
<configOptions>
...
</configOptions>
</configuration>
</execution>
...

Deployment Section

Running the Generated ValkyrAI Service:

Option 1: Run with Custom Database Settings

Run the service by specifying your database configuration:

java -jar generated/spring/target/valkyrai-api-<valkyrai_version>.jar \
--spring.datasource.url=jdbc:mysql://your.database.com:3306/valkyrai \
--spring.datasource.username=dbmasteruser \
--spring.datasource.password=YOUR_DB_PASSWORD \
--spring.jpa.hibernate.ddl-auto=update \
--spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQLDialect \
--server.port=8081

Option 2: Run via Maven with Default H2 Database

If you prefer to use the default in-memory H2 database OR if you have set the environment variables properly you can simply run:

mvn spring-boot:run

Option 3: Run via Maven with Custom Database Parameters

To connect to a different database while running via Maven, pass the parameters as arguments:

mvn spring-boot:run -Dspring-boot.run.arguments="\
--spring.datasource.url=jdbc:mysql://your.database.com:3306/valkyrai \
--spring.datasource.username=dbmasteruser \
--spring.datasource.password=YOUR_DB_PASSWORD \
--spring.jpa.hibernate.ddl-auto=update \
--spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQLDialect \
--server.port=8081"

Environment Variables:

// SECURITY: Rate limiting configuration
@Value("${valkyrai.rate-limit-rpm:60}")
private static int MAX_REQUESTS_PER_MINUTE;
@Value("${valkyrai.rate-limit-max-fails:5}")
private static int MAX_FAILED_ATTEMPTS;
@Value("${valkyrai.rate-limit-lockout-minutes:10}")
private static long LOCKOUT_DURATION_MINUTES;

Reserved Words

Avoid using reserved words as identifiers in your OpenAPI specification to prevent naming conflicts and code generation issues.

DataBase Reserved Words

You will need to avoid using a variety of reserved words for your object schemas depending upon which database you are using.

For example, here we try to create a property named "schema" -- which fails when Spring JPA tries to create a MySQL column:

2024-11-08T12:30:53.547-08:00  WARN 99739 --- [           main] o.h.t.s.i.ExceptionHandlerLoggedImpl     : GenerationTarget encountered exception accepting command : Error executing DDL "alter table exec_module add column schema varchar(3000)" via JDBC [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'schema varchar(3000)' at line 1]

org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL "alter table exec_module add column schema varchar(3000)" via JDBC [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'schema varchar(3000)' at line

Reserved Words (Partial List):

  • OpenAPI/YAML: meta,class
  • URL Characters: %, ?, *, @, !, (, )
  • Java Keywords: String, Float, Integer, Object, Long, Array, Boolean
  • SQL Keywords: UNION, JOIN, SELECT, INSERT, UPDATE, DELETE
  • ThorAPI Objects: ApiUtil, MediaType, Mono, Content

Spring Reserved Words

Typescript-Redux-Query Reserved Words

Workarounds:

If you must use a reserved word, modify it to avoid conflicts. For example:

  • "Content""ContentData"
  • "Mono""MonoResource"

Note: Be mindful of capitalization. ThorAPI may alter the case formatting of identifiers, which can affect your code if specific capitalization is required.

OpenAPI Workarounds

Sometimes you can simply use the OpenAPI workarounds to meet your naming requirements.

Here is an example of using the "_" underscore to create an aliased property name and then ensuring JPA assigns it to the "class" column in the database binding.

      properties:
...
_class:
type: string
nullable: false
x-field-extra-annotation: "@Column(name = \"class\")"
required: [_class]

Naming Conventions and Best Practices

  • Avoid Acronyms with Unusual Capitalization: Using acronyms or unconventional capitalization can cause issues during code generation.

  • Apply the KISS Principle: Keep your specifications simple and straightforward to facilitate smooth code generation and maintenance.

  • Prevent Conflicting Methods: You want to avoid create conflicting GET, PUT, POST, or DELETE methods in your OpenAPI specification.

  • You want to avoid Manually Edit Generated Files: Manual changes to generated code may be overwritten or lead to stale builds. Instead, modify the CodeGen templates located in /src/main/resources/templates.

  • Separate Concerns: Use separate OpenAPI specifications to create distinct services for different areas of functionality.

  • Consistent Validation Rules: Ensure your validation rules are logical and consistent. For example, avoid setting a pattern requiring 8 characters while also specifying a minimum length of 11.

  • Simplify Example Strings: Use simple, clear example strings to avoid parsing errors during code generation.

Troubleshooting Section

Encryption Failures

SecureField comes in 2 flavors: ThorAPI built-in and the SecureField KMS which is included with ValkyrAI.

Out of the box, ThorAPI requires setting of the THORAPI_SECRET_KEY value as the only way to change the key. Custom solutions built on top of ThorAPI can of course manage the keys any way you desire.

ValkyrAI on the other hand has a built-in Key Management System, which allows for automated rotation of keys, and the ability to manage multiple keys at once with the same databases.

If there is a problem with the key management, or a key is corrupted or otherwise somehow made unusable, then the SecureField system will not be able to decrypt previously encrypted values and will return a standard error message such as:

{'error':'decryption failure: Tag mismatch'}

Since there are innumerable ways in which key data might be compromised or made unavailable, it is outside of the scope of this README to consider all possibilities.

For this reason it is important that any key created by you and used with SecureField encryption be managed carefully.

NOTE: As long as you have the valid key associated with a piece of encrypted data, you will always be able to decrypt it using the open source ThorAPI decryption utilities.

Typescript Client Libraries and Component

Typescript Output

The generated Typescript client libraries are generally placed under: src/main/typescript/<projectname>/src/thor

The generated code includes:

  • Fully typed models
  • Fully typed API clients
  • Redux/RTQ slices for each API
  • Formik forms for each model object
  • Bootstrap components for each model object
  • Jest tests for each API

Using the Generated Typescript Client

You can use the generated Typescript client code in your React/Typescript applications by importing the relevant modules.

Customizing RTK Data Handling

Typescript RTK Query: Our current REDUX implementation uses RTK Query. This system provides convenient Service classes that allow for storing and fetching model Objects from the API

The general pattern is to implement your own thin redux store and import the generated reducers to combine with your root reducer.

The RTK Query system is largely independent of your store for better or worse.

You can easily extend the existing RTK Query code, installing your own reducers, middlewares, and services, and mutations, all using the standard RTK Query patters.

Here is an example of using a custom set of mutation and query services:

// custom redux for ACL granting
import {
useGrantAclPermissionMutation,
useRevokeAclPermissionMutation,
useGetRolesQuery,
useGetObjectAclEntriesQuery,
} from "../../redux/services/AclService";
import CoolButton from "@valkyr/component-library/CoolButton";

Typescript Code Generation Issues

If you are using the generated Typescript clients then there should be no issues in utilizing the generated code assuming the ThorAPI backend Java code compiles and is runnable.

That said, if you are modifying any front-end templates there are numerous gotchas.

The following is due to React using the double bracket to define CSS properties, which is the same type of delimiter used by the mustache template engine. So proper CSS code can get mangled when the template engine replaces the section of CSS between the {{}} symbols at which point you get non-compiling react code:

src/main/typescript/valkyr_labs_com/src/thorapi/redux/components/form/WorkflowStateForm.tsx:282:18
280| className="nice-form-control"
281| style=
282| >
| ^

One approach is to use delimiter "swapping" in mustache and use a different delimiter but we find this approach to be needlessly complex.

The preferred and easier solution is to change the template CSS section by separating the brackets with spaces like so:

... // change
style={{ display: 'block', marginBottom: '1rem' }} >
... // to
style={ { display: 'block', marginBottom: '1rem' } } >
...

AspectJ Not Woven

Because ThorAPI and SecureField depend upon AspectJ weaving to perform their operations, an issue with AspectJ weaving is a critical failure and must not be ignored or bypassed.

We are actively watching the AspectJ space for enhancements as well as potentially releasing our custom, updated AspectJ maven plugin.

For reliability reasons both ThorAPI and ValkyrAI perform AspectOf() tests check on the SecureField aspect and if there is any problem with weaving, will fail with the following exception:

Caused by: org.aspectj.lang.NoAspectBoundException: Exception while initializing com.valkyrlabs.valkyrai.controller.SecureFieldAspect: java.lang.NoSuchMethodException: com.valkyrlabs.valkyrai.controller.SecureFieldAspect.aspectOf()
at org.aspectj.lang.Aspects.aspectOf(Aspects.java:50)
at com.valkyrlabs.valkyrai.config.SecureFieldConfig.secureFieldAspect(SecureFieldConfig.java:24)

You might also encounter this error when initializing the AuditingAspect ... the solution is the same, check for any java process that might keep a lock on the Java build files (this haapens with VSCode unfortunately as of 5/25)clean and rebuild using ./vai command or 'mvn clean install'.

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.valkyrlabs.valkyrai.controller.AuditingAspect]: Factory method 'auditingAspect' threw exception with message: Exception while initializing com.valkyrlabs.valkyrai.controller.AuditingAspect: java.lang.NoSuchMethodException: com.valkyrlabs.valkyrai.controller.AuditingAspect.aspectOf()
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.lambda$instantiate$0(SimpleInstantiationStrategy.java:199)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiateWithFactoryMethod(SimpleInstantiationStrategy.java:88)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:168)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
... 26 common frames omitted

This problem will most likely occur when bypassing Maven build of the projects such as when using an IDE with a "javac" debugging compiler.

ThorAPI classes recompiled without AspectJ support are likely to cause data problems such as overwriting encrypted data with cleartext and should not be made available on any classpath.

Compile-time vs. load-time weaving

  • valkyrai/pom.xml now binds the AspectJ plugin to the standard compile and test-compile phases. Always run mvn -pl valkyrai clean package (or at least mvn -pl valkyrai spring-boot:run) before bootstrapping the service so the woven bytecode gets produced.
  • The vai helper relies on those woven classes and, by default, launches the exec jar without the AspectJ agent. This avoids the Xlint:cantFindType liquibase.exception.ChangeLogParseException failure that occurs if the agent tries to weave Spring Boot’s Liquibase analyzers at runtime.
  • If you truly need load-time weaving again, export VALKYR_ENABLE_LTW=true before running ./vai. The script will then resolve aspectjweaver from your Maven cache and pass -javaagent to the JVM. Leave the variable unset for the normal (and recommended) compile-time weaving path.

Other Potential SecureField Issues

BUG ALERT: As of 12/24, SecureField is only partially supported with Validations. We are aware of the inconvenience but for now if you are getting issues inserting data into a securefield and you have a small-ish size validation, you can expect failures when the system attempts to write ciphertext to a 'plaintext-sized' DB column.

2024-12-20T12:55:17.204-08:00 ERROR 644 --- [nio-8080-exec-2] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.dao.DataIntegrityViolationException: could not execute statement [Value too long for column "STATE_IDENTIFICATION CHARACTER VARYING(10)": "'50JvKzHOGFDj7jENnL1LHvbwWAF/np5TICGzRgbE1K70OhEUVL8=' (52)"; SQL statement:
insert into principal (accepted_cookies,accepted_tos,account_enabled,account_non_expired,account_non_locked,avatar_url,bio,created_date,credential_no
...

Workaround is to remove the field size limits for encrypted properties (min and max really as min will not work as expected on content that is 'artificially' larger).

Trying to Launch Spring Boot From a Deleted Generated Folder

If you run ./vai and choose "clean" option then your generated spring directory will be moved to the trash and if you attempt to execute the service from that folder it will likely fail with an error such as:


MacBook-Pro:spring user$ mvn spring-boot:run -Dspring-boot.run.arguments="--spring.datasource.url=jdbc:mysql://ls-asdf.cjbdbnvzknle.us-asdf-2.rds.amazonaws.com:3306/valkyrai?useCursorFetch=true&defaultFetchSize=100&autoReconnect=true&useUnicode=yes&characterEncoding=UTF-8 --spring.jpa.hibernate.ddl-auto=update --spring.datasource.username=dbmasteruser --spring.datasource.password=ItsTh3Sam301Dstuff --spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL8Dialect"
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
pwd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
pwd: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
Error occurred during initialization of VM
java.lang.InternalError: platform encoding not initialized
at jdk.internal.util.SystemProps$Raw.platformProperties(java.base@23.0.1/Native Method)
at jdk.internal.util.SystemProps$Raw.<init>(java.base@23.0.1/SystemProps.java:263)
at jdk.internal.util.SystemProps.initProperties(java.base@23.0.1/SystemProps.java:67)
at java.lang.System.initPhase1(java.base@23.0.1/System.java:2165)

You simply need to change directory back to the new generated spring folder. You can typically just step back up into the generated folder and back into the spring folder like so:


...

> cd ..
> cd spring

...

File Contention in Development Builds

Since ./vai and ThorAPI in general do a lot of file moving/deleting/creating typically inline to an active IDE running various parts of the project at any one time.

Sometimes errors can be caused by the fact that an IDE has a source file open, or classpath file is locked for any reason, a path cannot be deleted, or an unexpected condition (inclduing full disks, disk i/o problems, bad path or filename characters, corrupt files, etc. )

These errors list random filenames, often while compiling AspectJ:

(base) MBP-1:ValkyrAI$ mvn clean install -DskipTests
...
[INFO] --- jar:3.4.2:jar (default-jar) @ generator ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:42 min
[INFO] Finished at: 2024-10-23T14:52:07-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-jar-plugin:3.4.2:jar (default-jar) on project generator: Error assembling JAR: /workspace/ValkyrAI/target/classes/com/valkyrlabs/model/Rating$AjcClosure399.class -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
(base) MBP-1:ValkyrAI$

The solution is to close all IDE windows, terminals, and kill any process that is possibly holding open a file in the project path.

Then rerun the build:

(base) MBP-1:ValkyrAI$ mvn clean install -DskipTests
...
~/.m2/repository/com/valkyrlabs/thorapi/generator/1.0-SNAPSHOT/generator-1.0-SNAPSHOT-exec.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:02 min
[INFO] Finished at: 2024-10-23T14:42:27-07:00
[INFO] ------------------------------------------------------------------------
(base) MBP-1:ValkyrAI$

Spec Validation Errors: Here we see the Spec is not passing basic OpenAPI validations

[ERROR]
org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
| Error count: 26, Warning count: 7
Errors:
-attribute components.schemas.EventLog.properties is not of type `object`
-attribute components.schemas.ContentData.properties is not of type `object`
-attribute components.schemas.Login.properties is not of type `object
2024-10-15T17:44:18.692-07:00 ERROR 55734 --- [nio-8081-exec-8] com.valkyrlabs.thorapi.audit.AuditingAspect      : getCurrentAuditor: java.lang.ClassCastException: Cannot cast java.lang.String to java.util.UUID

CodeGen Skipping Existing Files

Sometimes CodeGen will not overwrite existing files which may be a preference:

[INFO] writing file ~/workspace/ValkyrAI/generated/spring/src/main/java/com/valkyrlabs/api/WorkflowApiDelegate.java
[INFO] Implementation file ~/workspace/ValkyrAI/generated/spring/src/main/java/com/valkyrlabs/api/WorkflowStateApiController.java is not overwritten

You can delete the generated files using the ./vai command and responding Y to the second prompt to clean generated files.

Spring Framework Dependency Incompatibilities

The specific versions of all imported libraries, especially the Spring Framework and AspectJ, have been meticulously selected and tested by hand to ensure version compatibility.

When upgrading any of the libraries in the pom.xml and package.json files, please test and confirm all of the functionality of your system thoroughly.

Missing Required Fields

Ensure all required fields are properly specified in your OpenAPI spec and database schema.

Validation Conflicts

Check for conflicting validation rules in your OpenAPI spec that may prevent data from being processed correctly.

For example:

someData:
maxLength: 10
minLength: 20
type: string
description: this is broken data cannot be inserted
example: no such value exists that is longer than 20 and shorter than 10 thus validation will fail on any value

In addition to making it impossible to insert data, using a minimum value as a validation can also act as a required field -- because there must be a minimum value entered, it becomes a required field, even if that is not the intention.

By proactively addressing these issues, you can prevent runtime errors and ensure a smooth development experience.

Common Spec Problems

String Columns Too Large

Currently there are some boundary issues when choosing these values for string "maxLength"... some databases only support creation of VARCHAR columns which are limited to approx 16k characters in length but when trying to create a 20k size character column, it fails to 'upcast' to a larger 'text' type.

---
bioUrl:
type: string
description: More in-depth information about you and your account
maxLength: 20000
	at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:40) ~[valkyrai-api-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
Caused by: java.sql.SQLSyntaxErrorException: Column length too big for column 'bio_url' (max = 16383); use BLOB or TEXT instead
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:112) ~[mysql-connector-j-9.0.0.jar!/:9.0.0]
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:114) ~[mysql-connector-j-9.0.0.jar!/:9.0.0]
at com.mysql.cj.jdbc.ServerPreparedStatement.serverExec

The workaround is to choose a significantly higher size which will force the properly sized column:

---
bioUrl:
type: string
description: More in-depth information about you and your account
maxLength: 100000

Runtime Spec Database Issues

This one seems to be more of a Hibernate/JPA issue in auto creating columns for "boolean" fields.

Somehow field was being created as a bit(16) not a bit(1). Manually altered the table and will be keeping an eye on that one.

exec-9] DEBUG c.v.v.jwt.JwtRequestFilterHardened - Found 2 roles for user: 477ec552-5d9c-4562-9591-28e58dc6094f
17:06:32.129 [http-nio-8080-exec-9] INFO c.v.t.a.DataObjectAuditingListener - DATAOBJECT CREATED:com.valkyrlabs.model.AclObjectIdentity | 7235a238-63ee-4f2d-af8c-38aee60fe90f | userExists: true
17:06:32.250 [http-nio-8080-exec-9] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - Cannot determine value type from string '1'
17:06:32.252 [http-nio-8080-exec-9] ERROR c.v.v.controller.AuthController - ACL grant failed
org.springframework.dao.DataIntegrityViolationException: Could not extract column [9] from JDBC ResultSet [Cannot determine value type from string '1'] [n/a]; SQL [n/a]
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:297)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:256)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:241)
...

Non-Fatal Test Errors

During maven test runs you may encounter Spring JPA DML/DDL issues creating tables for tests using the H2 embedded database engine.

These may or may not impact your testing depending upon whether your test code touches upon the tables that are not able to be created.

Of course it is critical to ensure these errors You want to avoid show up on you production database.

	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:385) [surefire-booter-3.2.5.jar:3.2.5]
at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162) [surefire-booter-3.2.5.jar:3.2.5]
at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:507) [surefire-booter-3.2.5.jar:3.2.5]
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:495) [surefire-booter-3.2.5.jar:3.2.5]
Caused by: org.h2.jdbc.JdbcSQLSyntaxErrorException: Syntax error in SQL statement "create table opportunity (status tinyint check (status between 0 and 3), [*]value float(53), created_date timestamp(6) with time zone, estimated_close_date timestamp(6) with time zone, last_accessed_date timestamp(6) with time zone, last_modified_date timestamp(6) with time zone, customer_id uuid, id uuid not null, last_accessed_by_id uuid, last_modified_by_id uuid, owner_id uuid, description varchar(255), key_hash varchar(255), primary key (id))"; expected "identifier"; SQL statement:
create table opportunity (status tinyint check (status between 0 and 3), value float(53), created_date timestamp(6) with time zone, estimated_close_date timestamp(6) with time zone, last_accessed_date timestamp(6) with time zone, last_modified_date timestamp(6) with time zone, customer_id uuid, id uuid not null, last_accessed_by_id uuid, last_modified_by_id uuid, owner_id uuid, description varchar(255), key_hash varc
...
format: double
description: Expected value of the opportunity.
status:
type: string
enum:
- open
- won
- lost
- inactive
description: Status of the opportunity.
estimatedCloseDate:
...

Inner Classes and Many to Many

We do not yet FULLY support the 'oneOf' operator because it generates inner classes which are then unavailble to the Spec enhancer so they do not become decorated with required DataObject interface fields like getId().

But there are 2 workarounds:

  1. manually add the fields to the inline object definitions in the OpenAPI spec yaml
  2. move the inline object definition in the OpenAPI spec yaml to a separate object and use a $ref
---
McpToolCallResponse:
type: object
properties:
_meta:
type: object
additionalProperties: true
content:
type: array
items:
oneOf:
- type: object
properties:
type:
type: string
enum:
- text
text:
type: string
required:
- type
- text
- type: object
properties:
type:
type: string
enum:

which results in:

[INFO] Finished at: 2025-02-19T14:02:45-08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.13.0:compile (default-compile) on project generator: Compilation failure: Compilation failure:
[ERROR] generated/spring/src/main/java/com/valkyrlabs/api/McpToolCallResponseContentInnerPageableRepository.java:[42,92] cannot find symbol
[ERROR] symbol: class TypeEnum
[ERROR] location: interface com.valkyrlabs.model.McpToolCallResponseContentInner
[ERROR] generated/spring/src/main/java/com/valkyrlabs/api/McpToolCallResponseContentInnerService.java:[98,125] cannot find symbol
[ERROR] symbol: class TypeEnum
[ERROR] location: interface com.valkyrlabs.model.McpToolCallResponseContentInner
[ERROR] generated/spring/src/main/java/com/valkyrlabs/model/McpResourceResponseContentsInner.java:[58,8] com.valkyrlabs.model.McpResourceResponseContentsInner is not abstract and does not override abstract method setId(java.util.UUID) in com.valkyrlabs.model.DataObject
[ERROR] generated/spring/src/main/java/com/valkyrlabs/model/...

Workaround is to not use an ENUM for the Type discriminator, simply use a String

Here is the case where you can manually add the id, modified_date, etc. to the model to workaround the compilation errors.

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] generated/spring/src/main/java/com/valkyrlabs/model/McpResourceContentResource.java:[58,8] com.valkyrlabs.model.McpResourceContentResource is not abstract and does not override abstract method setId(java.util.UUID) in com.valkyrlabs.model.DataObject
[INFO] 1 error

Final corrected input looks like: Note the use of $ref objects and the use of the "_meta" which will become the name of the Map holding these named objects:

---
McpToolCallResponse:
type: object
properties:
_meta:
type: object
additionalProperties: true
content:
type: array
items:
oneOf:
- $ref: "#/components/schemas/McpTextContent"
- $ref: "#/components/schemas/McpImageContent"
- $ref: "#/components/schemas/McpResourceContent"
isError:
type: boolean
required:
- content

Duplicating Generated Props

You must avoid manually adding duplicate auto-generated fields like id or created_date to your OpenAPI spec.

ThorAPI may or may not dedupe the spec properly and almost certainly this will introduce the potential for bugs and malfunctioning of the code that depends on these fields.

Unsupported OpeanAPI Features

"Invented" ModelString type... casued by commas in the poperties definition.

Of course this is a buggy yaml, but somehow the yaml validation passed and in fact we got all the way to typescript generation... which then resulted in the odd "ModelString" artifact problem.

---
customerId:
type: string, # <- somehow this passes yal validation?
format: uuid,
description: Owning customer account
---
...
import {


ModelString, // <- does not exist
ModelStringFromJSON, // <- does not exist
ModelStringToJSON, // <- does not exist
} from './';
...

...
/**
* Owning account
* @type {ModelString}
* @memberof UsageTransaction
*/
accountId?: ModelString; // <- missing type... should just be a string!
...

TODO: fix

org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL "create table oas_parameter (id binary(16) not null, created_date datetime(6), description varchar(255), in tinyint check (in between 0 and 3), key_hash varchar(255), last_accessed_by_id binary(16), last_accessed_date datetime(6), last_modified_by_id binary(16), last_modified_date datetime(6), name varchar(255), oas_operation_id binary(16), owner_id binary(16), required bit, primary key (id)) engine=InnoDB" via JDBC [You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'in tinyint check (in between 0 and 3), key_hash varchar(255), last_accessed_by_i' at line 1]
name:
type: string
in: # <- bad
type: string
enum:
- query
- header

changed to:

name:
type: string
oasIn: # <- workaround
type: string
enum:
- query
- header

Incompatible Spec Changes Break Validations

ThorAPI creates validated fields on Java objects that the server uses to ensure that data is correct at every stage of interaction.

These validation rules are controlled in your OpenAPI spec by the 'format', 'pattern', 'type', and especially 'minSize', 'maxSize' attributes which all need to be correct when setting and getting values on these fields.

Also using enums in your spec may cause constraint failures if new or changed enum values are introduced in the spec. In this case, the constraints must be dropped and then the app restarted to update the constraint validations.

An additional factor is the use of SecureField feature which creates a validation on a column that is sized differently to accomodate the larger size of encrypted data.

Due to this feature, if your changes to the spec cause a validation on an existing field to change, it may impact the column size or the data type and thus cause a failure during startup or runtime when it encounters the existing column and data.

2024-12-19T10:14:33.373-08:00 ERROR 27745 --- [nio-8081-exec-7] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction] with root cause

jakarta.validation.ConstraintViolationException: Validation failed for classes [com.valkyrlabs.model.LlmDetails] during update time for groups [jakarta.validation.groups.Default, ]
List of constraint violations:[
ConstraintViolationImpl{interpolatedMessage='size must be between 16 and 1024', propertyPath=apiKey, rootBeanClass=class com.valkyrlabs.model.LlmDetails, messageTemplate='{jakarta.validation.constraints.Size.message}'}
]
at org.hibernate.boot.beanvalidation.BeanValidationEventListener.validate(BeanValidationEventListener.java:161)

Object is not an @Entity

Using type: object or other primitives can cause references to fail becuase there is no database object defined to hold the value of the array. Since primitives cannot be described as a database table and column, this will cause runtime failures.

You must use a specific schema defined reference to hold the values of collections (Lists and Maps)

---
properties:
schemas:
type: array
items:
$ref: "#/components/schemas/OasObjectSchema"
securitySchemes:
type: array
items:
type: object

using type: object causes:

itionValueResolver.java:365) ~[spring-beans-6.1.13.jar!/:6.1.13]
... 99 common frames omitted
Caused by: org.hibernate.AnnotationException: Association 'com.valkyrlabs.model.OasComponents.securitySchemes' targets the type 'java.lang.Object' which is not an '@Entity' type
at org.hibernate.boot.model.internal.CollectionBinder.detectManyToManyProblems(CollectionBinder.java:2607) ~[hibernate-core-6.5.3.Final.jar!/:6.5.3.Final]
at org.hibernate.boot.model.internal.CollectionBinder.bindM

to fix we must define a schema to use for the items array. Here we define a new schema OasSecurityScheme:

...
properties:
schemas:
type: array
items:
$ref: '#/components/schemas/OasObjectSchema'
securitySchemes:
type: array
items:
$ref: '#/components/schemas/OasSecurityScheme'

OasSecurityScheme:
type: object
description: a security scheme to access the api
properties:
name:
type: string
description: the name of the security scheme
...

Conflict with JPA Query names

Even with escaping, ending a property name with "in" will cause the generated queries to be thrown off:

---
_in:
type: string
enum:
- query
- header
- path
- cookie

causes...

bs.model.OasParameter$InEnum); Reason: Failed to create query for method public abstract java.util.List com.valkyrlabs.api.OasParameterRepository.findOasParameterByIn(com.valkyrlabs.model.OasParameter$InEnum); Operator IN on in requires a Collection argument, found class com.valkyrlabs.model.OasParameter$InEnum in method public abstract java.util.List com.valkyrlabs.api.OasParameterRepository.findOasParameterByIn(com.valkyrlabs.model.OasParameter$InEnum)

Fix was somewhat invasive until we can create a fix:

---
location:
type: string
enum:
- query
- header
- path
- cookie

Capitalization Issues

Using a component name like:

---
OASSimpleSchema:
type: object

results in:

[ERROR] ~/workspace/ValkyrAI/generated/spring/src/main/java/com/valkyrlabs/api/OASSimpleSchemaApiDelegate.java:[223,86] cannot find symbol
[ERROR] symbol: variable oASSimpleSchema
[ERROR] location: class com.valkyrlabs.api.OASSimpleSchemaApiDelegate

OpenAPI Spec and DB Keys

Here we have an error in the Spec around a foreign-key relationship:


Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2024-10-04T11:59:33.695-07:00 ERROR 16313 --- [ main] o.s.boot.SpringApplication : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Collection 'com.valkyrlabs.model.BuildUpdate.dependencies' is 'mappedBy' a property named 'buildUpdateId' which does not exist in the target entity 'com.valkyrlabs.model.DependencyUpdate'

so we update the definition in the template:

DependencyUpdate:
type: object
properties:
version:
type: string
scope:
type: string
status:
type: string
enum:
- UP_TO_DATE
- OUTDATED
- MISSING

and we add:

---
type: object
properties:
buildUpdateId:
type: string
version:
type: string

This addition allows spring JPA and Hibernate to determine the foreign key relationship between the objects.

When you create one-to-many or many-to-one relationships in the OpenAPI schema, Spring JPA will auto-generate constraints in the DB:

...
KEY FKkbr705782fp21q0ph1o43gmwc (acl_object_identity_fk_id),
CONSTRAINT FKkbr705782fp21q0ph1o43gmwc FOREIGN KEY (acl_object_identity_fk_id) REFERENCES acl_object_identity (id)
)
...
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Association 'com.valkyrlabs.model.WorkflowState.value' targets the type 'java.lang.Object' which is not an '@Entity' type

Caused by inability to use Object as a 'leaf' entity in the OpenAPI spec (at this time)

---
value:
description: value of the property
type: array
items:
type: DataObject

ROADMAP: implement many-to-many without adding identity columns to the interface

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Collection 'com.valkyrlabs.model.ExecModule.specs' is 'mappedBy' a property named 'execModuleId' which does not exist in the target entity 'com.valkyrlabs.model.OpenApiSpec'
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1806)

Be vigilant for common issues that may arise with your OpenAPI specifications, which can lead to errors during code generation or runtime.

2024-05-18 13:01:22.700 ERROR 3766 --- [nio-8080-exec-4] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.orm.jpa.JpaSystemException: could not execute statement; nested exception is org.hibernate.exception.GenericJDBCException: could not execute statement] with root cause

java.sql.SQLException: Field 'config_id' doesn't have a default value

Possible Errors and Resolutions:

Spec Validation Errors:

The generator validates the input specifications before generating code. If there is a problem you might see an error like this:

[ERROR]
org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).

NOTE: it is NOT recommended to skip validation of the specification. Doing so will likely introduce bugs and code that will not compile, if it produces anything at all.

Missing Generated Files

In this case, there are references to 'missing' model components in the generated apis.

Typically this means the api has been simplified and/or changed and there are still obsolete references. Also can happen when copying in definitions from other specs, or due to simple typos and spelling or capitalization errors.

    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:314)
[ERROR]
org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
| Error count: 32, Warning count: 6
Errors:
-attribute paths.'/BuildCreate'(post).responses.201.content.'application/json'.schema.#/components/schemas/BuildCreate is missing
-attribute paths.'/BuildUpdate'(get).responses.200.content.'application/json'.schema.#/components/schemas/BuildUpdate is missing
-attribute paths.'/BuildUpdate/{id}'(put).requestBody.content.'application/json'.schema.#/components/schemas/BuildUpdate is missing

Changing those references to the simplified spec clears up the issue in the generated output...

Object Inheritance Support

In this error we are getting TSX errors in classes that are extending or inheriting from a base class via the 'allOf' OpenAPI keyword.

  yarn build

Type '(result: void, error: FetchBaseQueryError, { id }: Pick<Solution, "id"> & Partial<Solution>) => { type: "Solution"; id: unknown; }[]'
...

BuildOutput:
allOf:
- $ref: '#/components/schemas/BaseEntity'
- type: object
properties:
buildId:
type: string
format: uuid
...

Solution:
allOf:
- $ref: '#/components/schemas/BaseEntity'
- type: object
type: object
properties:
buildOutputId:
type: string
format: uuid
...

The reality is, because Solution is a member of BuildOutput, it does not need the fields from the base class in any case as they represent duplicate data in the system.

For that reason, it is a simple matter to remove the allOf from 'Solution' and the typescript generation is fixed.

Because this is a "leaf" object in the heirarchy there is no need for it to duplicate the "BaseEntity" fields:

Solution:
type: object
properties:
buildOutputId:
type: string
format: uuid

ROADMAP: support for allOf inheritance functionality in Typescript

Capitalization Problems

Here we see a common pitfall of using a complex capitalization. Because ThorAPI renames to-and-from Java to database schemas, the camelcase and snake case conversions can clobber any capitalzation that is has than one capital letter in a row...

[INFO] --- aspectj:1.15.0:compile (aj-compile) @ valkyrai-api ---
[INFO] Showing AJC message detail for messages of types: [error, warning, fail]
[ERROR] The public type OpenApiSpec must be defined in its own file
./generated/spring/src/main/java/com/valkyrlabs/model/OpenAPISpec.java:73
public class OpenApiSpec {
^^^^^^^^^^

In this case renaming the OpenAPI spec component clears up the issue:

description: ValkyrAI Design Spec
xml:
name: OpenAPISpec

capitalization of "OpenAPISpec" changed to...

description: ValkyrAI Design Spec
xml:
name: OpenApiSpec

Support for OpenAPI additionalProperties

Support for defining properties as "additionalProperties" not yet supported.

The code generator does not handle object representation yet.

- Operation:
type: object
properties:
summary:
type: string
operationId:
type: string
parameters:
type: array
items:
$ref: "#/components/schemas/Parameter"
responses:
type: object
additionalProperties:
$ref: "#/components/schemas/Response"

Using Objects as Properties in the Spec

Suppoted:

...
- underlineDoubleAccounting
color:
type: string
enum:
- Black
- White
...

Not yet suppoted:

...
- colorViolet
- colorGray80
alignment:
type: object
properties:
horizontal:
type: string
...

Workaround is to use a named property instead of an object:

...
- colorViolet
- colorGray80
alignment-horizontal:
type: string
enum:
- Left
- Center
- Right
...

Using Enums in the Spec

Enums are very useful, be sure that all of the specifics exist in the spec:

generated/spring/src/main/java/com/valkyrlabs/model/Role.java:[115,41] cannot find symbol
[ERROR] symbol: variable ANONYMOUS
[ERROR] location: class com.valkyrlabs.model.Role

is caused by a missing default value:

roleName:
type: string
description: the role
enum: [EVERYONE, STAFF, ADMIN]
x-enum-descriptions:
- All Authenticated Users
- Staff Role
- Admin Role
x-enum-varnames:
- ROLE_EVERYONE
- ROLE_STAFF
- ROLE_ADMIN
default: ANONYMOUS <-- this default is NOT in the enum list

WARNING: Enums generate constraints in the databse and may need to be manually changed. This can also cause out of bound exceptions in the databse as they are generated with constraints on valid values -- be sure to drop the database constraints if you change the list of enums!

Deployment Errors

Errors and warnings are to be expected during development while making changes.

Changing Property Text Lengths

One example is if we change the size of a column in the OpenAPI spec by using the 'maxLength' property.

When spring JPA restarts, it will attempt to resize the columns and if there is existing data that is larger than the new column size, the attempt will fail, and the server will continue.

2024-10-29T14:31:49.868-07:00  WARN 5679 --- [           main] o.h.t.s.i.ExceptionHandlerLoggedImpl     : GenerationTarget encountered exception accepting command : Error executing DDL "alter table address modify column country varchar(3)" via JDBC [Data truncation: Data too long for column 'country' at row 1]

org.hibernate.tool.schema.spi.CommandAcceptanceException: Error executing DDL "alter table address modify column country varchar(3)" via JDBC [Data truncation: Data too long for column 'country' at row 1]

The simple fix for this is to either increase the "maxLength" property in the OpenAPI spec to fit the data, or to truncate the data in the column to the specified new maxLength of the spec.

JPA Errors

While POSTing a complex DataObject to the API we get the following error:

org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [com.valkyrlabs.model.OasOpenAPISpec#464cbd9a-11f8-4162-90e6-5d6c8ce4996d]
at org.hibernate.event.internal.DefaultMergeEventListener.entityIsDetached(DefaultMergeEventListener.java:426) ~[hibernate-core-6.6.2.Final.jar!/:6.6.2.Final]
at org.hibernate.event.internal.De

Database Schema Mismatch

Verify that your database schema aligns with your entity definitions and that migrations have been applied correctly.

Data Truncation of Columns for Text Data

Once you successfully modify the tables you might have the problem of trying to enter data that once worked is now too long. You will need to adjust the size of the column with the maxLength property.

...
[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] threw exception

com.mysql.cj.jdbc.exceptions.MysqlDataTruncation: Data truncation: Data too long for column 'country' at row 1
...

Incompatible Java Libs

Jakarta vs Javax no need to get into the saga, but when we try to use jakarta persistence we are getting:

	... 106 common frames omitted
Caused by: java.lang.IllegalArgumentException: methods with same signature getSchemaManager() but incompatible return types: [interface org.hibernate.relational.SchemaManager, interface jakarta.persistence.SchemaManager]

since javax.persistence is open source we are switching back to javax.persistence.

Product Information


SecureKMS

ValkyrAI includes a built-in SecureField key management system.

Default operation is automatic as ValkyrAI manages the key associated with each row of data in your database.

If you change your default key ... which you set with the THORAPI_SECRET_KEY environment variable, the SecureKMS will ensure that previously encrypted records will be decryptable.

The keys are stored in the SecureField database table.

WARNING: YOUR DATA MAY BE PERMANENTLY LOST IF YOU LOSE ACCESS TO THE SECURE_KEYS TABLE OR OTHERWISE BUNGLE THE THORAPI_SECRET_KEY

ROADMAP: pluggable kms

Valor

Valor consists of a supervisor/advisor LLM which may consiste of one or more aggregated LLM services such as OpenAI or Anthropic Claude.

  • advise on OpenAI design
  • advise on API best practices
  • realtime threat monitoring
  • evolving stacks (update/self-heal/bugfix)
  • learning stacks (model, gather, analyze)
  • predictive decisioning
  • data analytics
  • inline data monitoring intelligence

Here we are testing local OLLAMA setup.

curl -X POST \
http://localhost:11434/api/chat \
-H 'Content-Type: application/json' \
-d '{"text": "Hello", "role": "user"}'

curl http://localhost:11434/api/generate -d '{
"model": "llama3.2:1b",
"prompt": "What is your name and what are you known for?",
"raw": true,
"stream": false
}'


ollama run llama3.2:1b

curl http://localhost:11434/api/chat -d '{
"model": "llama3.2:1b",
"messages": [
{
"role": "user",
"content": "What is your name?"
}
]
}'

VIDAR

Vidar -- named after a Norse god of locking things up and guarding them tight -- is a data encryption utility that is used to zip up and encrypt file system data and create a self-extracting encrypted jar file.

For example, Vidar files can be used to zip and encrypt an entire generated ValkyrAI project -- prior to generation of the final service for compression purposes.

./vidar
Enter a password for encryption:
Zipping the payload directory...

./vai

vai command -- short for ValkyrAI -- is a simple bash script that gives developers an easy way to run the ValkyrAI generator, manage and deploy the front and backend artifacts, generate necessary encryption keys, run the vidar script and more.

./vai

☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
..................................................................................................
░▒▓█▓▒░░▒▓█▓▒░░▒▓██████▓▒░░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓███████▓▒░ ░▒▓██████▓▒░░▒▓█▓▒░
░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▒▒▓█▓▒░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▒▒▓█▓▒░░▒▓████████▓▒░▒▓█▓▒░ ░▒▓███████▓▒░ ░▒▓██████▓▒░░▒▓███████▓▒░░▒▓████████▓▒░▒▓█▓▒░
░▒▓█▓▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓█▓▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
░▒▓██▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓████████▓▒░▒▓█▓▒░░▒▓█▓▒░ ░▒▓█▓▒░ ░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️ ©2024 Valkyr Labs Inc ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️ https://valkyrlabs.com/vai ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️ San Francisco, California ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️ support@valkyrlabs.com ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️ ☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️
☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️☮️

WELCOME TO THE VAI GUIDED RUNNER
Enter the workspace directory [/workspace/ValkyrAI]:
/workspace/ValkyrAI
Clean generated files from prior run...? (Y/n) [n]:
n
Skipping cleaning generated files.
Enhance the input OpenAPI with ThorAPI extensions...? (Y/n) [Y]:n
n
Skipping enhancing OpenAPI.
Build generator and generate code from enhanced OpenAPI...? (Y/n) [Y]:n
n
Skipping generator rebuild.
Do you want to run the backend instance for development? (Y/n) [Y]:n
n
Skipping backend instance run.
Do you want to build and run the frontend instance for development? (Y/n) [Y]:n
n
Skipping frontend instance run.
Do you want to deploy generated artifacts now? (Y/n) [Y]:n
n
Deployment skipped.
Do you want to SSH into loki.valkyrlabs.com to restart services? (Y/n) [Y]:n
n
Skipping SSH into server.
Do you want to generate a JWT Secret Key? (Y/n) [Y]:n
n
Skipping generating a JWT Secret Key...
Do you want to generate a THORAPI Secret Key? (Y/n) [Y]:n
n
Skipping generating a THORAPI Secret Key...
Do you want to create an encrypted VIDAR file? (Y/n) [Y]:n
n
Skipping generating a VIDAR encrypted file...
Script execution completed successfully.
(base) MBP-1:ValkyrAI $

GridHeim

GridHeim is an online spreadsheet component that works with ThorAPI REST apis.

The GridHeim component allows you to work with Spreadsheets via a secure REST api, as well as embed spreadsheet functionality in your applications.

Used internally by ValkyrAI and Valor, GridHeim is the ultimate end-user development UI allowing for easy building of data-management systems, formula-driven business logic, logic tables, dashboards and reports.

ROADMAP: full gridheim implementation


Resources

Workflow Engine Architecture

The ValkyrAI WF Engine works via a Quartz scheduler that executes Workflows on a schedule at intervals.

Workflows can also be run and managed on demand via REST endpoints.

Workflows consist of a series of Tasks which have a series of Execution Modules and Data Handlers that perform a variety of automated functions.

When run by Valor and via a collection of basic Exec Modules and in conjunction with OpenAPI Specs, the WF Engine is capable of nearly any task of any complexity.

Quartz scheduling is determined by a "CRON" string and can be set via the it's normal ThorAPI.

The api.hbs.yaml Spec Template

An example OpenAPI spec schema model in the api.hbs.yaml:

ChatResponse:
type: object
description: Chatresponse from the other side of the communication
properties:
content:
maxLength: 100000
minLength: 10
type: string
description: the response fro the chat service
example: We must cross the mighty Mississippi

... and the resulting output after running ThorAPI enhancement:

ChatResponse:
type: object
description: Chatresponse from the other side of the communication
properties:
content: { maxLength: 100000, minLength: 10, type: string, description: the
response fro the chat service, example: We must cross the mighty Mississippi, x-field-extra-annotation: "@Column(length = 100000)" }
id:
type: string
description: Unique identifier for object in the system
example: 8c8a76cc-191d-403c-a017-a7312fa60d82
format: uuid
x-field-extra-annotation: "@Id \n @GeneratedValue(generator = \"\
UUID\")\n"
ownerId:
{
type: string,
description: UUID of owner of the object in the system,
example: 41c047b3-b060-4313-817c-95d15c83a8e1,
format: uuid,
x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.CREATED_BY, enabled = true)",
}
createdDate:
{
type: string,
description: Date of object creation,
example: "2024-12-10T14:55:52.718-0800",
format: date-time,
x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.CREATED_DATE, enabled = true)",
}
keyHash:
{ type: string, description: "Data, including hash of the key(s) used
to encrypt this record." }
lastAccessedById:
{
type: string,
description: Last user to access object,
example: 4ed218b1-a423-4c87-9e9d-4387e2fcfdc5,
format: uuid,
x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.LAST_ACCESSED_BY, enabled = true)",
}
lastAccessedDate: { type: string, description: Timestamp of last access of
object, example: "2024-12-10T14:55:52.718-0800", format: date-time, x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.LAST_ACCESSED_DATE, enabled = true)" }
lastModifiedById:
{ type: string, description: Unique identifier for user who
last modifed the object in the system, example: 5ec21e3b-13f7-4b07-9c57-8a977a93d2c1, format: uuid, x-field-extra-annotation: " @AuditingField(fieldType
= AuditingField.FieldType.LAST_MODIFIED_BY, enabled = true)" }
lastModifiedDate:
type: string
description: Date of last object modification
example: 2024-12-10T14:55:52.719-0800
format: date-time
x-field-extra-annotation: "@JsonIgnore \n @AuditingField(fieldType\
\ = AuditingField.FieldType.LAST_MODIFIED_DATE, enabled = true)"

ValorIDE

ValorIDE is the ValkryAI powered Coding Agent for VSCode.

Using ValorIDE makes the most of ValkyrAI CodeGen productivity gains for developers.

Inline CodeGen and AI-powered Coding unlocks incredible productivity.

MCP Services

In the 6 months since this project began there have been accelerating changes across the space.

One great development has been "Model Context Protocol" or MCP.

MCP is going to be a big part of ValkyrAI and how it functions and the services it provides both as a consumer and publisher.

Here are some of our Roadmap Items:

  • MCP -> ThorAPI CodeGen as an MCP service
  • MCP <- utilize MCP as ValorIDE tooling
  • MCP <-> Utilize MCP as ValkyrAI tooling
  • MCP Publishing Protocol (Over RSS)
  • MCP -> Component Library Generator
  • MCP -> Preferences Stack
  • MCP -> Gridheim Sheetster: Data, Formulas, Import/Export
  • MCP -> Asset generation (calls various inference stack)
  • MCP -> configure the inference stack dynamically (used by Agents)
  • MCP -> configure the inference stack dynamically (used by Agents)

GLOSSARY

Spring Query Methods: https://docs.spring.io/spring-data/jpa/reference/jpa/query-methods.html

Mustache Lambdas:

lowercase {{#lambda.lowercase}}{{name}}{{/lambda.lowercase}}
uppercase {{#lambda.uppercase}}{{name}}{{/lambda.uppercase}}
snakecase {{#lambda.snakecase}}{{name}}{{/lambda.snakecase}}
pascalcase {{#lambda.pascalcase}}{{name}}{{/lambda.pascalcase}}
camelcase {{#lambda.camelcase}}{{name}}{{/lambda.camelcase}}
kebabcase {{#lambda.kebabcase}}{{name}}{{/lambda.kebabcase}}

RESULT: lowercase addresstype uppercase ADDRESSTYPE snakecase address_type pascalcase AddressType camelcase addressType uncamelize Address Type kebabcase address-type

Expand your knowledge and get additional support by exploring the following resources:

Contributing

We welcome contributions from our user community!

If you are a source code available customer and are interested in improving ValkyrAI, please follow these steps:

  1. Fork the Repository: Create a personal fork of the ValkyrAI repository on GitHub.

  2. Create a Feature Branch: Develop your feature or fix in a dedicated branch.

  3. Commit Your Changes: Make clear and descriptive commit messages.

  4. Open a Pull Request: Submit your changes for review, providing detailed information about the changes you've made.

  5. Participate in the Review Process: Engage with maintainers and address any feedback or requested changes.

For more detailed guidelines, please read our Contribution Guidelines.

Code of Conduct

Please note that all contributors are expected to adhere to our Code of Conduct. We are committed to fostering a welcoming and respectful community.

License


Changes

  • Added valor_inference_prompt.txt for the Inference Stack LLM Coding Model.

Contributing

Valkyr Labs products are licensed under a combination of open source licensing for core components such as SecureField in order that customers and the community may benefit from transparency and utility of our software, as well as contribute to its ongoing development and quality.

The nature of ThorAPI is open source friendly as a security-minded CodeGen solution we feel the increased security will benefit all of society and the spreading far and wide of SecureField protection and ThorAPI's beautiful consistent APIs will benefit customers, users, and organizations across the world.

For purposes of transparency and support as well as allowing users to customize their Valkyr stacks, we provide source code licensed under the Valkyr Labs Inc (VLI) Source License VLI Source License. .

ThorAPI is available under the MIT License.

By contributing to Valkyr Labs Inc repositories, you agree that your contributions will be licensed under the same license.


Disclaimer

WARNING!

AUTOMATED CODING MUST BE REVIEWED AND APPROVED BY HUMANS

NO NOT ALLOW AGENTIC CODE TO GO INTO PRODUCTION ENVIRONMENTS WITHOUT THOROUGH TESTING

ValkyrAI, Valor, ValorIDE, and ThorAPI are powerful tools intended to accelerate development and enhance security.

However, AI makes mistakes, it makes incorrect assumptions, and it can do bad things and do them at scale.

It is crucial to review and test all generated code thoroughly to ensure it meets your application's requirements and complies with all relevant regulations and best practices.


Spring, Spring Boot and Spring Cloud are trademarks of Pivotal Software, Inc. in the U.S. and other countries.

OpenAI Credentials & Fallback

This project supports calling OpenAI-compatible chat completions from two paths:

  • Controller path: valkyrai REST endpoint (LLMController)
  • Workflow path: adapters used by workflow modules (LlmAdapter/OpenAIAdapter)

To prevent accidental subsidizing of hosted users, environment-key fallback is disabled by default. Each user/workflow should provide its own API key unless you explicitly enable fallback for a controlled environment.

Behavior

  • Controller (LLMController): Uses the apiKey on the LlmDetails record. If missing and fallback is disabled, it returns a clear error. If fallback is enabled, it uses OPENAI_API_KEY from the environment/system property when available.
  • Workflow adapters (LlmAdapterFactoryOpenAIAdapter): If no key is provided and fallback is disabled, it throws an IllegalStateException with guidance. If fallback is enabled, it uses OPENAI_API_KEY when present.

Properties

  • valkyrai.openai.allowEnvFallback (default: false)
    • false: No fallback. A missing key fails fast with clear messaging.
    • true: Fallback to OPENAI_API_KEY (and system property) is allowed.

Environment Variables

  • OPENAI_API_KEY: API key to use when fallback is enabled.
  • OPENAI_ORG_ID (optional): Sets OpenAI-Organization header.
  • OPENAI_BASE_URL (optional): Overrides base URL used by the adapter; default is https://api.openai.com/v1.

Examples

application.yml (recommended: enable fallback only for dev)

spring:
profiles:
active: dev
---
spring:
config:
activate:
on-profile: dev
valkyrai:
openai:
allowEnvFallback: true

---
spring:
config:
activate:
on-profile: prod
valkyrai:
openai:
allowEnvFallback: false

Shell (dev only)

export OPENAI_API_KEY="sk-..."
# optional
export OPENAI_ORG_ID="org_..."
export OPENAI_BASE_URL="https://api.openai.com/v1"

Error Messages

  • Controller path (fallback disabled):

    • OpenAI provider is set but no API key provided for this service. Environment fallback is disabled. Set apiKey on LlmDetails or enable 'valkyrai.openai.allowEnvFallback=true' and configure OPENAI_API_KEY.
  • Workflow adapter path (fallback disabled):

    • OpenAI API key not provided. Set apiKey in the workflow configuration or enable 'valkyrai.openai.allowEnvFallback=true' and configure OPENAI_API_KEY.

Notes

  • Keep fallback disabled in production-hosted environments to avoid subsidizing user workloads.
  • When enabled, fallback uses environment variable first, then system property.
  • The adapter logs error bodies from OpenAI on non-2xx responses to aid diagnosis (e.g., 401 auth issues).