Developer Integration Guide

This guide walks you through integrating GrayMatter into your ValkyrAI workflows, ValorIDE agents, and external applications.

Credit charges are runtime-configured. Use the main site Pricing page for current credit rates, and treat examples in this guide as usage-shape guidance only.

Agentic Install And Readiness

Agents should not treat GrayMatter as an optional note file. Install and validate it before relying on durable memory or the RBAC-visible object graph.

git clone https://github.com/ValkyrLabs/GrayMatter.git
cd GrayMatter
test -x scripts/gm-activate || ./graymatter-bootstrap
scripts/gm-activate
scripts/gm-doctor --quick

For a ValkyrAI checkout with a sibling GrayMatter repo or installed Codex plugin cache:

scripts/graymatter-doctor.sh --quick

Readiness means the doctor reports:

• auth is write-capable
• memory layer is ready
• graph, strategic, and KPI layers are visible from the cached/live OpenAPI
• MCP contract is available for agent hosts
• deferred replay is clear or replayable

Run scripts/gm-doctor without --quick when you need live write/query proof and are comfortable with the small query credit cost.

Quick Start: 5-Minute Integration

1. Create a Memory Entry

import { MemoryEntryService } from "@valkyr-labs/api-client";

const memory = new MemoryEntryService({ token: process.env.JWT_TOKEN });

const entry = await memory.createEntry({
  type: "preference",
  title: "Always validate email before signup",
  text: "Check email format with RFC 5322 validator. Reject disposable domains. Confirm ownership via link.",
  tags: ["auth", "validation", "pattern"],
  metadata: { source: "best-practices", confidence: 0.98 },
});

console.log(`Stored: ${entry.id}`);

2. Query Memory Before Decision-Making

const prefs = await memory.queryEntries({
  query: "email validation",
  type: "preference",
  limit: 5,
});

// Use retrieved memories in your logic
const emailRules = prefs.items.map((item) => item.text).join("\n");
const validationPrompt = `Follow these rules:\n${emailRules}\nNow validate this email: ${userEmail}`;

3. Store Learnings After Execution

// After your agent completes work, capture insights
const result = await memory.createEntry({
  type: "decision",
  title: "Email validation saved 23% invalid signups",
  text: `Implementing strict email validation reduced signup failures by 23%. Disposable domain blocking was the biggest win (12% reduction).`,
  tags: ["auth", "metrics", "learning"],
  metadata: {
    source: "agent-run",
    duration: "2 days",
    impact: "23% reduction in invalid signups",
  },
});

---

Integration Patterns

Pattern 1: Pre-Task Memory Read (Context Enrichment)

Before executing a workflow task, fetch relevant memories to enrich the context.

Workflow YAML:

workflow:
  name: "Customer Onboarding"
  tasks:
    - id: "read-preferences"
      module: "memory_read"
      config:
        query: "customer-onboarding:process"
        type: "preference"
        limit: 10
      outputs:
        - onboarding_rules

    - id: "enrich-context"
      module: "custom_js"
      inputs:
        - onboarding_rules
      script: |
        return {
          rules: input.onboarding_rules,
          enriched: true
        }

    - id: "execute-onboarding"
      module: "stripe_customer_create"
      inputs:
        - enriched_rules
        - customer_data
      config:
        ruleSet: "{{enriched_rules}}"

Pros: Ensures every execution begins with current institutional knowledge.
Cons: Adds read cost and latency to every run.
Credit impact: Depends on the active pricing matrix and the number of memory reads per task.

Pattern 2: Post-Task Memory Write (Learning Capture)

After a task completes (especially on failure), capture learnings for future runs.

Workflow YAML:

- id: "capture-execution-result"
  module: "memory_write"
  inputs:
    - execution_result
  config:
    type: "context"
    title: "{{execution_result.action}} completed on {{execution_result.timestamp}}"
    text: |
      Action: {{execution_result.action}}
      Status: {{execution_result.status}}
      Duration: {{execution_result.duration}}
      Errors: {{execution_result.errors}}
    tags:
      - "execution:{{execution_result.action}}"
      - "status:{{execution_result.status}}"
      - "date:{{execution_result.date}}"

Pros: Builds audit trail and searchable execution history.
Cons: Writes on every run; manage write volume carefully.
Credit impact: Depends on the active pricing matrix and write volume.

Pattern 3: Conditional Memory-Driven Branching

Use memory query results to decide which branch to take.

Workflow YAML:

- id: "check-billing-rules"
  module: "memory_read"
  config:
    query: "billing-override:special-cases"
    type: "preference"
  outputs:
    - billing_exceptions

- id: "apply-rules-or-standard"
  module: "branching"
  inputs:
    - billing_exceptions
  branches:
    - condition: "length(billing_exceptions) > 0"
      tasks:
        - module: "stripe_apply_override"
          config:
            overrides: "{{billing_exceptions}}"
    - condition: "else"
      tasks:
        - module: "stripe_apply_standard"
          config:
            plan: "standard"

Pros: Business rules live in queryable memories, not hardcoded.
Cons: Adds read cost and query latency.
Credit impact: Depends on the active pricing matrix and query volume.

Pattern 4: Agent Self-Improvement Loop

Agents store observations, adapt behavior based on historical patterns.

ValorIDE Agent:

import { Agent, AgentContext } from "@valoride/agent";
import { MemoryEntryService } from "@valkyr-labs/api-client";

export class self-improving-agent extends Agent {
  private memory: MemoryEntryService;

  async initialize(ctx: AgentContext) {
    this.memory = new MemoryEntryService({
      token: ctx.auth.token
    });
  }

  async execute(input: string, ctx: AgentContext) {
    // 1. Fetch agent's learned patterns
    const patterns = await this.memory.queryEntries({
      query: input,
      tags: ["agent-pattern"],
      limit: 5
    });

    // 2. Combine patterns with current task
    const enrichedPrompt = `
      You are ${ctx.agent.name}.

      Your learned patterns for similar tasks:
      ${patterns.items.map(p => `- ${p.text}`).join("\n")}

      Current task: ${input}
    `;

    // 3. Execute with enriched context
    const result = await ctx.llm.generate(enrichedPrompt);

    // 4. Observe outcome and store if successful
    if (result.confidence > 0.8) {
      await this.memory.createEntry({
        type: "decision",
        title: `Pattern for: ${input.substring(0, 50)}`,
        text: result.reasoning,
        tags: ["agent-pattern", `agent:${ctx.agent.id}`],
        metadata: {
          confidence: result.confidence,
          taskType: input.split(":")[0],
          successCount: 1
        }
      });
    }

    return result;
  }
}

Pros: Agent improves with each run; learns from successes and failures.
Cons: Requires careful management of pattern storage; old patterns may become stale.
Credit impact: Plan for at least one read and one conditional write per run; current rates are shown on Pricing.

---

Cost Management

Estimating Memory Costs

Scenario	Reads	Writes	Operations	Total Credits
Single task with 5 memory reads	5	0	0	5
Single task read + write result	5	1	0	6
Daily compact + reindex	0	0	1×10 + 1×15	25
Agent 30 runs/day with read+write	150	30	0	180
Total monthly (small team)	4,500	900	75	5,475

Optimization Tips

1. Batch reads: Instead of reading 1 memory per task, read 5-10 in a single query.

   • Credit impact: Batching usually reduces repeated read charges versus separate calls.

2. Throttle writes: Not every execution deserves a memory write.

   • Only write: errors, anomalies, high-confidence learnings.
   • Cost savings: 50-70% write reduction.

3. Use compact/reindex sparingly: Monthly, not daily (unless hit rate drops below 70%).

   • Credit impact: Avoids unnecessary maintenance runs.

4. Tag strategically: Use tags to narrow queries and avoid full table scans.

   • Credit impact: Narrow queries reduce operation volume and downstream token use.

5. Monitor burn rate: Watch /memory/stats for burnRate spikes.

---

Security & Access Control

Securing Sensitive Data in Memory

Use @SecureField annotation to encrypt sensitive data at rest:

const entry = await memory.createEntry({
  type: "preference",
  title: "Stripe API integration",
  text: "API_KEY=sk_live_xyz... (this will be encrypted at rest)", // SecureField
  tags: ["payment", "stripe"],
  metadata: {
    encrypted: true,
    keyId: "stripe-api-key-v1",
  },
});

Decryption is transparent in-memory; encrypted in the database.

Role-Based Access

Control who can read/write memory by role:

// Only STRIPE_ADMIN can read/write payment memories
const paymentMemories = await memory.queryEntries({
  query: "payment",
  requiredRoles: ["STRIPE_ADMIN"],
});

Audit Trails

All memory operations are logged immutably:

const logs = await memory.getAuditLog({
  entryId: "550e8400-e29b-41d4-a716-446655440000",
  action: "READ",
  limit: 100,
});

logs.forEach((log) => {
  console.log(`${log.principal} ${log.action} at ${log.timestamp}`);
});

---

Testing with GrayMatter

Unit Testing

Mock memory calls in unit tests:

import { jest } from "@jest/globals";
import { MemoryEntryService } from "@valkyr-labs/api-client";

describe("EmailValidator", () => {
  it("should reject disposable domains", async () => {
    const mockMemory = {
      queryEntries: jest.fn().mockResolvedValue({
        items: [
          {
            id: "1",
            text: "Reject disposable domains: tempmail.com, 10minutemail.com",
          },
        ],
      }),
    };

    const validator = new EmailValidator(mockMemory);
    const result = await validator.validate("user@tempmail.com");

    expect(result.valid).toBe(false);
    expect(result.reason).toContain("disposable");
  });
});

Integration Testing

Use a test account with isolated memory namespace:

beforeEach(async () => {
  // Create test memory entry
  testEntry = await memory.createEntry({
    type: "preference",
    title: "Test pattern",
    text: "Test",
    tags: ["test-suite", `test-run:${Date.now()}`],
  });
});

afterEach(async () => {
  // Clean up test memories
  await memory.deleteEntry(testEntry.id);
});

it("should query and use test pattern", async () => {
  const results = await memory.queryEntries({
    tags: [`test-run:${Date.now()}`],
  });

  expect(results.total).toBe(1);
  expect(results.items[0].id).toBe(testEntry.id);
});

Load Testing Memory

Simulate memory operations at scale:

async function loadTest() {
  const start = Date.now();
  const promises = [];

  // Simulate 100 concurrent reads
  for (let i = 0; i < 100; i++) {
    promises.push(memory.queryEntries({ query: "payment", limit: 10 }));
  }

  const results = await Promise.all(promises);
  const duration = Date.now() - start;

  console.log(
    `100 concurrent reads: ${duration}ms (${(duration / 100).toFixed(0)}ms avg)`,
  );
  // Expected: <500ms total, <5ms per read (with caching)
}

---

Common Pitfalls & Solutions

⚠️ Pitfall 1: Memory Bloat from Over-Writing

Problem: Agent writes on every run → 1,000s of nearly-identical entries.

Solution:

// ❌ Don't do this
for (let i = 0; i < result.length; i++) {
  await memory.createEntry({
    type: "context",
    text: `Processed ${result[i].id}`,
    tags: ["execution"],
  });
}

// ✅ Do this instead
const summary = await memory.createEntry({
  type: "context",
  title: `Batch processed ${result.length} items`,
  text: `Processed items: ${result.map((r) => r.id).join(", ")}`,
  tags: ["execution", `batch:${Date.now()}`],
  metadata: {
    count: result.length,
    batchId: nanoid(),
  },
});

⚠️ Pitfall 2: Stale Memory Causing Incorrect Decisions

Problem: Agent reads old preference → makes outdated decision.

Solution:

// ✅ Check recency
const prefs = await memory.queryEntries({ query: "billing-rate" });
const recentPrefs = prefs.items.filter((p) => {
  const age = Date.now() - new Date(p.updatedAt).getTime();
  return age < 7 * 24 * 60 * 60 * 1000; // Less than 7 days old
});

if (recentPrefs.length === 0) {
  console.warn("No recent billing preferences found. Using defaults.");
}

⚠️ Pitfall 3: Uncontrolled Query Costs

Problem: Agent queries all 10,000 memory entries on each run.

Solution:

// ❌ Don't do this
const all = await memory.queryEntries({ limit: 10000 }); // high-volume read pattern

// ✅ Do this instead
const targeted = await memory.queryEntries({
  query: "payment",
  tags: ["stripe", "recent"], // Narrow down with tags
  limit: 10, // Limit results
}); // lower-volume read pattern

---

Monitoring & Observability

Watch Memory Health

setInterval(
  async () => {
    const stats = await memory.getMemoryStats();

    console.log({
      hitRate: `${(stats.memory.hitRate * 100).toFixed(1)}%`,
      utilization: `${(stats.context.usedPct * 100).toFixed(1)}%`,
      burnRate: `${Math.round(stats.cost.burnRate)} credits/hour`,
    });

    // Alert if unhealthy
    if (stats.memory.hitRate < 0.7) {
      console.error("⚠️  Hit rate low. Consider reindex.");
    }
    if (stats.context.usedPct > 0.85) {
      console.error("⚠️  High utilization. Consider compact.");
    }
  },
  60 * 60 * 1000,
); // Every hour

Track Agent Memory Usage

const usage = await memory.getMemoryRunsHistory({ limit: 30 });

const byAgent = {};
usage.runs.forEach((run) => {
  byAgent[run.agentId] = (byAgent[run.agentId] || 0) + run.creditsCharged;
});

// Display as whole credits (no decimals - credits are indivisible cents)
console.table(byAgent);