Governed applied intelligence infrastructure.
- Positioning
- Audit-grade conversational and retrieval AI under regulatory pressure
- Heritage
- Thirteen years of conversational AI at Genesys, rebuilt for the LLM substrate
- Extensions
- keystone-engage · keystone-counsel · keystone-verify
- Substrate
- Hash-chained audit · authorization-first retrieval · sealed eval artifacts
What Keystone is, in plain language.
Keystone Applied Intelligence is a platform for building AI systems that regulated enterprises can actually deploy. It is three extensions running on one shared substrate, with governance, authorization, and evaluation designed in from the first commit rather than bolted on afterward.
The three extensions cover distinct regulated verticals. keystone-engage is a governed conversational agent for regulated customer interaction. keystone-counsel is authorization-aware retrieval for legal and financial advisory content. keystone-verify is a standalone evaluation harness that runs against any RAG or agent endpoint over HTTP, including the other two. GETKEYSTONE ORG
All three sit on the same substrate: an agents registry, a task state machine with explicit ownership, a hash-chained tamper-evident audit ledger, a NATS JetStream event bus, MCP-exposed tools with agent-scoped authorization, and cost-aware dispatch. The shared substrate is documented in the substrate architecture: governed retrieval and agent API with hybrid search, query-time ACL, severity-tier HITL routing, HMAC audit chain, and per-step evidence gating. Extensions plug into it; they do not each rebuild it.
The differentiator, stated directly.
Most LLM-era implementations lack the operational rigor the contact center industry already developed for compliance and governance reasons. Keystone is that rigor rebuilt for the LLM substrate. Severity-tier human-in-the-loop routing traces to bot-to-human escalation with formal severity classification. Per-step evidence gating traces to frame-based dialog slot validation. The HMAC action audit chain traces to contact center compliance logging. Fail-closed at retrieval traces to confidence-threshold escalation. Publishing the failing run alongside the passing run traces to contact center quality management. These are not metaphors. They are the same patterns from thirteen years of enterprise contact center work, rebuilt for the new substrate.
A RAG demo shows retrieval-augmented generation working on a corpus. Keystone shows what has to be true around retrieval and generation for a regulated enterprise to deploy them: who is authorized to ask what, what fails closed instead of guessing, what the audit chain records, how the eval ledger preserves failures, and how the coordination between agents is legible under trace. Retrieval and generation are components. Governance, authorization, and evaluation are the platform.
The substrate underneath the three extensions.
The word "platform" is only truthful when the extensions share real infrastructure. Here is what they share, and what a viewer can inspect in the repos.
Six substrate elements, each inspectable.
Agents registry
Every agent registered with identity, role, tempo (fast, medium, slow, deferred), and cost profile. Five specialists in Engage today. Every audit entry references an agent by id. ENG-DOCS
Task state machine
Nine states with validated transitions, heartbeat tracking, and an explicit takeover protocol. Tasks have owners. Stuck tasks are recoverable. This is not a queue.
Hash-chained audit ledger
HMAC-verified append-only entries with prev-hash and curr-hash. Every retrieval, every tool call, every authorization decision, every escalation. Tamper-evident by design.
NATS JetStream event bus
A JetStream event bus fans out task-lifecycle events across the substrate: the coordination layer for tempo-heterogeneous multi-agent operation.
MCP tool exposure
Model Context Protocol server entry point on Engage. Tool authorization takes both user identity and agent identity as input. Different agents get different scopes.
Cost-aware dispatch
Every audit entry records input tokens, output tokens, model used, cost in cents, and session-rolling cost. Dispatch carries a budget. Evaluation reports cost alongside accuracy.
The three extensions, each with a real scenario.
A viewer with ten minutes should be able to see one scenario from each extension: the input, the substrate's response, and the artifact that proves what happened. All scenarios run locally.
What runs
The Engage Coordinator runs a five-phase pipeline across a set of registered specialist agents, covering empathy, escalation risk, engagement, budget, and monitoring. Each phase emits an audit entry. The event bus emits task lifecycle events. Short-circuit conditions halt the pipeline when escalation severity crosses tier. ENG-DOCS
Scenario shape
The Engage source is proprietary. The block below shows the audit-output shape for a distressed-customer turn — not source code. Architecture and capability detail: keystone-engage docs. Live demos and read-only technical review on request — see access.
Shape of a session's audit chain
Field names are the real schema. Specific hashes and per-step cost values are from a representative session; a live make audit-dump produces the same shape.
entry_003 agent=<empathy> tempo=fast cost_cents=… input_tokens=… output_tokens=… latency_ms=… prev_hash=8a4f… curr_hash=b219… entry_004 agent=<escalation> tempo=fast cost_cents=… severity_tier=<tier> hitl_required=true prev_hash=b219… curr_hash=e75c… entry_005 agent=<budget> tempo=medium cost_cents=… short_circuit=false prev_hash=e75c… curr_hash=1f0a…
What this proves
Multi-agent coordination with tempo heterogeneity, per-agent audit trail, cost recorded on every step, tamper-evident chain via prev/curr hash, and severity-tier HITL routing that triggers before the outbound response is composed. ENG-EVAL 100/100 baseline sealed. ENG-V0 FAILING preserved next to it. SEALED ARTIFACTS
What runs
Counsel's retrieval pipeline runs classification-aware pgvector ACL filtering. The authorization check is not a post-retrieval filter in application code. It is a WHERE clause in the database query. If a document classification is not authorized for the caller, the row does not come back at all. Fail-closed is the default; a low-confidence retrieval triggers refusal, not a best-effort guess. COU-DOCS
The filter, at the layer that matters
Authorization is applied inside the retrieval query, not as a post-retrieval filter in application code. The caller's authorized classifications constrain the candidate rows before ranking, so any row the caller is not permitted to see is never returned in the first place. The exact query, operators, and thresholds are not published.
Two callers, same query, different outcomes
The fail-closed behavior and the zero-leak guarantee are the real policy. Chunk counts and confidence values are from a representative query.
# senior counsel: authorized — results returned $ counsel-query --as senior_counsel --query "client suitability guidance" → authorized results returned (multiple classifications) # junior advisor: not authorized — fails closed $ counsel-query --as junior_advisor --query "client suitability guidance" → 0 authorized results → FAIL_CLOSED: refusing to answer; escalation logged → audit_entry: authz_denied=true, chunks_leaked=0
What this proves
Authorization is enforced at the database, not in application code. A bug in the orchestrator cannot leak content the caller is not permitted to see, because the query never returned it. Fail-closed with zero chunks leaked is the default behavior, verified against 8 of 8 adversarial ACL probes on the retrieval baseline. COU-EVAL 30/30 baseline sealed. EVAL ARTIFACTS
What runs
Verify is an endpoint-agnostic evaluation harness: a profile system, a pure-function judge engine, a click CLI, and a sealed artifact reporter. It ships with two reference profiles targeting Engage and Counsel, and it runs against any HTTP endpoint that matches the profile contract. Results write to results.json with per-case assertions and to run_metadata.json with latency and cost stats. VER-REPO
Run it against Engage
Case counts 186 · 558 · 0 are from the sealed agent-v1 baseline. Elapsed and cost values shown are representative.
$ verify run \
--profile profiles/engage.yaml \
--endpoint http://localhost:PORT/engage \
--cases evals/engage-baseline.yaml \
--seal-on-pass
→ 186 cases · 558 executions
→ 153 strict pass · 33 characterization · 0 failures
→ elapsed 4m 12s · total cost cents 218.4
→ results written: runs/2026-11-04_engage-agent-v1/
→ sealed: runs/2026-11-04_engage-agent-v1.sealed
Sealed failing run, preserved next to sealed passing run
runs/ ├── 2026-05-13_agent-v0.sealed # FAILING, 66 cases, found 4 real bugs ├── 2026-05-20_agent-v1.sealed # PASSING, 186 cases, all v0 bugs fixed └── ...
What this proves
A methodology that found four real bugs in the author's own system and preserved the failing run as an artifact rather than deleting it. Cost-per-run is a first-class output. The framework is public, reusable, and points at any HTTP endpoint including endpoints outside Keystone. VER-FRAMEWORK-V1 REFERENCE PROFILES
The eval ledger, on the record.
Every claim on this page has an evidence chip. Here is the published eval ledger those chips point to. Numbers are from the sealed artifact for each entry, not from memory.
Publishing the failing run alongside the passing run is a discipline the contact center industry already built for compliance and quality management. Bad calls were not hidden; they were analyzed. Keystone applies the same discipline to model behavior. The agent-v0 sealed failing run is not an embarrassment. It is the evidence that the evaluation methodology finds real bugs.
Local-first infrastructure
Everything above runs on hardware I own. PostgreSQL 16 with pgvector for the data plane. A local open-weights LLM served via Ollama on a GPU node. NATS JetStream for messaging. OpenTelemetry with GenAI semantic conventions into a self-hosted Grafana Tempo instance. Two sites on a private network, one primary and one for disaster recovery plus the demo host. No customer data leaves the network, and there is no dependency on a cloud model provider for core operation. LOCAL-FIRST SELF-HOSTED
What I built.
Repo links do not tell a viewer who did the work. This section states it directly. I am Arnaldo Sepulveda. I designed and shipped the following, personally, over eighteen months of independent engineering alongside thirteen years at Genesys.
keystone-{component}/{type}-v{n}). The retrieval-v1 baseline. The agent-v0 failing run that surfaced four real bugs, sealed and preserved. The agent-v1 canonical run with 186 cases and 558 executions. The Engage 100/100 sealed baseline. The Counsel 30/30 sealed baseline.Why this matters for regulated enterprise AI.
Regulated buyers do not deploy AI systems that cannot demonstrate governance, authorization, evaluation, and locality. The five points below are the ones a serious evaluator will check.
Local-first is the deployment reality of regulated industries.
Healthcare, government, financial services, and telecom customers frequently require that customer data does not leave the network. Every Keystone component runs on local hardware. Local model serving via Ollama and vLLM. Local database. Local messaging. Local observability. This is the norm in the contact center industry and increasingly the norm in Canadian regulated deployment. A demo that cannot show this cannot ship in these environments.
Structural governance beats heuristic guardrails.
Guardrail models that classify prompts and responses are useful defense in depth. They are not the primary control. The primary control is structural: authorization checks that fail closed, tool scopes enforced before dispatch, human approval gates on irreversible actions, and tamper-evident audit records. Keystone's controls are structural. A prompt injection that manipulates the model still fails at the scope check, because the scope check does not consult the model.
Authorization enforced at the database, not in application code.
The ACL filter in Counsel is a WHERE clause in the pgvector query, not a post-retrieval filter in Python. Unauthorized rows never reach the application. A bug in the orchestrator, a prompt injection, or a hallucinated citation cannot leak content the caller is not permitted to see, because the query did not return it. This is the OWASP LLM06 (sensitive information disclosure) defense, implemented at the layer that matters.
Sealed failing runs are the evidence that evaluation is real.
A published eval that shows only passing runs is a marketing artifact. A published eval that preserves the failing run next to the passing run is a working evidence chain. The agent-v0 failing artifact surfaced four real system bugs. All four were fixed. The failing run is still there. This is contact-center quality management applied to model behavior, and it is the discipline that ISO/IEC 42001 and the NIST AI RMF measure function actually asks for.
Multi-agent substrate ready on day one, populated as the work matures.
The agents registry, the task state machine, the tempo dimension, and the cost fields all exist in v1 even though v1 runs one primary coordinator across five specialist phases. Adding cooperating agents at different tempos in v2 is a population change, not a schema migration. Most 2026 multi-agent systems will pay this cost twice: once to build, once to retrofit governance. Keystone paid it once, up front.
The contact center industry spent two decades solving governance, escalation, evidence, and quality management for conversational systems under regulatory pressure. The LLM era is rediscovering the need for that rigor from scratch. Keystone is not a rediscovery. It is a direct continuation of the discipline, applied to the new substrate. Thirteen years of Knowledge Center retrieval, chat suite, e-services, and contact intelligence at Genesys is the direct source.
Deeper inspection.
The evaluation framework and eval ledger are public and inspectable. Architecture and extension documentation is the first-class public surface at docs.getkeystone.ai. Private repo access is available on request for interview-depth technical review. Reach out for the interview conversation, a deep architecture walkthrough, or a live demo of any scenario above.
Public repositories
Documentation
Verify the evidence
# the evaluation framework — open, endpoint-agnostic git clone https://github.com/getkeystone/keystone-verify # the published eval ledger — sealed passing and failing artifacts git clone https://github.com/getkeystone/keystone-ledger # architecture & extension detail: docs.getkeystone.ai/ # private repo access for technical review: docs.getkeystone.ai/access/