docs: add paper citations, fix pedagogical issues, improve SKILL.md

- Cite arxiv 2602.23922 (Invariant-Driven Automated Testing) in all major docs
- Add Progressive Complexity section to SKILL.md for LLM guidance
- Fix SKILL.md Fast Start example to use deterministic ID generation
- Fix getting-started.md failure output inconsistency
- Fix auth-patterns.md TypeScript syntax in JS doc
- Fix fastify-structure.md Date.now() in test helper
- Fix observe.md misleading workspace heading
- Build: clean | Tests: 849 pass, 0 fail
This commit is contained in:
John Dvorak
2026-04-30 11:34:00 -07:00
parent 6c39bd0a6c
commit 8d7382417d
11 changed files with 53 additions and 6 deletions
+2
View File
@@ -4,6 +4,8 @@ Behavioral confidence for Fastify services.
APOPHIS checks whether route behavior holds across operations, states, and protocol flows. APOPHIS checks whether route behavior holds across operations, states, and protocol flows.
Inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): instead of only checking payload shape, APOPHIS encodes intended behavior as executable contracts and verifies them with property-based and stateful testing.
Supported Node.js versions: 20.x and 22.x. Supported Node.js versions: 20.x and 22.x.
```bash ```bash
+30 -1
View File
@@ -7,6 +7,8 @@ description: Use this skill when adding or improving APOPHIS contract-driven tes
APOPHIS verifies API behavior across operations, state changes, protocol flows, and dependencies. Use it when schema validation is not enough to answer whether an endpoint did the right thing. APOPHIS verifies API behavior across operations, state changes, protocol flows, and dependencies. Use it when schema validation is not enough to answer whether an endpoint did the right thing.
Inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): encode intended behavior as executable contracts, then verify them with property-based and stateful testing.
## When To Use ## When To Use
Use this skill when the operator asks to: Use this skill when the operator asks to:
@@ -76,6 +78,7 @@ When entering a Fastify codebase:
import Fastify from 'fastify' import Fastify from 'fastify'
import swagger from '@fastify/swagger' import swagger from '@fastify/swagger'
import apophis from 'apophis-fastify' import apophis from 'apophis-fastify'
import crypto from 'crypto'
const app = Fastify() const app = Fastify()
await app.register(swagger) await app.register(swagger)
@@ -114,8 +117,9 @@ app.post('/users', {
} }
} }
}, async (req, reply) => { }, async (req, reply) => {
const id = `usr-${crypto.createHash('sha256').update(req.body.email).digest('hex').slice(0, 8)}`
reply.status(201) reply.status(201)
return { id: 'usr-1', ...req.body } return { id, ...req.body }
}) })
await app.ready() await app.ready()
@@ -350,6 +354,31 @@ Operator framing:
> The failing seed gives us a reproducible behavioral example. I'll replay it first so we can distinguish a real regression from source drift or nondeterministic app state. > The failing seed gives us a reproducible behavioral example. I'll replay it first so we can distinguish a real regression from source drift or nondeterministic app state.
## Progressive Complexity
Start simple and add depth only where it pays off:
**Level 1 — Status and shape**: Every route gets an expected status code and key field existence.
```apostl
status:201
response_body(this).id != null
```
**Level 2 — Cross-route behavior**: Constructors check retrievability; mutators check persistence.
```apostl
response_code(GET /users/{response_body(this).id}) == 200
response_body(GET /users/{response_body(this).id}).email == request_body(this).email
```
**Level 3 — Isolation and boundaries**: Tenant, auth, and idempotency checks.
```apostl
if request_headers(this).x-tenant-id != null then response_headers(this).x-tenant-id == request_headers(this).x-tenant-id else true
```
**Level 4 — Protocol and dependency flows**: Variants, scenarios, outbound contracts, and chaos.
Add level 2 before level 4. Do not skip level 2 for resource APIs.
## Anti-Patterns ## Anti-Patterns
Do not: Do not:
+1 -1
View File
@@ -144,7 +144,7 @@ See `docs/protocol-extensions-spec.md` for full JWT extension configuration.
`getToken` runs per request. Handle refresh inline: `getToken` runs per request. Handle refresh inline:
```javascript ```javascript
let cachedToken: string | null = null let cachedToken = null
const auth = createAuthExtension({ const auth = createAuthExtension({
name: 'jwt-with-refresh', name: 'jwt-with-refresh',
+2
View File
@@ -2,6 +2,8 @@
Inject controlled failures into contract tests to validate resilience guarantees. Inject controlled failures into contract tests to validate resilience guarantees.
Chaos testing applies the invariant-driven verification approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) under adverse conditions: if a contract must hold, it should still hold when dependencies fail, responses are delayed, or payloads are corrupted.
## Usage ## Usage
```javascript ```javascript
+3 -1
View File
@@ -374,8 +374,10 @@ import { mkdirSync, rmSync } from 'fs'
import { tmpdir } from 'os' import { tmpdir } from 'os'
import { join } from 'path' import { join } from 'path'
let testCounter = 0
export function createTestWorkspace() { export function createTestWorkspace() {
const dir = join(tmpdir(), `apophis-test-${Date.now()}`) const dir = join(tmpdir(), `apophis-test-${++testCounter}`)
mkdirSync(dir, { recursive: true }) mkdirSync(dir, { recursive: true })
return { return {
+3 -1
View File
@@ -2,6 +2,8 @@
Get from install to your first behavioral bug in 10 minutes. Get from install to your first behavioral bug in 10 minutes.
APOPHIS is inspired by [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): instead of only validating request and response shape, encode intended behavior as executable contracts and let the tool find violations automatically.
## Prerequisites ## Prerequisites
- Node.js 20.x or 22.x - Node.js 20.x or 22.x
@@ -70,7 +72,7 @@ Expected
response_code(GET /users/{response_body(this).id}) == 200 response_code(GET /users/{response_body(this).id}) == 200
Observed Observed
GET /users/usr-123 returned 404 GET /users/usr-7d865e returned 404
Why this matters Why this matters
The resource created by POST /users is not retrievable. The resource created by POST /users is not retrievable.
+2
View File
@@ -2,6 +2,8 @@
APOPHIS is designed to be safe and predictable for LLM-generated Fastify services. APOPHIS is designed to be safe and predictable for LLM-generated Fastify services.
It applies the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) to LLM-assisted development: constrained vocabulary, deterministic replay, and executable contracts give coding agents a verifiable loop between generated changes and behavioral correctness.
## Why APOPHIS Is Good for LLM-Generated Services ## Why APOPHIS Is Good for LLM-Generated Services
Coding agents benefit from: Coding agents benefit from:
+4 -2
View File
@@ -2,6 +2,8 @@
Runtime visibility and drift detection without blocking by default. Runtime visibility and drift detection without blocking by default.
Observe extends the invariant framework from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) to production environments: contracts run continuously against live traffic to detect behavioral drift without affecting requests.
## What Observe Does ## What Observe Does
`apophis observe` validates your runtime observe configuration: `apophis observe` validates your runtime observe configuration:
@@ -153,9 +155,9 @@ observe: {
} }
``` ```
## Workspace Support ## Monorepo Validation
For monorepos, use `apophis doctor --workspace` to validate observe configuration across all workspace packages. For monorepos, use `apophis doctor --workspace` to validate observe configuration across all workspace packages. `observe` itself does not support `--workspace`; use `doctor` to check config in each package.
## Mode Mismatch ## Mode Mismatch
+2
View File
@@ -6,6 +6,8 @@
This specification defines protocol-specific extensions for APOPHIS, driven by the Arbiter team's requirements for testing OAuth 2.1, WIMSE S2S, Transaction Tokens (RFC 8693), SPIFFE/SPIRE, and related security protocols. This specification defines protocol-specific extensions for APOPHIS, driven by the Arbiter team's requirements for testing OAuth 2.1, WIMSE S2S, Transaction Tokens (RFC 8693), SPIFFE/SPIRE, and related security protocols.
APOPHIS is grounded in [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021). Protocol extensions add domain-specific predicates (JWT, X.509, SPIFFE) to the core invariant framework.
Arbiter maintains 58 protocol conformance test files covering 138 behaviors across 7 specifications. These extensions bridge the gap between declarative APOSTL contracts and the domain-specific predicates required for security protocol validation. Arbiter maintains 58 protocol conformance test files covering 138 behaviors across 7 specifications. These extensions bridge the gap between declarative APOSTL contracts and the domain-specific predicates required for security protocol validation.
### 1.1 Current Shipped vs Not-Shipped Snapshot ### 1.1 Current Shipped vs Not-Shipped Snapshot
+2
View File
@@ -2,6 +2,8 @@
Run scenario, stateful, and chaos checks against non-production Fastify services. Run scenario, stateful, and chaos checks against non-production Fastify services.
Qualify extends the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021) with multi-step protocol flows, stateful sequences, and controlled fault injection.
## What Qualify Does ## What Qualify Does
`apophis qualify` runs deeper testing than verify: `apophis qualify` runs deeper testing than verify:
+2
View File
@@ -2,6 +2,8 @@
Deterministic contract verification for CI and local development. Deterministic contract verification for CI and local development.
APOPHIS implements the invariant-driven approach from [Invariant-Driven Automated Testing](https://arxiv.org/abs/2602.23922) (Malhado Ribeiro, 2021): encode intended behavior as executable formulas, then verify them automatically with property-based generation and deterministic replay.
## When to Use It ## When to Use It
- **Local development**: Quick feedback on behavioral changes - **Local development**: Quick feedback on behavioral changes