chore: crush git history - reborn from consolidation on 2026-03-10
This commit is contained in:
+199
@@ -0,0 +1,199 @@
|
||||
# Verify Mode
|
||||
|
||||
Deterministic contract verification for CI and local development.
|
||||
|
||||
## What Verify Does
|
||||
|
||||
`apophis verify` runs behavioral contracts against your Fastify routes:
|
||||
|
||||
1. Discovers routes from your Fastify app
|
||||
2. Filters routes by profile config and CLI flags
|
||||
3. Generates test data from JSON Schema
|
||||
4. Executes routes and checks `x-ensures` contracts
|
||||
5. Reports pass/fail with deterministic seed and replay command
|
||||
|
||||
## When to Use It
|
||||
|
||||
- **Local development**: Quick feedback on behavioral changes
|
||||
- **CI pipelines**: Catch regressions before merge
|
||||
- **Pre-commit hooks**: Fast smoke verification
|
||||
|
||||
## Profile Examples
|
||||
|
||||
### Quick (local smoke)
|
||||
|
||||
```javascript
|
||||
profiles: {
|
||||
quick: {
|
||||
name: 'quick',
|
||||
mode: 'verify',
|
||||
preset: 'safe-ci',
|
||||
routes: ['POST /users']
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### CI (PR checks)
|
||||
|
||||
```javascript
|
||||
profiles: {
|
||||
ci: {
|
||||
name: 'ci',
|
||||
mode: 'verify',
|
||||
preset: 'safe-ci',
|
||||
routes: []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Run with: `apophis verify --profile ci --changed`
|
||||
|
||||
### Deep (nightly verification)
|
||||
|
||||
```javascript
|
||||
profiles: {
|
||||
deep: {
|
||||
name: 'deep',
|
||||
mode: 'verify',
|
||||
preset: 'safe-ci',
|
||||
routes: []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Route Filtering
|
||||
|
||||
Filter routes with the `--routes` flag:
|
||||
|
||||
```bash
|
||||
# Single route
|
||||
apophis verify --routes "POST /users"
|
||||
|
||||
# Multiple routes (comma-separated)
|
||||
apophis verify --routes "POST /users,PUT /users/:id"
|
||||
|
||||
# Wildcards
|
||||
apophis verify --routes "POST /users/*"
|
||||
|
||||
# All routes (empty or omitted)
|
||||
apophis verify --profile quick
|
||||
```
|
||||
|
||||
## `--changed` Flag
|
||||
|
||||
Run only routes modified in the current git branch:
|
||||
|
||||
```bash
|
||||
apophis verify --profile ci --changed
|
||||
```
|
||||
|
||||
If no routes changed, exits 0 with a message.
|
||||
|
||||
## Failure Output Format
|
||||
|
||||
When a contract fails, APOPHIS prints:
|
||||
|
||||
```text
|
||||
Contract violation
|
||||
POST /users
|
||||
Profile: quick
|
||||
Seed: 42
|
||||
|
||||
Expected
|
||||
response_code(GET /users/{response_body(this).id}) == 200
|
||||
|
||||
Observed
|
||||
GET /users/usr-123 returned 404
|
||||
|
||||
Why this matters
|
||||
The resource created by POST /users is not retrievable.
|
||||
|
||||
Replay
|
||||
apophis replay --artifact reports/apophis/failure-2026-04-28T12-30-22Z.json
|
||||
|
||||
Next
|
||||
Check the create/read consistency for POST /users and GET /users/{id}.
|
||||
```
|
||||
|
||||
## Replay Workflow
|
||||
|
||||
1. Copy the replay command from failure output
|
||||
2. Run it with the recorded route, seed, and artifact data; source or dependency drift can change the outcome
|
||||
3. Fix the bug in your handler
|
||||
4. Re-run verify to confirm the fix
|
||||
|
||||
```bash
|
||||
apophis replay --artifact reports/apophis/failure-2026-04-28T12-30-22Z.json
|
||||
```
|
||||
|
||||
## Machine Output for CI
|
||||
|
||||
Use concise formats to reduce log volume in large verify runs:
|
||||
|
||||
- `--format json-summary` — single JSON with summary, failures, warnings. Omits per-step traces.
|
||||
- `--format ndjson-summary` — three NDJSON lines: `run.started`, `run.summary`, `run.completed`.
|
||||
|
||||
### Filtering examples
|
||||
|
||||
```bash
|
||||
# Extract only failed routes from full ndjson
|
||||
apophis verify --profile quick --format ndjson | jq 'select(.type == "route.failed")'
|
||||
|
||||
# Write artifact to disk and parse the file instead of stdout
|
||||
apophis verify --profile quick --format json --artifact-dir reports/apophis
|
||||
```
|
||||
|
||||
## Exit Codes
|
||||
|
||||
| Code | Meaning |
|
||||
|---|---|
|
||||
| 0 | All contracts passed |
|
||||
| 1 | One or more behavioral contracts failed |
|
||||
| 2 | Config error or no routes matched |
|
||||
| 3 | Internal APOPHIS error |
|
||||
| 130 | Interrupted (SIGINT) |
|
||||
|
||||
## Config Example
|
||||
|
||||
```javascript
|
||||
// apophis.config.js
|
||||
export default {
|
||||
mode: 'verify',
|
||||
profile: 'quick',
|
||||
profiles: {
|
||||
quick: {
|
||||
name: 'quick',
|
||||
mode: 'verify',
|
||||
preset: 'safe-ci',
|
||||
routes: ['POST /users']
|
||||
},
|
||||
ci: {
|
||||
name: 'ci',
|
||||
mode: 'verify',
|
||||
preset: 'safe-ci',
|
||||
routes: []
|
||||
}
|
||||
},
|
||||
presets: {
|
||||
'safe-ci': {
|
||||
name: 'safe-ci',
|
||||
depth: 'quick',
|
||||
timeout: 5000,
|
||||
parallel: false,
|
||||
chaos: false,
|
||||
observe: false
|
||||
}
|
||||
},
|
||||
environments: {
|
||||
local: {
|
||||
name: 'local',
|
||||
allowVerify: true,
|
||||
allowObserve: true,
|
||||
allowQualify: false,
|
||||
allowChaos: false,
|
||||
allowBlocking: true,
|
||||
requireSink: false
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
Reference in New Issue
Block a user