179 lines
4.3 KiB
Markdown
179 lines
4.3 KiB
Markdown
# LLM-Safe Adoption
|
|
|
|
APOPHIS is designed to be safe and predictable for LLM-generated Fastify services.
|
|
|
|
## Why APOPHIS Is Good for LLM-Generated Services
|
|
|
|
Coding agents benefit from:
|
|
|
|
- **Constrained vocabulary**: Small set of CLI commands and config options
|
|
- **Official scaffolds**: Tested templates that produce valid config
|
|
- **Policy guards**: CI catches unsafe modes and malformed setup
|
|
- **Deterministic output**: Fixed seed, config, schemas, and deterministic handlers produce repeatable output
|
|
- **Behavioral contracts**: Agents write `x-ensures` clauses, APOPHIS verifies them
|
|
|
|
## Official Scaffolds
|
|
|
|
Use `apophis init` with a preset:
|
|
|
|
| Preset | Use Case |
|
|
|---|---|
|
|
| `safe-ci` | General CI-safe setup |
|
|
| `llm-safe` | Ultra-minimal for LLM-generated code |
|
|
| `platform-observe` | Observe-mode policy and runtime drift reporting |
|
|
| `protocol-lab` | Multi-step flows and stateful testing |
|
|
|
|
```bash
|
|
apophis init --preset llm-safe
|
|
```
|
|
|
|
## apophis doctor Checks
|
|
|
|
Run `apophis doctor` to validate your setup:
|
|
|
|
- **Dependencies**: Checks for `fastify`, `@fastify/swagger`
|
|
- **Config validation**: Rejects unknown keys, unsafe modes
|
|
- **Route discovery**: Confirms routes are discoverable
|
|
- **Safety checks**: Blocks qualify in production, missing sinks
|
|
- **Docs drift**: Validates examples in CI mode
|
|
|
|
```bash
|
|
apophis doctor
|
|
```
|
|
|
|
## CI Policy Guards
|
|
|
|
Add these checks to your CI pipeline:
|
|
|
|
```yaml
|
|
name: APOPHIS Checks
|
|
|
|
on: [push, pull_request]
|
|
|
|
jobs:
|
|
apophis:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '20'
|
|
cache: 'npm'
|
|
- run: npm ci
|
|
- run: npx apophis doctor
|
|
- run: npx apophis verify --profile ci --changed
|
|
```
|
|
|
|
## Template Examples
|
|
|
|
### Minimal LLM-Safe Config
|
|
|
|
```javascript
|
|
// apophis.config.js
|
|
export default {
|
|
mode: 'verify',
|
|
profile: 'llm-check',
|
|
profiles: {
|
|
'llm-check': {
|
|
name: 'llm-check',
|
|
mode: 'verify',
|
|
preset: 'llm-safe',
|
|
routes: []
|
|
}
|
|
},
|
|
presets: {
|
|
'llm-safe': {
|
|
name: 'llm-safe',
|
|
depth: 'quick',
|
|
timeout: 3000,
|
|
parallel: false,
|
|
chaos: false,
|
|
observe: false
|
|
}
|
|
},
|
|
environments: {
|
|
local: {
|
|
name: 'local',
|
|
allowVerify: true,
|
|
allowObserve: false,
|
|
allowQualify: false,
|
|
allowChaos: false,
|
|
allowBlocking: false,
|
|
requireSink: false
|
|
}
|
|
}
|
|
};
|
|
```
|
|
|
|
### Route Template with Behavioral Contract
|
|
|
|
```javascript
|
|
app.post('/users', {
|
|
schema: {
|
|
'x-category': 'constructor',
|
|
'x-ensures': [
|
|
// BEHAVIORAL: Creating a user must make it retrievable
|
|
'response_code(GET /users/{response_body(this).id}) == 200'
|
|
],
|
|
body: {
|
|
type: 'object',
|
|
properties: {
|
|
name: { type: 'string', minLength: 1 }
|
|
},
|
|
required: ['name']
|
|
},
|
|
response: {
|
|
201: {
|
|
type: 'object',
|
|
properties: {
|
|
id: { type: 'string' },
|
|
name: { type: 'string' }
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}, async (request, reply) => {
|
|
const { name } = request.body;
|
|
const id = `usr-${Date.now()}`;
|
|
reply.status(201);
|
|
return { id, name };
|
|
});
|
|
```
|
|
|
|
### CI Policy Guard Script
|
|
|
|
```javascript
|
|
// scripts/apophis-ci-guard.js
|
|
import { execSync } from 'node:child_process';
|
|
|
|
// Run doctor
|
|
const doctorResult = execSync('npx apophis doctor', { encoding: 'utf-8' });
|
|
console.log(doctorResult);
|
|
|
|
// Run verify
|
|
const verifyResult = execSync('npx apophis verify --profile ci --changed', { encoding: 'utf-8' });
|
|
console.log(verifyResult);
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
1. **Start with presets**: Avoid raw manual config until the project needs explicit overrides.
|
|
2. **Run doctor first**: Catch setup issues before running verify.
|
|
3. **Use `--changed` in CI**: Only verify routes that changed in the PR.
|
|
4. **Commit config**: Store `apophis.config.js` in version control.
|
|
5. **Pin versions**: Pin `apophis-fastify` version in `package.json`.
|
|
|
|
## Troubleshooting
|
|
|
|
### "Unknown config key"
|
|
|
|
APOPHIS rejects unknown keys to prevent hallucinated config. Check the key name against the config schema.
|
|
|
|
### "Qualify blocked in production"
|
|
|
|
Qualify mode is blocked in production by default. Use a non-production environment or explicitly allow it in your environment policy.
|
|
|
|
### "Missing sink config"
|
|
|
|
Observe mode requires a sink config in staging/production. Add `requireSink: true` to your environment policy and configure a sink.
|