apophis-fastify/docs/attic/root-history/FEEDBACK_PROTOCOL_CONFORMANCE_FROM_ARBITER.md

Feedback: Protocol Conformance and Bilingual Representation Testing

Status: Feedback from Arbiter integration work Date: 2026-04-27

Context

We have been extending APOPHIS across Arbiter route families successfully for resource-oriented APIs:

1. Billing routes
2. User directory routes
3. Device management routes

That work went well once we moved to explicit schemas, explicit x-ensures, and avoided schema helpers that hard-coded one response shape.

Where things got much harder was OAuth 2.1.

The issue is not that OAuth is "too complex to test". The issue is that OAuth is a protocol with:

1. multiple representations for the same endpoint
2. cross-step state transfer
3. redirects, cookies, and form-encoded requests
4. wire-level requirements that must remain spec-compliant by default

In Arbiter, OAuth endpoints must stay bilingual:

1. plain JSON by default for RFC compliance
2. LDF only when explicitly requested via Accept

Today, APOPHIS pushes us toward a single response shape per route contract. That works well for resource APIs, but it creates pressure to distort protocol endpoints just to make them fit the contract runner.

The key outcome we want is:

APOPHIS should let us test rich protocols without forcing us to change compliant production behavior.

What Already Works Well

These existing capabilities are the right building blocks:

1. request_headers(this), response_headers(this), cookies(this)
2. redirect_count(this), redirect_url(this).0, redirect_status(this).0
3. stateful testing
4. protocol extensions roadmap in docs/protocol-extensions-spec.md
5. outbound mocking and deterministic seeded execution

This feedback is not asking for a rewrite. It is asking for a thin layer that composes these pieces into a protocol-testing model.

Core Gap

APOPHIS currently fits best when a route has one canonical success body shape and one canonical error body shape.

OAuth 2.1 does not look like that:

1. POST /oauth/token is plain JSON by default
2. the same endpoint may also return LDF when Accept: application/ldf+json
3. GET /oauth/authorize often returns redirects instead of bodies
4. multi-step flows pass state via cookies, redirect query params, auth codes, refresh tokens, and headers

The problem is not only schema generation. The deeper problem is that APOPHIS lacks a first-class way to say:

1. run the same route under multiple negotiated representations
2. assert on the semantic payload independent of representation
3. capture values from one step and feed them into later steps
4. test a protocol scenario without replacing the route's default wire behavior

Recommended Changes

1. Add Representation-Aware Contracts

Routes need multiple contract variants for the same endpoint.

Example need:

1. default Accept: application/json -> plain OAuth JSON
2. explicit Accept: application/ldf+json -> LDF fragment wrapping the same semantic payload

Suggested direction:

Add a route-level annotation for negotiated variants, for example:

schema: {
  'x-variants': [
    {
      name: 'json',
      when: 'request_headers(this).accept == null || request_headers(this).accept matches /application\/json/',
      response: {
        200: { type: 'object', properties: { access_token: { type: 'string' } } },
        400: { type: 'object', properties: { error: { type: 'string' } } }
      },
      ensures: [
        'if status:200 then response_body(this).access_token != null else true'
      ]
    },
    {
      name: 'ldf',
      when: 'request_headers(this).accept matches /application\/(ldf\+json|vnd\.ldf\+json)/',
      response: {
        200: { type: 'object', properties: { type: { const: "LinkedDataFragment" }, fragment_type: { const: "Document" }, data: { type: 'object' } } }
      },
      ensures: [
        'if status:200 then response_body(this).fragment_type == "Document" else true'
      ]
    }
  ]
}

This would let one route remain spec-compliant by default while still being richly testable under negotiated formats.

2. Add a Semantic Payload Accessor

This is the smallest feature with the biggest payoff.

Today, formulas need to know whether the body is:

1. raw JSON: response_body(this).access_token
2. LDF: response_body(this).data.access_token

That is exactly the wrong abstraction boundary for bilingual endpoints.

Suggested addition:

1. response_payload(this)

Semantics:

1. if body is an LDF fragment with data, return body.data
2. otherwise return body

Then the same formula works for both representations:

if status:200 then response_payload(this).access_token != null else true
if status:400 then response_payload(this).error == "unsupported_grant_type" else true

Keep response_body(this) exactly as it is. response_payload(this) is the normalized semantic view.

This single feature would dramatically reduce contract duplication for negotiated responses.

3. Add Variant Execution to contract()

The test runner should be able to run the same route under multiple header sets.

Suggested shape:

await fastify.apophis.contract({
  depth: 'quick',
  routes: ['POST /oauth/token'],
  variants: [
    { name: 'json', headers: { accept: 'application/json' } },
    { name: 'ldf', headers: { accept: 'application/ldf+json' } }
  ]
})

This should:

1. reuse existing scope/header logic
2. report failures per route per variant
3. not require separate route registrations or test harnesses

This is much more useful than forcing a route to always return one representation.

4. Add a Protocol Scenario Runner

The docs currently say protocol state machines are out of scope and should use separate integration tests.

We think this boundary is too strict.

Not everything about OAuth needs to be declarative, but APOPHIS should still own the execution model for protocol scenarios.

What is needed is not a third giant testing engine. It is a thin scripted layer over the existing HTTP executor, formula evaluator, flake detection, state handling, and extensions.

Suggested API:

await fastify.apophis.scenario({
  name: 'oauth21.refresh_rotation',
  steps: [
    {
      name: 'login',
      request: {
        method: 'POST',
        url: '/end-user/login',
        body: { userKey: 'u1', password: 'pw' },
        headers: { accept: 'application/json' }
      },
      expect: [
        'status:200'
      ],
      capture: {
        session_cookie: 'response_headers(this)["set-cookie"]'
      }
    },
    {
      name: 'authorize',
      request: {
        method: 'GET',
        url: '/oauth/authorize?...',
        headers: {
          accept: 'text/html',
          cookie: '$login.session_cookie'
        }
      },
      expect: [
        'status:302',
        'redirect_count(this) == 1'
      ],
      capture: {
        code: 'redirect_query(this).0.code'
      }
    },
    {
      name: 'token',
      request: {
        method: 'POST',
        url: '/oauth/token',
        headers: {
          accept: 'application/json',
          'content-type': 'application/x-www-form-urlencoded'
        },
        form: {
          grant_type: 'authorization_code',
          code: '$authorize.code'
        }
      },
      expect: [
        'status:200',
        'response_payload(this).access_token != null'
      ],
      capture: {
        refresh_token: 'response_payload(this).refresh_token'
      }
    }
  ]
})

This would let APOPHIS test OAuth 2.1, device authorization, WIMSE S2S, transaction tokens, and similar protocol flows in a uniform system.

5. Add First-Class Capture/Rebind Support

Protocol testing needs more than previous().

We need first-class support for:

1. capturing from response body
2. capturing from response headers
3. capturing from cookies
4. capturing from redirect URLs
5. rebinding captured values into later request URLs, headers, query, body, and form fields

This is the difference between route testing and protocol testing.

Examples:

1. capture auth code from redirect query
2. capture refresh token from token response
3. capture session cookie from login response
4. capture request_uri from PAR response
5. reuse all of them in later steps

6. Add a Cookie Jar to Scenario and Stateful Execution

OAuth and browser-like flows depend on cookies persisting across requests.

Today APOPHIS can inspect cookies in formulas, but protocol scenarios need an actual cookie jar that automatically:

1. records Set-Cookie
2. applies matching cookies on subsequent requests
3. can still be overridden explicitly

Without this, login -> authorize -> consent flows remain awkward and externalized.

7. Add First-Class application/x-www-form-urlencoded Request Support

Token, PAR, revocation, introspection, and device flows rely heavily on form encoding.

APOPHIS should support request generation and scenario steps with:

1. form bodies
2. automatic content-type: application/x-www-form-urlencoded
3. schema-driven field generation for form posts

This should be a first-class capability, not a string-construction escape hatch.

8. Add Better Redirect Introspection Helpers

You already expose redirect count, status, and URL. That is close, but protocol testing needs one more step.

Suggested additions:

1. redirect_query(this).0.code
2. redirect_query(this).0.state
3. redirect_fragment(this).0.access_token

That would remove a lot of brittle URL parsing from tests.

9. Add Representation and Media-Type Predicates

Protocol routes often care as much about wire format as about semantic payload.

Suggested additions:

1. response_media_type(this)
2. request_media_type(this)
3. representation(this) returning values like json, ldf, html, redirect, empty

This enables formulas like:

if request_headers(this).accept matches /application\/ldf\+json/ then representation(this) == "ldf" else true
if status:302 then representation(this) == "redirect" else true

10. Add Protocol Packs Built on Top of the Above

Once the pieces above exist, APOPHIS could support reusable protocol packs without hardcoding protocol logic into core.

Examples:

1. oauth21ProfilePack()
2. rfc8628DeviceAuthorizationPack()
3. rfc8693TokenExchangePack()

These packs should be implemented as:

1. scenario definitions
2. invariant bundles
3. representation variants
4. extension requirements

That would let applications opt into rich conformance testing without rewriting bespoke harnesses.

Suggested Minimal Design

If you want the smallest possible cut that still unlocks this space, we recommend doing only these first:

1. response_payload(this)
2. contract({ variants: [...] })
3. scenario runner with capture/rebind
4. cookie jar in scenarios/stateful tests
5. form-urlencoded request support

Those five changes would already make OAuth 2.1 protocol testing meaningfully tractable.

Why This Matters

Without these features, APOPHIS is strongest on CRUD and hypermedia resources, but weak on standards conformance for real protocols.

That forces teams into a bad tradeoff:

1. either change production routes to fit APOPHIS better
2. or bypass APOPHIS for the most important protocol tests

The better outcome is:

1. production routes stay spec-compliant
2. APOPHIS understands negotiated representations
3. APOPHIS can execute and verify protocol flows directly

That would make APOPHIS useful not just for application contract testing, but for standards-grade protocol verification.

Concrete Arbiter Example

For Arbiter specifically, this would let us test OAuth routes in the correct way:

1. Accept: application/json -> verify plain RFC responses
2. Accept: application/ldf+json -> verify LDF/hypermedia responses
3. same semantic formulas via response_payload(this)
4. same route, same handler, same production behavior
5. cross-step protocol assertions for authorize -> token -> refresh -> revoke

That is the capability gap we hit.

Bottom Line

APOPHIS is already close.

It has most of the primitives. What it lacks is the protocol-testing composition layer.

If you add:

1. representation-aware contracts
2. semantic payload normalization
3. variant execution
4. scenario capture/rebind
5. cookie jar + form support

then rich OAuth 2.1 conformance testing becomes something APOPHIS can own directly instead of something that has to live in a separate bespoke harness.