Appendix D - Conformance Test Matrix

This appendix defines “conformance” as reproducible input, rule execution, and evidence judgment, rather than treating successful access to an online service on one occasion as complete proof. The test executor should record the test suite version, fixture version, version of the implementation under test, configuration summary, execution time, environment type, and redacted output; every case must produce a stable case ID, a PASS or FAIL conclusion, the actual error cause, and locatable evidence files. For rejection cases, the pass condition is not merely that the request fails. It must also show that the rejection occurred at the correct boundary, that no publication or index state that should not exist was written, and that the error code or reason conforms to the agreed contract. The currently directly reproducible offline evaluation is centered on oan-third-party-node-admission-test-kit/fixtures/v1-alpha; its case-matrix.md and requirements-traceability-matrix.md define the fixed fixtures, expected outputs, and checks for 49 requirements. This suite verifies the evaluation package and protocol judgment materials; it does not automatically amount to end-to-end proof against the official online Root, chain, content distribution platform, or off-chain trust indexer.

Every test record should contain at least the following fields:

Field Recording requirement
Case identifier Use the stable case ID in the manifest; do not use a temporary line number in a test file as the unique identifier.
Preconditions Record the fixture, protocol version, authorization-domain set, governance snapshot, simulated responses, and necessary service status.
Operation and input Record the called interface, HTTP method, request-body summary, query conditions, and signature or hash inputs; private keys, tokens, and database connection strings must be redacted.
Actual result Record the status code, normalized result, error code, lifecycle, version, cursor, and necessary response summary.
Pass criterion Compare the actual result item by item with the outcome, reasonCode, and normative clauses in the expected fixture.
Evidence artifacts Save the JSON report, Markdown report, references to the input and expected fixtures, version, and redaction rules.

Test failures fall into three categories: protocol or security failures that block publication, identity or authorization failures that block node admission, and operational or freshness failures that do not block publication but must trigger alerts and leave an audit trail. The test suite must not silently retry, automatically correct input, or ignore unknown fields merely to obtain a “pass.” If the implementation has an explicit compatibility mode, it should be marked separately in the report. On-chain events, real content distribution, and cross-node network timing that are not covered by the current V1-alpha fixture should be marked as uncovered, rather than treating offline pass results as evidence that they are covered.

Test execution results should distinguish “the test result is a pass” from “the object under test accepts the input.” For an input that is expected to be rejected, the service returning a rejection is the correct behavior of the object under test, so the test case itself should be recorded as PASS rather than counting the business rejection directly as a failure. The minimum record in a report may use the following structure:

{
  "caseId": "fixture.registrar.rejects-did-mismatch",
  "expectedOutcome": "invalid",
  "actualOutcome": "invalid",
  "verdict": "PASS",
  "reasonCode": "DID_ID_MISMATCH",
  "sideEffect": "no_resource_record_written",
  "evidence": ["evidence/fixture.registrar.rejects-did-mismatch.json"]
}

expectedOutcome describes the fixture expectation, actualOutcome describes the implementation’s actual judgment, and verdict describes whether the test passed; these three meanings must not be merged into an ambiguous status field. Write tests should also compare the number of resource records, publication-queue entries, and index entries before and after the request to prove that the rejection produced no subsequent side effects.

Different test levels have different conclusion scopes: offline fixtures check rules and report generation, node integration tests check HTTP routes, persistence, and calls between nodes, and end-to-end tests check the propagation of publication, distribution, discovery, and governance state. The report should record the level and must not merge offline rule checks, service health checks, and complete business acceptance into a single “system passed” conclusion.

D.1 Resource Registration Tests

Registration tests should use agent_service, skill, mcp_server, and tool_api as the primary four resource categories. Positive samples should simultaneously satisfy consistency among resourceDid, the DID Document id, DID Document metadata, ResourcePackage metadata, the VC subject, and hash fields, and should contain a verifiable subject-control proof, resource type, and authorization domain. Registrar interface-layer tests should cover /resources/register and its compatibility entry point /resources/submit, verifying shape validation, authorization-domain validation, submission to the Root platform, and the boundary at which the registration credential and Root platform response are returned. Negative samples should replace one fact at a time, such as a DID mismatch, VC subject mismatch, package-hash mismatch, missing packageProof, missing credential proof, inconsistent control challenge, incomplete metadata, or a suspended resource, so that one fixture does not contain multiple indistinguishable errors.

Test item Prerequisite data and steps Pass and rejection criteria Evidence and impact
Minimal positive registration Load registrar-resource.valid.json and submit a fixed ResourcePackage to the test Registrar. Accept; resource type, DID, VC, authorization domain, and hashes are all bound; the returned result must not claim a publication stage beyond the one actually reached. Request summary, response summary, resource record, and expected fixture; after passing, allow entry into subsequent publication tests.
Field and type consistency Use did-mismatch, credential-subject-mismatch, and metadata-incomplete separately. Reject with a stable reason; no valid resource record readable by a Discovery node may be written. Error response and before/after storage comparison; failure blocks registration publication.
Proof and hash binding Use tampered-proof, credential-hash-mismatch, package-hash-mismatch, and missing-proof fixtures. Reject; validation must cover the proof subject, issuer, DID Document hash, metadata hash, and package hash. Validation path and redacted proof summary; failure blocks publication.
Duplicate submission and update Submit the same DID and version twice, then submit an incremented version with a controller signature. Duplicate submission must not create two conflicting current versions; an update must pass control and version rules; historical and current versions must have distinguishable meanings. Two responses, resource records, version list, and retry log; an abnormal duplicate write blocks publication.

The current fixed fixtures directly cover positive registration and multiple rejection conditions. Real Registrar database idempotency, concurrent writes, and propagation of updates for the same DID still need to be executed in a running-service scenario and marked in the report as live integration coverage; offline fixtures cannot substitute for them.

The four resource categories should use the same set of binding checks while using their respective valid resourceType and DID semantic codes. Test data may be organized as follows, making it possible to reuse the same assertions without applying fields from one resource category incorrectly to another:

Resource type Example DID subject code Minimal business materials Key assertion
Agent service AG DID Document, service endpoint, capability tags Service endpoint and resource type are consistent with the DID Document
Skill package SK DID Document, package information, download or manifest address Package hash, version, and resource metadata are consistent
MCP service MC DID Document, MCP service endpoint, protocol binding Protocol binding and endpoint reference are consistent
Tool API TL DID Document, API endpoint, input/output description Tool/API type and interface description are consistent

The same negative case should change only one fact under test, such as replacing only the DID Document id while keeping the VC, hashes, authorization domain, and lifecycle unchanged. This allows the failure reason to point accurately to the corresponding clause and prevents the service from masking the target defect with an earlier validation failure.

D.2 Root Platform Publication Tests

Root platform publication tests should separate Registrar acceptance from Root platform acceptance, validation, and publication. The input contains the registration service node’s identity materials, the resource DID Document, the ResourcePackage, registration credential, subject-control proof, version, and hashes. The Root platform first validates node authorization and the request signature, then validates resource binding; only after all checks pass does it create a trusted Root platform publication proof and a publication fact available for distribution. Tests should check that a failure leaves no intermediate “publish first, validate later” state and that there is no path by which a Discovery node accepts a result merely because the Registrar returned success. For duplicate delivery, reprocessing must remain idempotent: the same publication fact must not generate a conflicting version, while changed content or proof must be judged again under version and control rules.

The current third-party admission test suite does not include the Root, chain, or real publication queue within the V1-alpha evaluation scope. Therefore, the offline part of this section can validate only publication input materials and expected rules; it cannot claim complete coverage of real Root platform publication. When running services, record the Root platform interface response, publication job ID, resource version, trusted Root platform publication proof summary, rejection reason, and reprocessing result. Logs containing private governance materials must not enter the public report.

The key assertion for Root platform publication tests is “validate first, then create a publishable fact.” They may be executed in the following order:

  1. Prepare the identity materials for an authorized Registrar node and a resource registration submission.
  2. Separately tamper with the node-to-node signature, resource control proof, DID Document hash, ResourcePackage hash, and authorization domain, and call POST /root/resources/verify-and-publish for each case.
  3. Check the HTTP result, machine-readable error, resource version, publication queue, and trusted Root platform publication proof for each rejection.
  4. Send the original valid request again and confirm that duplicate publication does not create a conflicting current version.

The storage and queue comparison in step 3 is necessary evidence; saving only the HTTP response cannot prove that the Root platform left no partial publication state for a failed request.

D.3 Distribution and Synchronization Tests

Distribution tests center on “the publication fact reaches an authorized node in a verifiable manner.” First establish a baseline with full synchronization, then perform incremental synchronization using a cursor, injecting duplicate events, out-of-order events, missing events, checkpoint recovery, and content-hash errors separately. A receiving node must not overwrite data based only on arrival time; it should decide whether to accept it according to the resource DID, version, publication proof, and cursor rules. Materials that are missing or have an incorrect hash should remain pending or failed and must not enter a visible index. Recovery tests should record the last successful cursor, retry count, backoff result, and final convergence state, distinguishing “transfer complete,” “local validation complete,” and “visible to discovery.” A content distribution platform cache hit or HTTP success can prove only a transport-layer result; it cannot replace the receiver’s validation of the trusted Root platform publication proof.

The boundary between this section and the current V1-alpha fixture should be written explicitly into the report: the existing test suite validates authorization domains, resource visibility, provenance, and fixed evidence rules, but its manifest explicitly excludes a real CDN, Root queue, and on-chain synchronization. After the implementation is connected to complete distribution tests, use repeatable resource versions and manually constructed cursors, and save four categories of evidence from the sender, receiver, validator, and indexer. Any failure that could cause an old version to become visible again, a revocation state to be lost, or an unvalidated resource to enter the index should be treated as a publication-blocking issue.

Distribution tests must at least verify that cursor advancement and resource acceptance are related but distinct results: a receiving node may successfully read one page of data while rejecting one resource in that page because of a proof or hash error. Whether the cursor advances and whether the failed item enters a retry queue must be recorded according to the established interface and implementation rules. Re-sending the same publication fact should be safely replayable; sending different content while reusing the same resource DID and version should enter a conflict-handling path and must not silently overwrite accepted content.

{
  "afterCursor": 1800,
  "nextCursor": 1812,
  "hasMore": true,
  "accepted": 11,
  "rejected": 1,
  "rejectedReasons": ["invalid_root_proof"]
}

The example above is an illustrative result-summary structure for a test report; the fields should follow the actual content distribution interface response. Advancement of nextCursor does not mean that every item has been validated and accepted. The report must also save the DID and reason for each rejected item.

D.4 Discovery and Semantic Search Tests

Discovery tests should cover exact DID queries for /discovery/resources/query, resource type, protocol, capability tags, authorization domain, and lifecycle filtering, as well as natural-language queries and the query-explanation interface, if enabled by the implementation. Test input first passes syntax, scope, and authorization-domain checks, then is converted into structured conditions, tag conditions, or semantic-retrieval conditions. Results must retain the resource DID, version, source, governance status, and necessary proof summary. Relevance scores, keyword hits, tag matches, and semantic similarity may be used only for ranking or explanation; they must not bypass authorization-domain checks, suspension/revocation filtering, DID Document validation, or trusted Root platform publication proof checks.

Query category Representative input Behavior that must be validated
Exact DID query A fixed resource DID or a nonexistent DID. Hit the unique resource or return a stable empty result; do not return another resource because of textual similarity.
Structured compound query A combination of resourceType, protocol, capabilityTags, limit, and authorization domain. Parse fields according to protocol rules; reject invalid enumerations, negative values, or values over the limit; pagination must neither duplicate nor omit accepted results.
Semantic query For example, Find an MCP server for security audit. Convert query text into explainable candidate conditions; returned results must identify semantic, tag, text, or other sources, and must not treat similarity as a trust proof.
Visibility boundary In-domain, out-of-domain, suspended, revoked, protocol-mismatched, and type-mismatched resources. Return only resources within the authorization domain and with an available status; an empty result is valid and must not be filled with recommendations.
Deduplication and explanation Multiple versions of the same DID, duplicate index records, and an explain request. Deduplicate according to DID and version rules, and explain candidate sources and filtering reasons.

fixtures/v1-alpha/discovery already provides fixed cases for visibility, empty results, arbitrary tag matching, invalid filters, invalid tags, invalid resource types, invalid protocols, limit boundaries, and overly short queries; these cases should be used as the regression baseline. The specific quality of semantic models, vector stores, and natural-language ranking must not be inferred from fixed-rule fixtures; record the model version, index build time, and explanation result separately.

Discovery-result tests should record candidate generation, filtering, ranking, and proof checking separately. Even if a resource is highly matched by text or capability tags, it must not appear in callable results if the authorization domain does not cover it, its lifecycle is unavailable, its DID Document is inconsistent with the ResourcePackage, or its trusted Root platform publication proof is invalid. Conversely, an empty result does not mean that the discovery service is faulty. It constitutes a recall or indexing defect only when a valid resource satisfying all filter conditions exists but the response is still empty.

Test stage Observed value Pass condition
Input parsing Query text, DID, resource type, tags, protocol, and limit Valid input enters the corresponding retrieval path; invalid input fails in a controlled manner
Candidate generation Candidate DIDs, match sources, and original scores Candidates come from locally indexed data; resources are not invented
Governance filtering Authorization domain, lifecycle, and publication-proof status Candidates outside the boundary are excluded
Result output DID, version, source, status, and proof summary Ranking does not change the trust conclusion; results are reviewable

D.5 Governance Authorization Tests

Governance tests must treat node authorization, resource control, and resource publication results as three independent judgments. A node may perform actions allowed by its role only when its node DID Document, authorization credential, authorization domain, protocol version, and lifecycle status are consistent. The resource owner’s control proof establishes only the right to operate the resource and cannot replace node authorization. Use active, suspended, revoked, unknown, and non-canonical case variants separately, and verify that sensitive operations reject or fail closed when the status cannot be confirmed or has become invalid. For authorization domains, cover the empty set, wildcard, parent and child domains, sibling domains, duplicates, case variations, and unknown domains, confirming that a tag tree or resource description does not enlarge the node’s authorization scope.

The governance, node, domains, and lifecycle fixtures provide stable inputs for offline judgment. The source, event sequence, or projection time in the governance snapshot must be saved together with the test report. Projection latency of the off-chain trust indexer, on-chain finality, and real revocation propagation belong to the running integration scope and cannot be claimed complete based only on snapshot tests.

Governance authorization tests should preferably use a closed-loop assertion of “authorization fact -> node action -> resource visibility.” For a revoked Discovery node, do not merely verify that the authorization snapshot status is revoked; also verify that its synchronization or query action is rejected, that its existing authorization scope does not continue to expand, and that already indexed resources do not become valid results again because of an old cache. For an authorization-domain change, check both the new and old domains to avoid validating only the added domain while missing whether the old permission was correctly withdrawn.

D.6 Version Update and Lifecycle Tests

Lifecycle tests center on state transitions and version relationships rather than using timestamp order as the sole basis. Establish an active resource, publish a new version with a valid proof from the original controller, and then verify the relationship among the current version, historical versions, and same-DID query results. Next apply suspension, revocation, and restoration facts separately, and check whether the statuses seen by the Registrar, Root platform, content distribution platform, Discovery node, and caller change within their allowed propagation boundaries. Updates without controller authorization, version rollback, same-version content conflicts, and restoration without a basis should all be rejected. Discovery queries should return only the available current version by default. If the interface supports historical queries, it must state the status of historical records clearly and must not allow an old version to bypass current governance status and become callable again.

The existing lifecycle fixture can validate the normative form of status values and active/inactive semantics. Same-DID updates, real historical persistence, and cross-node convergence should use service integration tests, retaining the DID Document hash, ResourcePackage hash, version, control proof, publication proof, and index time for every submission.

Records for a same-DID update should contain at least the following relationships:

same resource DID
  -> controller proof for the update
  -> strictly selected package version
  -> new DID document/package hashes
  -> Root publication evidence
  -> Discovery current-version result

Even when only the description or endpoint is updated, recompute the affected DID Document hash, metadata hash, or ResourcePackage hash and validate the corresponding proof; incrementing the version number cannot replace content binding. Historical-query tests should also confirm that historical versions are traceable but do not bypass the current suspension, revocation, or authorization status.

D.7 Security and Abuse-Prevention Tests

Security tests should validate “rejection with no side effects.” Fixed signature samples confirm that a valid Ed25519 signature can be verified and that the same payload after tampering cannot pass. At the service level, also replace the request nonce, request ID, audience, path, protocol version, and signer, and check replay, cross-interface reuse, and unauthorized calls. Request size, query limit, text length, invalid JSON, incorrect Content-Type, malicious endpoints, and abnormal authorization domains should be limited before expensive semantic retrieval, network requests, or persistence are entered. Rate-limit and timeout tests should record response status, wait time, resource consumption, and recovery, and must not use unlimited retries to hide rejection.

Sensitive-evidence tests must ensure that private keys, access tokens, database connection strings, internal file paths, and unredacted control materials do not enter reports, public website interfaces, or ordinary node responses. security/sensitive-evidence.json and report-redaction checks are the currently reproducible baseline. Real services should additionally sample application logs, reverse-proxy logs, and error responses to confirm that exceptional branches do not disclose internal stacks or secrets.

Side-effect checks in security tests should cover three locations: input buffers or temporary files before request processing, server-side persistent records, and downstream publication or index queues. For oversized requests, malicious endpoints, and invalid signatures, a passing result should include request limiting, no resource-record write, no node-to-node publication call, and no public error stack. For rate-limit tests, distinguish “the service rejects the request” from “the service process is unavailable”: the former is a policy result, while the latter is a stability defect.

D.8 Third-Party Node Admission Tests

Third-party node tests begin with node identity consistency, requiring no contradiction among configuration, DID Document, status response, authorization credential, role, endpoint, protocol version, lifecycle, and authorization domain. After admission, verify separately whether a Registrar node can accept and submit within its authorization domain, and whether a Discovery node can synchronize and query only Root platform publication content within its authorized scope. Also verify that invalid nodes, out-of-domain resources, protocol-version mismatches, and revoked status do not continue to receive permission because of configuration caches or historical credentials. Logging and failure-recovery tests should check that the request ID, cursor, rejection reason, and retry record are sufficient to reconstruct the event without containing private keys or internal secrets.

The current suite’s node-identity.*, authorization-domain, governance-snapshot, and discovery-visibility fixtures are suitable as offline admission gates for third-party nodes. Real node admission must additionally use isolated endpoints and an independent database; test nodes must not be connected directly to the official production data plane. The report should state the node under test, authorization domain, configuration version, and revocation/restoration times.

The admission conclusion for a third-party node should be split into five observation areas: identity, protocol, authorization, business, and operations. Consistent identity materials establish “who the node is”; successful protocol calls establish “whether the interface can communicate”; only a passing authorization and business loop establishes “whether the node can complete registration, synchronization, or discovery within its authorization scope”; logging, rate-limiting, and recovery evidence are used to judge whether the node can operate continuously. Any failed observation area should retain its specific failure reason in the report and must not be offset by another area passing.

D.9 DID Resolution and Method Conformance Tests

Resolution tests should validate the method prefix, semantic code, subject-type mapping, identifier character set, and normalization rules of did:oan; case correction, string parseability, or a server returning JSON must not be treated as proof of control. DID Document tests must check consistency among id, controllers, public keys, authentication/assertion references, service endpoints, resource type, and resource metadata, and must provide deterministic failure for an incorrect method, incorrect segment count, invalid characters, unknown subject code, or incorrect service reference. If the implementation provides historical-version resolution, also record the version, hash, and status used for resolution, distinguishing the current resolution result from a historical record.

Test evidence should contain identifiers before and after normalization, parsed fields, the verification method, and the error reason. Private keys may be used for signing at test runtime but must not be written to fixture reports.

Method conformance tests should also distinguish the syntax, structure, and control layers: the syntax layer confirms that the string has the did:oan form; the structure layer confirms that the DID Document’s id, verification methods, and service references are mutually consistent; the control layer requires generating a proof with the corresponding private key and verifying it with the public key. Passing only the first two layers cannot establish that a particular subject controls the resource. Fixed suffixes used in test vectors can reproduce resolution results but should not be interpreted as key-derivation results in a production environment.

D.10 API Compatibility and Error-Contract Tests

API tests should check successful and failed contracts at the same time. For each interface, record at least the method, path, Content-Type, required fields, handling of unknown fields, status code, error-body structure, error reason, pagination fields, and retry guidance. A client must not treat any 2xx response as business success, nor automatically interpret a network timeout as a resource rejection. For compatibility versions, distinguish ignorable extension fields, protocol versions that must be rejected, and differences in field order or JSON representation that must not change semantics. Retries apply only to explicitly retryable transport or temporary dependency failures. Requests carrying signatures and write side effects must use request ID, nonce, or server-side idempotency rules to prevent duplicate submission.

oan-protocol-common/docs/error-codes.md, protocol types, and the requests fixtures in the test suite are the main sources for checking the error contract. The current fixtures cover an empty request body, invalid method, malformed JSON, missing fields, oversized request, incorrect Content-Type, and incorrect query type. When a service adds an error code, it should update the common error documentation, client mapping, and expected fixture together.

Compatibility tests should assert request-delivery results and business-semantic results separately. An unknown ignorable field should not change the meaning of existing fields; an unknown protocol version should produce a clear incompatibility result; a missing required field should fail before business write; and a network timeout can be recorded only as an unknown transport result, not directly judged as registration failure. For interfaces with write side effects, retry tests must use a fixed requestId, nonce, or equivalent idempotency condition, and check whether the server generated duplicate VCs, duplicate versions, or duplicate publication tasks.

Scenario Expected test conclusion Evidence to save
Valid request and known version Business processing succeeds Status code, response summary, and resource status
Add an unknown extension field Process according to interface compatibility rules Original request and field-processing result
Missing required field Controlled rejection with no write side effect Error code and before/after storage counts
Retry after network timeout No duplicate business fact is created Two request identifiers, responses, and final record

D.11 Freshness, Convergence, and Recovery Tests

Freshness tests should record observation time, fact-production time, synchronization cursor, index-update time, and permitted maximum delay separately. The test flow may first publish a resource with a unique version and hash, confirm Root platform acceptance, observe the states of the content distribution platform, Discovery node, and off-chain trust indexer, and finally compare whether all layers converge on the same DID, version, status, and proof. After deliberately interrupting the network, restarting a service, deleting temporary transfer state, or injecting duplicate events, recovery must continue from a verifiable cursor or a full rebuild; it must not infer missing facts from “the latest response.” An old snapshot may be used for non-authoritative display or limited degradation, but authorization, revocation, and new-publication decisions must follow explicit freshness boundaries.

The report should list each layer’s observedAt, source, cursor, version, lifecycleState, validation result, and latency at a minimum. For results exceeding the threshold, mark whether the outcome blocks sensitive operations, degrades to read-only, or only produces an operational alert. The current V1-alpha suite does not provide a complete real-network convergence evaluation, so this part should be published separately from integration reports for the Root, CDN, Discovery node, and off-chain trust indexer.

Convergence tests should use a traceable resource version as the correlation key rather than comparing only the number of resources returned by each interface. The following minimum evidence chain may be retained in the report:

{
  "resourceDid": "did:oan:SKFI:7YpQm9Kx2VnRb6Ts3WfHa4Cd5Ej8LgNz",
  "version": "1.0.1",
  "rootPublishedAt": "2026-09-07T01:02:03Z",
  "cdnObservedAt": "2026-09-07T01:02:05Z",
  "discoveryObservedAt": "2026-09-07T01:02:07Z",
  "trustIndexObservedAt": "2026-09-07T01:02:08Z",
  "didDocumentHash": "sha256:example",
  "packageHash": "sha256:example",
  "converged": true
}

The times, DID, and digests in the example are for illustrating report fields only. Actual acceptance should judge converged according to the threshold configured for the test environment, while retaining raw responses, log correlation identifiers, and cursors from every layer. Only then can propagation delay, index failure, lagging governance state, and version-binding errors be distinguished.

References

Source Type Link
oan-protocol-common Code repository: test data structures, schemas, and error contract https://github.com/wolfbrother/oan-protocol-common
oan-root-services Code repository: Root platform validation and publication test objects https://github.com/OpenAgenet/oan-root-services
oan-registrar-node Code repository: registration protocol and validation test objects https://github.com/OpenAgenet/oan-registrar-node
oan-discovery-node Code repository: discovery, indexing, and semantic-query test objects https://github.com/OpenAgenet/oan-discovery-node
oan-trust-indexer Code repository: governance-state and synchronization test objects https://github.com/OpenAgenet/oan-trust-indexer
OAN Yellow Paper arXiv yellow paper: security properties and verification requirements https://arxiv.org/abs/2606.03163
Efficient Agent Discovery Profile IETF draft: testing directions for the discovery profile https://datatracker.ietf.org/doc/draft-xu-efficient-agent-discovery-profile/
On this page