This guide explains how to integrate the MCP conformance test suite into your language SDK repository. The conformance framework tests your MCP implementation against the protocol specification to ensure compatibility.

Install and run conformance tests:

# Client testing (framework starts a test server, runs your client against it)
npx @modelcontextprotocol/conformance client --command "your-client-command" --scenario initialize

# Server testing (your server must already be running)
npx @modelcontextprotocol/conformance server --url http://localhost:3000/mcp --scenario server-initialize

The framework starts a test server and spawns your client against it. Your client receives the server URL as its final command-line argument.

# Run a single scenario
npx @modelcontextprotocol/conformance client \
  --command "python tests/conformance/client.py" \
  --scenario initialize

# Run a suite of tests
npx @modelcontextprotocol/conformance client \
  --command "python tests/conformance/client.py" \
  --suite auth

Available client suites: all, core, extensions, auth, metadata, sep-835

Your client should:

1. Accept the server URL as its last argument
2. Read MCP_CONFORMANCE_SCENARIO env var to determine which scenario is being tested
3. Read MCP_CONFORMANCE_CONTEXT env var for scenario-specific data (e.g., OAuth credentials)

Your server must be running before invoking the conformance tool. The framework connects to it as an MCP client.

# Start your server first
your-server --port 3001 &

# Then run conformance tests
npx @modelcontextprotocol/conformance server \
  --url http://localhost:3001/mcp \
  --suite active

Available server suites: active (default), all, pending

Note: Server testing requires you to manage server lifecycle (start, health-check, cleanup) yourself.

The expected-failures feature lets your CI pass while you work on fixing known issues. It catches regressions by failing when:

• A previously passing test starts failing (regression)
• A previously failing test starts passing (stale baseline - remove the entry)

Create a YAML file (e.g., conformance-baseline.yml):

server:
  - tools-call-with-progress
  - resources-subscribe
client:
  - auth/client-credentials-jwt

npx @modelcontextprotocol/conformance server \
  --url http://localhost:3000/mcp \
  --expected-failures ./conformance-baseline.yml

The conformance repo provides a reusable GitHub Action that handles Node.js setup and conformance execution.

name: Conformance Tests
on: [push, pull_request]

jobs:
  conformance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up your SDK
        run: |
          # Your SDK setup (pip install, npm install, etc.)
          pip install -e .

      - uses: modelcontextprotocol/[email protected]
        with:
          mode: client
          command: 'python tests/conformance/client.py'
          suite: auth
          expected-failures: ./conformance-baseline.yml

name: Conformance Tests
on: [push, pull_request]

jobs:
  conformance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up and start server
        run: |
          pip install -e .
          python -m myserver --port 3001 &
          # Wait for server to be ready
          timeout 15 bash -c 'until curl -s http://localhost:3001/mcp; do sleep 0.5; done'

      - uses: modelcontextprotocol/[email protected]
        with:
          mode: server
          url: http://localhost:3001/mcp
          suite: active
          expected-failures: ./conformance-baseline.yml

See src/conformance/everything-client.ts in the TypeScript SDK for a reference implementation. The recommended pattern is a single client that routes behavior based on the scenario:

import os
import sys
import json

def main():
    server_url = sys.argv[-1]  # URL passed as last argument
    scenario = os.environ.get("MCP_CONFORMANCE_SCENARIO", "")
    context = json.loads(os.environ.get("MCP_CONFORMANCE_CONTEXT", "{}"))

    if scenario.startswith("auth/"):
        run_auth_scenario(server_url, scenario, context)
    else:
        run_default_scenario(server_url)

if __name__ == "__main__":
    main()

See src/conformance/everything-server.ts in the TypeScript SDK for a reference implementation that handles all server scenarios.

• Conformance README
• Design documentation
• TypeScript SDK conformance examples