Skip to main content

​
What is Compliance Testing?

Compliance testing verifies that your MCP server adheres to the MCP protocol specification. It checks handshake procedures, capability negotiations, tool definitions, and error handling patterns to ensure your server works correctly with any MCP client.

​
Key Benefits

  • Protocol conformance - Ensure your server follows MCP standards
  • Interoperability - Verify compatibility with different MCP clients
  • Specification coverage - Test all required protocol features
  • Error handling validation - Check proper error responses and edge cases

​
How It Works

​
Configuration

​
Basic Compliance Test

{
  "suite_id": "mcp_compliance_check",
  "name": "MCP Protocol Compliance",
  "suite_type": "compliance",
  "strict_mode": true,
  "test_cases": [
    {
      "protocol_version": "2025-06-18",
      "check_categories": ["handshake", "capabilities", "tools"],
      "required_capabilities": ["tools", "resources"]
    }
  ]
}

​
Test Configuration Fields

FieldTypeDefaultDescription
protocol_versionstring”2025-06-18”MCP protocol version to validate against
check_categoriesarray[“handshake”, “capabilities”, “tools”, “resources”]Categories of compliance checks to run
required_capabilitiesarray[]Specific capabilities that must be present

​
Suite-Level Settings

SettingTypeDefaultDescription
strict_modebooleantrueEnforce strict protocol compliance

​
Check Categories

​
1. Handshake Testing ("handshake")

Validates the MCP connection establishment: 
{
  "check_categories": ["handshake"],
  "protocol_version": "2025-06-18"
}
Verifies:
  • Proper initialization sequence
  • Protocol version negotiation
  • Client/server capability exchange
  • Error handling during setup

​
2. Capability Discovery ("capabilities")

Tests capability advertisement and validation: 
{
  "check_categories": ["capabilities"],
  "required_capabilities": ["tools", "resources", "prompts"]
}
Verifies:
  • All advertised capabilities are functional
  • Required capabilities are present
  • Capability metadata is properly formatted
  • Dynamic capability updates work correctly

​
3. Tool Validation ("tools")

Comprehensive tool testing: 
{
  "check_categories": ["tools"],
  "required_capabilities": ["tools"]
}
Verifies:
  • All declared tools can be invoked
  • Tool schemas match actual parameters
  • Tool responses follow expected formats
  • Error handling for invalid tool calls

​
4. Resource Testing ("resources")

Resource access and management: 
{
  "check_categories": ["resources"],
  "required_capabilities": ["resources"]
}
Verifies:
  • Resource enumeration works correctly
  • Resource content can be retrieved
  • Permission handling is implemented
  • Resource URIs are properly formatted

​
5. Advanced Features ("advanced")

Advanced MCP protocol features: 
{
  "check_categories": ["advanced"]
}
Verifies:
  • Sampling support (server requesting LLM completions)
  • Elicitation support (server requesting user input)
  • Notification support
  • Error handling for edge cases

​
Strict vs Permissive Mode

​
Strict Mode ("strict_mode": true)

  • Enforces exact specification compliance
  • Fails on any protocol deviations
  • Requires all standard capabilities
  • Validates message format precisely
Use when:
  • Building production MCP servers
  • Need guaranteed interoperability
  • Preparing for deployment
  • Testing against reference implementations

​
Permissive Mode ("strict_mode": false)

  • Allows minor protocol variations
  • Warns but doesn’t fail on non-critical issues
  • Accepts partial capability implementations
  • More forgiving of format variations
Use when:
  • Developing and debugging servers
  • Testing experimental features
  • Working with legacy implementations
  • Gradual migration to full compliance

​
Protocol Version Testing

​
Current Version Compliance

{
  "protocol_version": "2025-06-18",
  "check_categories": ["handshake", "capabilities", "tools", "resources"],
  "strict_mode": true
}

​
Version-Specific Checks

Different protocol versions may have different requirements: 
{
  "suite_type": "compliance",
  "test_cases": [
    {
      "protocol_version": "2024-11-05",
      "check_categories": ["handshake", "tools"]
    },
    {
      "protocol_version": "2025-06-18",
      "check_categories": ["handshake", "tools", "resources", "streaming"]
    }
  ]
}

​
Example Compliance Configurations

​
Basic Server Validation

{
  "suite_id": "basic_compliance",
  "name": "Essential MCP Compliance",
  "suite_type": "compliance",
  "strict_mode": false,
  "test_cases": [
    {
      "protocol_version": "2025-06-18",
      "check_categories": ["handshake", "capabilities"],
      "required_capabilities": ["tools"]
    }
  ]
}

​
Comprehensive Testing

{
  "suite_id": "full_compliance",
  "name": "Complete MCP Protocol Validation",
  "suite_type": "compliance",
  "strict_mode": true,
  "test_cases": [
    {
      "protocol_version": "2025-06-18",
      "check_categories": ["handshake", "capabilities", "tools", "resources"],
      "required_capabilities": ["tools", "resources", "prompts"]
    }
  ]
}

​
Production Readiness

{
  "suite_id": "production_compliance",
  "name": "Production MCP Server Validation",
  "suite_type": "compliance",
  "strict_mode": true,
  "test_cases": [
    {
      "protocol_version": "2025-06-18",
      "check_categories": ["handshake", "capabilities", "tools", "resources", "advanced"],
      "required_capabilities": ["tools", "resources"],
      "metadata": {
        "environment": "production",
        "criticality": "high"
      }
    }
  ]
}

​
Server Configuration

​
Server Config Format

{
  "url": "https://your-mcp-server.com/mcp",
  "name": "server_name"
}
Note: The type field was removed from server configurations for simplicity.

​
Running Compliance Tests

# Create compliance test suite
mcp-t create suite

# Select "compliance" when prompted for suite type

# Run compliance tests
mcp-t run compliance-suite-id server-id --verbose

​
Next Steps

  1. Run your first compliance test
  2. Learn about security testing for auth validation
  3. Explore conversational testing for user experience
  4. Understand server configuration for proper setup
⌘I