Quickstart Guide
Prerequisites
Before testing, you’ll need an MCP server to test against. You’ll also need:- Python 3.11 or higher
- An Anthropic API key (get one here)
- An OpenAI API key (get one here)
1
Install MCP Testing
2
Set Up API Keys
Export your API keys as environment variables:
3
Complete Interactive Onboarding
Run the interactive quickstart command:This command will:
Show Setup Guide
You’ll see a comprehensive guide with all available commands and workflows.Validate Your API Keys
The system checks that yourANTHROPIC_API_KEY is properly configured. If missing, you’ll see:Create Your First Configuration
The quickstart will prompt you to create:Server Configuration:- Server ID (easy to remember, e.g., “my-server”)
- Server name (descriptive name)
- Server URL (your MCP server endpoint)
- Suite ID (easy to remember, e.g., “basic-tests”)
- Suite name (descriptive name)
- Test cases with:
- Test ID
- User message (what the AI agent will say)
- Success criteria (what constitutes a pass)
Set Up Shell Completion
Optionally configure tab completion for the CLI commands.4
Run Your First Test
Now run the test suite you just created:
Next Steps
Explore More Commands
Understand Your Results
Each test includes:- Conversation transcript - Full dialogue between AI agent and your server
- Judge evaluation - LLM analysis of whether success criteria were met
- Tool calls - Which MCP tools were used during the conversation
- Confidence score - How confident the judge is in the verdict
Learn Core Concepts
Now that you’ve run your first test, learn more about:- Testing Philosophy - Why test MCP servers this way?
- Test Suites - How to structure comprehensive tests