---
name: powerbi-mcp-server-tester
description: Automated testing framework specifically for the PowerBI MCP server using MCP Inspector and Playwright. Use this skill when you need to test the PowerBI MCP server's functionality, verify Azure AD authentication, validate PowerBI tool implementations (workspaces, datasets, queries), or generate test reports. Only tests read-only operations and never executes destructive operations. Ensures POWERBI_TENANT_ID, POWERBI_CLIENT_ID, and POWERBI_CLIENT_SECRET environment variables are set before testing.
---

# PowerBI MCP Server Tester

Automated testing framework for the PowerBI MCP server that validates functionality, Azure AD authentication, and PowerBI tool implementations using MCP Inspector and Playwright automation.

## Prerequisites

Before testing, ensure:

1. **Environment Variables**: PowerBI credentials must be set in `.env` file
   - `POWERBI_TENANT_ID` - Azure AD tenant ID
   - `POWERBI_CLIENT_ID` - Application (client) ID
   - `POWERBI_CLIENT_SECRET` - Client secret value

2. **MCP Inspector**: Installed globally via npx (no manual installation needed)

3. **Playwright MCP**: Must be enabled as an MCP server

4. **PowerBI MCP Server**: Must be in the powerbi-mcp project directory with server code in `src/`

## Testing Workflow

### Step 1: Verify Prerequisites

Ask the user to confirm:
- "Please confirm your PowerBI environment variables (POWERBI_TENANT_ID, POWERBI_CLIENT_ID, POWERBI_CLIENT_SECRET) are set in your `.env` file"
- "Should I test all PowerBI tools or focus on specific ones (workspaces, datasets, queries)?"

### Step 2: Launch MCP Inspector

Start the MCP Inspector with the PowerBI MCP server (assumes you're in the powerbi-mcp directory):

```bash
# Background task - don't block on this
npx @modelcontextprotocol/inspector uv run run-server
```

Wait 5-10 seconds for Inspector to start, then read the output to get the Inspector URL.

### Step 3: Automated Testing with Playwright

Use Playwright MCP to:

1. **Navigate to Inspector**
   - Open the Inspector URL from the background task output

2. **Verify Connection**
   - Click "Connect" button
   - Wait for "Connected" status
   - Verify server name and version appear

3. **List Available Tools**
   - Click "List Tools" button
   - Count and record all available tools
   - Identify which tools are read-only (GET/list/read operations)

4. **Test Each Read-Only Tool**
   For each read-only tool:
   - Click on the tool to view its details
   - Review parameters and their types
   - If the tool has optional parameters, test with defaults first
   - If the tool has required parameters, use sample values or ask user for test values
   - Click "Run Tool" button
   - Wait for result (success or error)
   - Record the outcome

5. **Take Screenshot**
   - Capture screenshot of successful test results
   - Save as `mcp-test-results-<timestamp>.png`

6. **Close Browser**
   - Clean up Playwright session

### Step 4: Generate Test Report

Create a comprehensive test report including:

```markdown
# PowerBI MCP Server Test Report

**Server**: powerbi-mcp
**Date**: <timestamp>
**Tester**: Claude (powerbi-mcp-server-tester skill)

## Summary

- Total Tools: <count>
- Read-Only Tools Tested: <count>
- Tests Passed: <count>
- Tests Failed: <count>
- Success Rate: <percentage>%

## Connection Test

- ✅/❌ Server Started
- ✅/❌ Inspector Connected
- ✅/❌ Authentication Successful
- Server Version: <version>

## Tool Test Results

### <tool-name-1>

- **Status**: ✅ PASS / ❌ FAIL
- **Parameters Tested**: <list>
- **Response Time**: <ms>
- **Result**: <brief summary>
- **Notes**: <any observations>

### <tool-name-2>
...

## Issues Found

<List any errors, warnings, or unexpected behaviors>

## Recommendations

<Suggest improvements or areas needing attention>

## Test Artifacts

- Screenshot: `mcp-test-results-<timestamp>.png`
- Inspector Output: `<task-output-file>`
```

## Safety Guidelines

**CRITICAL - Read-Only Testing Only:**

- ✅ **DO Test**: `get_*`, `list_*`, `read_*`, `query_*`, `fetch_*`, `show_*`
- ❌ **NEVER Test**: `create_*`, `update_*`, `delete_*`, `remove_*`, `modify_*`, `write_*`, `post_*`, `put_*`, `patch_*`

Before running any tool:
1. Check the tool name for destructive keywords
2. Review the tool description for side effects
3. When in doubt, skip the tool and note it in the report

If a tool's safety is unclear, ask the user: "The tool `<name>` might be destructive. Should I test it? (I recommend skipping unless you're certain it's read-only)"

## Example Usage

**User**: "Test my PowerBI MCP server"

**Claude**:
1. Confirms environment variables are set
2. Launches Inspector with PowerBI server
3. Tests `get_workspaces`, `get_datasets`, `get_dataset` (skips `query_dataset` if it requires specific IDs)
4. Generates report showing all 4 tools, connection success, test results
5. Provides screenshot and recommendations

## Troubleshooting

**Inspector won't start:**
- Check server path is correct
- Verify all dependencies installed (`uv sync`)
- Review error output from background task

**Connection fails:**
- Verify environment variables are set correctly
- Check authentication credentials are valid
- Review server logs for auth errors

**Tools fail with "not found" errors:**
- May need specific IDs (workspace, dataset, etc.)
- Ask user for sample IDs to use in testing
- Skip tools that require production data

**Playwright errors:**
- Ensure Playwright MCP is enabled
- Check browser has time to load (add delays if needed)
- Review element selectors match current Inspector UI

## Advanced Features

### Custom Test Parameters

Ask user to provide test data:
```
"For testing get_dataset, I need a dataset ID. Can you provide one from your workspace?"
```

### Focused Testing

Test only specific tools:
```bash
# User specifies: "Only test the workspace tools"
# Claude tests: get_workspaces, list_workspaces, etc.
```

### Continuous Testing

For CI/CD integration, save report to file and return exit code based on success rate.

## Implementation Notes

- Use Bash tool with `run_in_background=true` for Inspector
- Use Playwright MCP for all browser automation
- Wait appropriate delays for async operations (5s for page load, 3s for tool execution)
- Handle errors gracefully and include in report
- Always close browser and stop background tasks when done
