From 5a0b99d1383500d52d72f13d0075e06543ddac99 Mon Sep 17 00:00:00 2001 From: Jacob Magar Date: Fri, 13 Mar 2026 17:26:36 -0400 Subject: [PATCH] docs: add non-destructive action smoke test script MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds scripts/test-actions.sh — a mcporter-based smoke test that exercises all 42 non-destructive MCP actions across 10 tools. Two-phase design: phase 1 runs param-free reads; phase 2 extracts resource IDs from those responses to test the ID-required reads (container details/logs, network details, disk details, log content, VM details, API key get). Also adds docs/test-actions.md with full usage, coverage table, skip rationale, and cleanup notes. Fixed three bugs discovered during test run: - Connectivity check used curl -f which treats 406 (correct MCP response to plain GET) as failure - run_test_capture wrote status lines to stdout causing captured $() to contain mixed text+JSON, breaking all Phase 2 ID extraction - run_test echoed full JSON response to terminal on every call Co-Authored-By: Claude --- docs/test-actions.md | 119 ++++++++++++ pyproject.toml | 2 +- scripts/test-actions.sh | 397 ++++++++++++++++++++++++++++++++++++++++ uv.lock | 2 +- 4 files changed, 518 insertions(+), 2 deletions(-) create mode 100644 docs/test-actions.md create mode 100755 scripts/test-actions.sh diff --git a/docs/test-actions.md b/docs/test-actions.md new file mode 100644 index 0000000..21efab9 --- /dev/null +++ b/docs/test-actions.md @@ -0,0 +1,119 @@ +# test-actions.sh + +Non-destructive smoke test for every readable action in the Unraid MCP server, +executed via `mcporter call`. + +## Usage + +```bash +# Against local server (default: http://localhost:6970/mcp) +./scripts/test-actions.sh + +# Against a specific server +./scripts/test-actions.sh http://10.1.0.2:6970/mcp + +# Via environment variable +UNRAID_MCP_URL=http://10.1.0.2:6970/mcp ./scripts/test-actions.sh +``` + +Exit code is `0` when all executed tests pass, `1` if any fail. + +## Prerequisites + +- `mcporter` on `$PATH` +- `python3` on `$PATH` (used for JSON result inspection and ID extraction) +- Unraid MCP server reachable at the target URL + +Start the server if needed: + +```bash +docker compose up -d # production container +uv run unraid-mcp-server # local dev +``` + +## What It Tests + +The script runs in two sequential phases. + +### Phase 1 — Param-free reads + +All actions that require no arguments beyond `action` itself. + +| Tool | Actions tested | +|---|---| +| `unraid_info` | `overview`, `array`, `network`, `registration`, `connect`, `variables`, `metrics`, `services`, `display`, `config`, `online`, `owner`, `settings`, `server`, `servers`, `flash`, `ups_devices`, `ups_device`, `ups_config` | +| `unraid_array` | `parity_status` | +| `unraid_storage` | `disks`, `shares`, `unassigned`, `log_files` | +| `unraid_docker` | `list`, `networks`, `port_conflicts`, `check_updates`, `sync_templates`, `refresh_digests` | +| `unraid_vm` | `list` | +| `unraid_notifications` | `overview`, `list`, `warnings`, `recalculate` | +| `unraid_rclone` | `list_remotes`, `config_form` | +| `unraid_users` | `me` | +| `unraid_keys` | `list` | +| `unraid_health` | `check`, `test_connection`, `diagnose` | + +### Phase 2 — ID-discovered reads + +The script extracts IDs from Phase 1 responses and uses them to test actions +that require a specific resource identifier. Each is skipped if Phase 1 returned +no matching resources. + +| Action | Source of ID | +|---|---| +| `docker: details` | first container from `docker: list` | +| `docker: logs` | first container from `docker: list` | +| `docker: network_details` | first network from `docker: networks` | +| `storage: disk_details` | first disk from `storage: disks` | +| `storage: logs` | first path from `storage: log_files` | +| `vm: details` | first VM from `vm: list` | +| `keys: get` | first key from `keys: list` | + +## What Is Skipped (and Why) + +Actions are skipped for one of three reasons shown in the output: + +| Label | Meaning | +|---|---| +| `destructive (confirm=True required)` | Action requires `confirm=True`; running it would permanently modify or delete data | +| `mutation — state-changing` | Action modifies live system state (container lifecycle, VM state, settings) | +| `mutation — creates …` | Action creates a new resource | + +### Full skip list + +**unraid_info**: `update_server`, `update_ssh` +**unraid_array**: `parity_start`, `parity_pause`, `parity_resume`, `parity_cancel` +**unraid_storage**: `flash_backup` +**unraid_docker**: `start`, `stop`, `restart`, `pause`, `unpause`, `update`, `remove`, `update_all`, `create_folder`, `set_folder_children`, `delete_entries`, `move_to_folder`, `move_to_position`, `rename_folder`, `create_folder_with_items`, `update_view_prefs`, `reset_template_mappings` +**unraid_vm**: `start`, `stop`, `pause`, `resume`, `reboot`, `force_stop`, `reset` +**unraid_notifications**: `create`, `create_unique`, `archive`, `unread`, `archive_all`, `archive_many`, `unarchive_many`, `unarchive_all`, `delete`, `delete_archived` +**unraid_rclone**: `create_remote`, `delete_remote` +**unraid_keys**: `create`, `update`, `delete` +**unraid_settings**: all 9 actions + +## Cleanup + +The script creates **no temporary files and no background processes**. There is +nothing to clean up on exit or interrupt. Status lines and error details go to +the terminal via stderr; captured JSON responses from `run_test_capture` live +only in local shell variables and are discarded when the script exits. + +`set -euo pipefail` ensures the script exits immediately on unexpected errors. +Arithmetic increments use `((var++)) || true` to safely handle the zero-to-one +transition without triggering the pipefail exit. + +## Output Format + +``` + PASS + FAIL + + SKIP (reason) +``` + +Summary line at the end: + +``` +Results: 42 passed 0 failed 37 skipped (79 total) +``` + +If any tests fail, the summary lists each failed label for quick triage. diff --git a/pyproject.toml b/pyproject.toml index 6bf9485..638b9bd 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -10,7 +10,7 @@ build-backend = "hatchling.build" # ============================================================================ [project] name = "unraid-mcp" -version = "0.4.2" +version = "0.4.3" description = "MCP Server for Unraid API - provides tools to interact with an Unraid server's GraphQL API" readme = "README.md" license = {file = "LICENSE"} diff --git a/scripts/test-actions.sh b/scripts/test-actions.sh new file mode 100755 index 0000000..0edc5fb --- /dev/null +++ b/scripts/test-actions.sh @@ -0,0 +1,397 @@ +#!/usr/bin/env bash +# test-actions.sh — Test all non-destructive Unraid MCP actions via mcporter +# +# Usage: +# ./scripts/test-actions.sh [MCP_URL] +# +# Default MCP_URL: http://localhost:6970/mcp +# Skips: destructive (confirm=True required), state-changing mutations, +# and actions requiring IDs not yet discovered. +# +# Phase 1: param-free reads +# Phase 2: ID-discovered reads (container, network, disk, vm, key, log) + +set -euo pipefail + +MCP_URL="${1:-${UNRAID_MCP_URL:-http://localhost:6970/mcp}}" + +# ── colours ────────────────────────────────────────────────────────────────── +RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m' +CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m' + +PASS=0; FAIL=0; SKIP=0 +declare -a FAILED_TESTS=() + +# ── helpers ─────────────────────────────────────────────────────────────────── + +mcall() { + # mcall + local tool="$1" args="$2" + mcporter call \ + --http-url "$MCP_URL" \ + --allow-http \ + --tool "$tool" \ + --args "$args" \ + --output json \ + 2>&1 +} + +_check_output() { + # Returns 0 if output looks like a successful JSON response, 1 otherwise. + local output="$1" exit_code="$2" + [[ $exit_code -ne 0 ]] && return 1 + echo "$output" | python3 -c " +import json, sys +try: + d = json.load(sys.stdin) + if isinstance(d, dict) and (d.get('isError') or d.get('error') or 'ToolError' in str(d)): + sys.exit(1) +except Exception: + pass +sys.exit(0) +" 2>/dev/null +} + +run_test() { + # Print result; do NOT echo the JSON body (kept quiet for readability). + local label="$1" tool="$2" args="$3" + printf " %-60s" "$label" + local output exit_code=0 + output=$(mcall "$tool" "$args" 2>&1) || exit_code=$? + if _check_output "$output" "$exit_code"; then + echo -e "${GREEN}PASS${NC}" + ((PASS++)) || true + else + echo -e "${RED}FAIL${NC}" + ((FAIL++)) || true + FAILED_TESTS+=("$label") + # Show first 3 lines of error detail, indented + echo "$output" | head -3 | sed 's/^/ /' + fi +} + +run_test_capture() { + # Like run_test but echoes raw JSON to stdout for ID extraction by caller. + # Status lines go to stderr so the caller's $() captures only clean JSON. + local label="$1" tool="$2" args="$3" + local output exit_code=0 + printf " %-60s" "$label" >&2 + output=$(mcall "$tool" "$args" 2>&1) || exit_code=$? + if _check_output "$output" "$exit_code"; then + echo -e "${GREEN}PASS${NC}" >&2 + ((PASS++)) || true + else + echo -e "${RED}FAIL${NC}" >&2 + ((FAIL++)) || true + FAILED_TESTS+=("$label") + echo "$output" | head -3 | sed 's/^/ /' >&2 + fi + echo "$output" # pure JSON → captured by caller's $() +} + +skip_test() { + local label="$1" reason="$2" + printf " %-60s${YELLOW}SKIP${NC} (%s)\n" "$label" "$reason" + ((SKIP++)) || true +} + +section() { + echo "" + echo -e "${CYAN}${BOLD}━━━ $1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +# ── connectivity check ──────────────────────────────────────────────────────── + +echo "" +echo -e "${BOLD}Unraid MCP Non-Destructive Action Test Suite${NC}" +echo -e "Server: ${CYAN}$MCP_URL${NC}" +echo "" +printf "Checking connectivity... " +# Use -s (silent) without -f: a 4xx/406 means the MCP server is up and +# responding correctly to a plain GET — only "connection refused" is fatal. +HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "$MCP_URL" 2>/dev/null || echo "000") +if [[ "$HTTP_CODE" == "000" ]]; then + echo -e "${RED}UNREACHABLE${NC}" + echo "Start the server first: docker compose up -d OR uv run unraid-mcp-server" + exit 1 +fi +echo -e "${GREEN}OK${NC} (HTTP $HTTP_CODE)" + +# ═══════════════════════════════════════════════════════════════════════════════ +# PHASE 1 — Param-free read actions +# ═══════════════════════════════════════════════════════════════════════════════ + +section "unraid_info (19 query actions)" +run_test "info: overview" unraid_info '{"action":"overview"}' +run_test "info: array" unraid_info '{"action":"array"}' +run_test "info: network" unraid_info '{"action":"network"}' +run_test "info: registration" unraid_info '{"action":"registration"}' +run_test "info: connect" unraid_info '{"action":"connect"}' +run_test "info: variables" unraid_info '{"action":"variables"}' +run_test "info: metrics" unraid_info '{"action":"metrics"}' +run_test "info: services" unraid_info '{"action":"services"}' +run_test "info: display" unraid_info '{"action":"display"}' +run_test "info: config" unraid_info '{"action":"config"}' +run_test "info: online" unraid_info '{"action":"online"}' +run_test "info: owner" unraid_info '{"action":"owner"}' +run_test "info: settings" unraid_info '{"action":"settings"}' +run_test "info: server" unraid_info '{"action":"server"}' +run_test "info: servers" unraid_info '{"action":"servers"}' +run_test "info: flash" unraid_info '{"action":"flash"}' +run_test "info: ups_devices" unraid_info '{"action":"ups_devices"}' +run_test "info: ups_device" unraid_info '{"action":"ups_device"}' +run_test "info: ups_config" unraid_info '{"action":"ups_config"}' +skip_test "info: update_server" "mutation — state-changing" +skip_test "info: update_ssh" "mutation — state-changing" + +section "unraid_array" +run_test "array: parity_status" unraid_array '{"action":"parity_status"}' +skip_test "array: parity_start" "mutation — starts parity check" +skip_test "array: parity_pause" "mutation — pauses parity check" +skip_test "array: parity_resume" "mutation — resumes parity check" +skip_test "array: parity_cancel" "mutation — cancels parity check" + +section "unraid_storage (param-free reads)" +STORAGE_DISKS=$(run_test_capture "storage: disks" unraid_storage '{"action":"disks"}') +run_test "storage: shares" unraid_storage '{"action":"shares"}' +run_test "storage: unassigned" unraid_storage '{"action":"unassigned"}' +LOG_FILES=$(run_test_capture "storage: log_files" unraid_storage '{"action":"log_files"}') +skip_test "storage: flash_backup" "destructive (confirm=True required)" + +section "unraid_docker (param-free reads)" +DOCKER_LIST=$(run_test_capture "docker: list" unraid_docker '{"action":"list"}') +DOCKER_NETS=$(run_test_capture "docker: networks" unraid_docker '{"action":"networks"}') +run_test "docker: port_conflicts" unraid_docker '{"action":"port_conflicts"}' +run_test "docker: check_updates" unraid_docker '{"action":"check_updates"}' +run_test "docker: sync_templates" unraid_docker '{"action":"sync_templates"}' +run_test "docker: refresh_digests" unraid_docker '{"action":"refresh_digests"}' +skip_test "docker: start" "mutation — changes container state" +skip_test "docker: stop" "mutation — changes container state" +skip_test "docker: restart" "mutation — changes container state" +skip_test "docker: pause" "mutation — changes container state" +skip_test "docker: unpause" "mutation — changes container state" +skip_test "docker: update" "mutation — updates container image" +skip_test "docker: remove" "destructive (confirm=True required)" +skip_test "docker: update_all" "destructive (confirm=True required)" +skip_test "docker: create_folder" "mutation — changes organizer state" +skip_test "docker: set_folder_children" "mutation — changes organizer state" +skip_test "docker: delete_entries" "destructive (confirm=True required)" +skip_test "docker: move_to_folder" "mutation — changes organizer state" +skip_test "docker: move_to_position" "mutation — changes organizer state" +skip_test "docker: rename_folder" "mutation — changes organizer state" +skip_test "docker: create_folder_with_items" "mutation — changes organizer state" +skip_test "docker: update_view_prefs" "mutation — changes organizer state" +skip_test "docker: reset_template_mappings" "destructive (confirm=True required)" + +section "unraid_vm (param-free reads)" +VM_LIST=$(run_test_capture "vm: list" unraid_vm '{"action":"list"}') +skip_test "vm: start" "mutation — changes VM state" +skip_test "vm: stop" "mutation — changes VM state" +skip_test "vm: pause" "mutation — changes VM state" +skip_test "vm: resume" "mutation — changes VM state" +skip_test "vm: reboot" "mutation — changes VM state" +skip_test "vm: force_stop" "destructive (confirm=True required)" +skip_test "vm: reset" "destructive (confirm=True required)" + +section "unraid_notifications" +run_test "notifications: overview" unraid_notifications '{"action":"overview"}' +run_test "notifications: list" unraid_notifications '{"action":"list"}' +run_test "notifications: warnings" unraid_notifications '{"action":"warnings"}' +run_test "notifications: recalculate" unraid_notifications '{"action":"recalculate"}' +skip_test "notifications: create" "mutation — creates notification" +skip_test "notifications: create_unique" "mutation — creates notification" +skip_test "notifications: archive" "mutation — changes notification state" +skip_test "notifications: unread" "mutation — changes notification state" +skip_test "notifications: archive_all" "mutation — changes notification state" +skip_test "notifications: archive_many" "mutation — changes notification state" +skip_test "notifications: unarchive_many" "mutation — changes notification state" +skip_test "notifications: unarchive_all" "mutation — changes notification state" +skip_test "notifications: delete" "destructive (confirm=True required)" +skip_test "notifications: delete_archived" "destructive (confirm=True required)" + +section "unraid_rclone" +run_test "rclone: list_remotes" unraid_rclone '{"action":"list_remotes"}' +run_test "rclone: config_form" unraid_rclone '{"action":"config_form"}' +skip_test "rclone: create_remote" "mutation — creates remote" +skip_test "rclone: delete_remote" "destructive (confirm=True required)" + +section "unraid_users" +run_test "users: me" unraid_users '{"action":"me"}' + +section "unraid_keys" +KEYS_LIST=$(run_test_capture "keys: list" unraid_keys '{"action":"list"}') +skip_test "keys: create" "mutation — creates API key" +skip_test "keys: update" "mutation — modifies API key" +skip_test "keys: delete" "destructive (confirm=True required)" + +section "unraid_health" +run_test "health: check" unraid_health '{"action":"check"}' +run_test "health: test_connection" unraid_health '{"action":"test_connection"}' +run_test "health: diagnose" unraid_health '{"action":"diagnose"}' + +section "unraid_settings (all mutations — skipped)" +skip_test "settings: update" "mutation — modifies settings" +skip_test "settings: update_temperature" "mutation — modifies settings" +skip_test "settings: update_time" "mutation — modifies settings" +skip_test "settings: configure_ups" "destructive (confirm=True required)" +skip_test "settings: update_api" "mutation — modifies settings" +skip_test "settings: connect_sign_in" "mutation — authentication action" +skip_test "settings: connect_sign_out" "mutation — authentication action" +skip_test "settings: setup_remote_access" "destructive (confirm=True required)" +skip_test "settings: enable_dynamic_remote_access" "destructive (confirm=True required)" + +# ═══════════════════════════════════════════════════════════════════════════════ +# PHASE 2 — ID-discovered read actions +# ═══════════════════════════════════════════════════════════════════════════════ + +section "Phase 2: ID-discovered reads" + +# ── docker container ID ─────────────────────────────────────────────────────── +CONTAINER_ID=$(echo "$DOCKER_LIST" | python3 -c " +import json, sys +try: + d = json.load(sys.stdin) + containers = d.get('containers') or d.get('data', {}).get('containers') or [] + if isinstance(containers, list) and containers: + c = containers[0] + cid = c.get('id') or c.get('names', [''])[0].lstrip('/') + if cid: + print(cid) +except Exception: + pass +" 2>/dev/null || true) + +if [[ -n "$CONTAINER_ID" ]]; then + run_test "docker: details (id=$CONTAINER_ID)" \ + unraid_docker "{\"action\":\"details\",\"container_id\":\"$CONTAINER_ID\"}" + run_test "docker: logs (id=$CONTAINER_ID)" \ + unraid_docker "{\"action\":\"logs\",\"container_id\":\"$CONTAINER_ID\",\"tail_lines\":20}" +else + skip_test "docker: details" "no containers found to discover ID" + skip_test "docker: logs" "no containers found to discover ID" +fi + +# ── docker network ID ───────────────────────────────────────────────────────── +NETWORK_ID=$(echo "$DOCKER_NETS" | python3 -c " +import json, sys +try: + d = json.load(sys.stdin) + nets = d.get('networks') or d.get('data', {}).get('networks') or [] + if isinstance(nets, list) and nets: + nid = nets[0].get('id') or nets[0].get('Id') + if nid: + print(nid) +except Exception: + pass +" 2>/dev/null || true) + +if [[ -n "$NETWORK_ID" ]]; then + run_test "docker: network_details (id=$NETWORK_ID)" \ + unraid_docker "{\"action\":\"network_details\",\"network_id\":\"$NETWORK_ID\"}" +else + skip_test "docker: network_details" "no networks found to discover ID" +fi + +# ── disk ID ─────────────────────────────────────────────────────────────────── +DISK_ID=$(echo "$STORAGE_DISKS" | python3 -c " +import json, sys +try: + d = json.load(sys.stdin) + disks = d.get('disks') or d.get('data', {}).get('disks') or [] + if isinstance(disks, list) and disks: + did = disks[0].get('id') or disks[0].get('device') + if did: + print(did) +except Exception: + pass +" 2>/dev/null || true) + +if [[ -n "$DISK_ID" ]]; then + run_test "storage: disk_details (id=$DISK_ID)" \ + unraid_storage "{\"action\":\"disk_details\",\"disk_id\":\"$DISK_ID\"}" +else + skip_test "storage: disk_details" "no disks found to discover ID" +fi + +# ── log path ────────────────────────────────────────────────────────────────── +LOG_PATH=$(echo "$LOG_FILES" | python3 -c " +import json, sys +try: + d = json.load(sys.stdin) + files = d.get('log_files') or d.get('files') or d.get('data', {}).get('log_files') or [] + if isinstance(files, list) and files: + p = files[0].get('path') or (files[0] if isinstance(files[0], str) else None) + if p: + print(p) +except Exception: + pass +" 2>/dev/null || true) + +if [[ -n "$LOG_PATH" ]]; then + run_test "storage: logs (path=$LOG_PATH)" \ + unraid_storage "{\"action\":\"logs\",\"log_path\":\"$LOG_PATH\",\"tail_lines\":20}" +else + skip_test "storage: logs" "no log files found to discover path" +fi + +# ── VM ID ───────────────────────────────────────────────────────────────────── +VM_ID=$(echo "$VM_LIST" | python3 -c " +import json, sys +try: + d = json.load(sys.stdin) + vms = d.get('vms') or d.get('data', {}).get('vms') or [] + if isinstance(vms, list) and vms: + vid = vms[0].get('uuid') or vms[0].get('id') or vms[0].get('name') + if vid: + print(vid) +except Exception: + pass +" 2>/dev/null || true) + +if [[ -n "$VM_ID" ]]; then + run_test "vm: details (id=$VM_ID)" \ + unraid_vm "{\"action\":\"details\",\"vm_id\":\"$VM_ID\"}" +else + skip_test "vm: details" "no VMs found to discover ID" +fi + +# ── API key ID ──────────────────────────────────────────────────────────────── +KEY_ID=$(echo "$KEYS_LIST" | python3 -c " +import json, sys +try: + d = json.load(sys.stdin) + keys = d.get('keys') or d.get('apiKeys') or d.get('data', {}).get('keys') or [] + if isinstance(keys, list) and keys: + kid = keys[0].get('id') + if kid: + print(kid) +except Exception: + pass +" 2>/dev/null || true) + +if [[ -n "$KEY_ID" ]]; then + run_test "keys: get (id=$KEY_ID)" \ + unraid_keys "{\"action\":\"get\",\"key_id\":\"$KEY_ID\"}" +else + skip_test "keys: get" "no API keys found to discover ID" +fi + +# ═══════════════════════════════════════════════════════════════════════════════ +# SUMMARY +# ═══════════════════════════════════════════════════════════════════════════════ + +TOTAL=$((PASS + FAIL + SKIP)) +echo "" +echo -e "${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +echo -e "${BOLD}Results: ${GREEN}${PASS} passed${NC} ${RED}${FAIL} failed${NC} ${YELLOW}${SKIP} skipped${NC} (${TOTAL} total)" + +if [[ ${#FAILED_TESTS[@]} -gt 0 ]]; then + echo "" + echo -e "${RED}${BOLD}Failed tests:${NC}" + for t in "${FAILED_TESTS[@]}"; do + echo -e " ${RED}✗${NC} $t" + done +fi + +echo "" +[[ $FAIL -eq 0 ]] && exit 0 || exit 1 diff --git a/uv.lock b/uv.lock index 1f9a2cc..b85bfb6 100644 --- a/uv.lock +++ b/uv.lock @@ -1729,7 +1729,7 @@ wheels = [ [[package]] name = "unraid-mcp" -version = "0.4.1" +version = "0.4.2" source = { editable = "." } dependencies = [ { name = "fastapi" },