Prepare the test boundary
Document tenant and asset mapping, coverage timezone, severity rules, runbook versions and permitted changes. Separate credentials by person or accountable service identity and grant only the access needed for the test. Define the emergency stop: who can suspend actions, revoke access and return the queue to the retained team. Do not begin if the buyer cannot execute that rollback.
Ten cases to sample
| Case | Required observation |
|---|---|
| N01. Routine permitted fix | Correct asset, approved action and ticket evidence |
| N02. Approval-required change | No unauthorized execution; approval trail |
| N03. Noisy recurring alert | Agreed suppression duration and retained visibility |
| N04. Duplicate signal | One coherent incident without losing evidence |
| N05. Backup failure | Job repair distinguished from restore verification |
| N06. Failed patch | Stop rule, rollback owner and business update |
| N07. Lost credential access | Detection, escalation and no false resolution |
| N08. Wrong tenant mapping | Access blocked and mapping corrected |
| N09. External-vendor fault | Evidence package and accepted vendor handoff |
| N10. Critical shift-change incident | Next owner accepts before prior shift exits |
Choose thresholds before the first case. A correct escalation can be a pass when the provider is not authorized to fix the issue. A fast unapproved change is a fail. Evaluate the five clocks separately using the SLA guide; acknowledgement and service recovery are measured separately.
For the backup case, the backup scope and restore log separates the selected point, verification mode, recovery approval and application acceptance. Keep an approval hold distinct from a failed technical action.
For patching, use the patch acceptance cases to retain offline devices, denied restarts, failed installations and application faults as separate results. The same blank pilot log can hold these authorized cases.
Pilot case log
Keep the first result, evidence, correction and retest together for each case. The blank log includes 10 suggested cases, the five timer labels and a separate scope-acceptance decision.
Use Word for an editable working copy with space for each observation and retest, or open Markdown in a text editor. Both contain the same blank cases. Complete your copy for your approved environment.
Document a usable result
Illustrative log entry: case N03, planned noisy alert, runbook version 2, signal at 22:00 UTC, ticket at 22:02, approved suppression until 22:30, reactivation verified at 22:31, buyer reviewer accepts. These times are invented to show the required evidence. Your actual log should include timestamps, identity, action, outcome, exception and reviewer decision.
Do not average away a material failure. Keep an unapproved change, tenant mismatch or missing rollback visible even if other cases pass. Repeat the failed case after correction and retain the failed case and its retest. A small pilot demonstrates only the sampled environment and period, not a universal service level.
Decide before expanding
Expand only when blocking failures are corrected, the responsibilities are understood and the buyer has an accepted runbook. Hold when evidence is missing; reject the proposed scope when a critical requirement cannot be met. Keep the original queue and access-revocation path available during transition. Compare candidates in the directory and send the same RFP so pilot results concern matched obligations.