Self-healing Tests with LLM — AI 自動修壞掉的 Selector

「跑 E2E 的時間 20%、修 selector 的時間 80%」是 QA 痛點 Top 3。Self-healing test 用 LLM 自動找出新 selector、把維護時間砍掉 70%。這篇給你完整工具地圖 + 自建方案。

關鍵：self-heal 後必須 alert + review、不能靜默修復、否則會掩蓋真 bug。

核心思路

1. 寫 wrapper 攔截 selector 失敗
2. 失敗時拿 DOM snapshot + 原 selector
3. 餵給 LLM「依照原 selector 意圖、在新 DOM 找替代」
4. 試新 selector
5. 成功 → 繼續 + 記 log
6. 失敗 → 報錯

實作範例

import { Page, Locator } from '@playwright/test';
import Anthropic from '@anthropic-ai/sdk';

const claude = new Anthropic();

async function smartLocator(page: Page, selector: string, hint?: string): Promise<Locator> {
  try {
    const loc = page.locator(selector);
    await loc.waitFor({ timeout: 3000 });
    return loc;
  } catch {
    // Selector 找不到 → 啟動 self-healing
    console.warn(`⚠️ Selector "${selector}" not found, trying AI...`);

    const html = await page.content();
    const truncated = html.slice(0, 30000);  // 控制 token

    const response = await claude.messages.create({
      model: 'claude-sonnet-4-6',
      max_tokens: 200,
      messages: [{
        role: 'user',
        content: `原 selector: ${selector}
${hint ? `用途: ${hint}` : ''}

下面是當前 HTML。請建議一個能找到「同一元素」的新 selector。只回 selector 字串、不要解釋。

HTML:
${truncated}`,
      }],
    });

    const newSelector = response.content[0].text.trim();
    console.log(`🤖 AI 建議新 selector: ${newSelector}`);

    // 記到 healing log
    await logHeal(selector, newSelector);

    return page.locator(newSelector);
  }
}

// 使用
test('login with self-healing', async ({ page }) => {
  await page.goto('/login');
  const loginBtn = await smartLocator(page, '#login-btn', '登入按鈕');
  await loginBtn.click();
});

healing log

import fs from 'fs';

async function logHeal(oldSel: string, newSel: string) {
  const log = {
    at: new Date().toISOString(),
    old_selector: oldSel,
    new_selector: newSel,
    file: __filename,
  };
  fs.appendFileSync('healing.log', JSON.stringify(log) + '\n');

  // 也發 Slack
  await fetch(process.env.SLACK_WEBHOOK!, {
    method: 'POST',
    body: JSON.stringify({
      text: `🤖 Self-heal: ${oldSel} → ${newSel}`,
    }),
  });
}

🤖 Self-heal triggered

File: tests/login.spec.ts
Old selector: #login-btn
New selector: button[data-testid="submit-login"]
Time: 2026-06-17 14:32

Action needed:
- Was this a planned UI change? → Update test
- Was this a regression? → Report bug

// pages/LoginPage.ts
export class LoginPage {
  constructor(private page: Page) {}

  // 用 smart locator wrapper
  get emailInput() { return smartLocator(this.page, '[name=email]', 'Email 輸入框'); }
  get passwordInput() { return smartLocator(this.page, '[name=password]', 'Password 輸入框'); }
  get submitBtn() { return smartLocator(this.page, 'button[type=submit]', '送出按鈕'); }

  async login(email: string, password: string) {
    await (await this.emailInput).fill(email);
    await (await this.passwordInput).fill(password);
    await (await this.submitBtn).click();
  }
}

POM 給結構、self-healing 給彈性 = 完美組合。

name: E2E with Self-Healing
on: [pull_request]

jobs:
  e2e:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npx playwright test
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

      # 收集 healing log
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: healing-log
          path: healing.log

      # 多於 X 次 heal → fail PR
      - if: always()
        run: |
          COUNT=$(wc -l < healing.log)
          if [ $COUNT -gt 10 ]; then
            echo "❌ Too many self-heals ($COUNT) — likely UI mass change"
            exit 1
          fi

成本控制

// 每次 heal 約 $0.02-0.05（Claude Sonnet）
// 100 個 E2E、每月 50 次 heal = $2.50/月 — 划算

// 但無 budget cap 會炸
const MAX_HEALS_PER_RUN = 20;
let healCount = 0;

async function smartLocator(...) {
  if (healCount >= MAX_HEALS_PER_RUN) {
    throw new Error(`Heal budget exceeded (${MAX_HEALS_PER_RUN})`);
  }
  healCount++;
  // ...
}

1. Self-healing 解的是 cosmetic 變化、不是業務 bug
2. 永遠 alert + log、不要靜默修復
3. POM + Self-healing = E2E 維護地獄解決組合
4. 設 budget cap、API 費用會爆
5. 金融 / 醫療要 audit log、別當救命神器

Self-healing test 是 2026 後 QA 維護生產力的 game-changer。自動化 selector 飄移、保留人類判斷力。從 30 個 case 自建 wrapper 起步、3 個月後你會把維護時間從每週 8 小時砍到 2 小時。

延伸：

• Page Object Model 實戰
• Flaky Test 排雷指南
• AI 共存的 QA 工具箱