vibetunnel/web/docs/playwright-performance.md

Playwright Performance Optimization Guide

Overview

This guide provides comprehensive strategies for optimizing Playwright test execution speed based on 2024-2025 best practices. Our goal is to achieve significant performance improvements while maintaining test reliability.

Current Performance Baseline

Before optimization:

• 31 tests taking ~12+ minutes in CI
• Sequential execution with limited parallelization
• No sharding or distributed execution
• Full browser context creation for each test

Optimization Strategies

1. Parallel Execution and Workers

Impact: 3-4x speed improvement

// playwright.config.ts
export default defineConfig({
  // Optimize worker count based on CI environment
  workers: process.env.CI ? 4 : undefined,
  
  // Enable full parallelization
  fullyParallel: true,
  
  // Limit failures to avoid wasting resources
  maxFailures: process.env.CI ? 10 : undefined,
});

Implementation:

• Use CPU core count for optimal worker allocation
• Enable fullyParallel for test-level parallelization
• Set maxFailures to stop early on broken builds

2. Test Sharding for CI

Impact: 4-8x speed improvement with proper distribution

# .github/workflows/playwright.yml
strategy:
  matrix:
    shardIndex: [1, 2, 3, 4]
    shardTotal: [4]
steps:
  - run: pnpm exec playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

Implementation:

• Split tests across 4 shards for parallel CI execution
• Each shard runs on separate CI job
• Aggregate results after all shards complete

3. Browser Context Optimization

Impact: 20-30% speed improvement

// fixtures/session-fixtures.ts
export const test = base.extend<{
  authenticatedContext: BrowserContext;
}>({
  authenticatedContext: async ({ browser }, use) => {
    // Create context once per worker
    const context = await browser.newContext({
      storageState: 'tests/auth.json'
    });
    await use(context);
    await context.close();
  },
});

Strategies:

• Reuse authentication state across tests
• Share expensive setup within workers
• Use project-specific contexts for different test types

4. Smart Waiting and Selectors

Impact: 10-20% speed improvement

// Bad: Static waits
await page.waitForTimeout(5000);

// Good: Dynamic waits
await page.waitForSelector('[data-testid="session-ready"]', {
  state: 'visible',
  timeout: 5000
});

// Better: Wait for specific conditions
await page.waitForFunction(() => {
  const terminal = document.querySelector('vibe-terminal');
  return terminal?.dataset.ready === 'true';
});

Best Practices:

• Never use static timeouts
• Use data-testid attributes for fast selection
• Avoid XPath selectors
• Wait for specific application states

5. Resource Blocking

Impact: 15-25% speed improvement

// playwright.config.ts
export default defineConfig({
  use: {
    // Block unnecessary resources
    launchOptions: {
      args: ['--disable-web-security', '--disable-features=IsolateOrigins,site-per-process']
    },
  },
});

// In tests
await context.route('**/*.{png,jpg,jpeg,gif,svg,woff,woff2,ttf}', route => route.abort());
await context.route('**/analytics/**', route => route.abort());

6. Test Organization and Isolation

Impact: 30-40% improvement through smart grouping

// Group related tests that can share setup
test.describe('Session Management', () => {
  test.beforeAll(async ({ browser }) => {
    // Expensive setup once per group
  });
  
  test('should create session', async ({ page }) => {
    // Test implementation
  });
  
  test('should delete session', async ({ page }) => {
    // Test implementation
  });
});

7. Global Setup for Authentication

Impact: Saves 2-3 seconds per test

// global-setup.ts
import { chromium } from '@playwright/test';

async function globalSetup() {
  const browser = await chromium.launch();
  const page = await browser.newPage();
  
  // Perform authentication
  await page.goto('http://localhost:4022');
  await page.fill('[name="username"]', 'testuser');
  await page.fill('[name="password"]', 'testpass');
  await page.click('[type="submit"]');
  
  // Save storage state
  await page.context().storageState({ path: 'tests/auth.json' });
  await browser.close();
}

export default globalSetup;

8. CI/CD Optimizations

Impact: 50% faster CI builds

# Cache Playwright browsers
- uses: actions/cache@v3
  with:
    path: ~/.cache/ms-playwright
    key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}

# Install only required browsers
- run: pnpm exec playwright install chromium

# Use Linux for CI (faster and cheaper)
runs-on: ubuntu-latest

9. Headless Mode

Impact: 20-30% speed improvement

// playwright.config.ts
export default defineConfig({
  use: {
    headless: true, // Always true in CI
    video: process.env.CI ? 'retain-on-failure' : 'off',
    screenshot: 'only-on-failure',
  },
});

10. Performance Monitoring

// Add performance tracking
test.beforeEach(async ({ page }, testInfo) => {
  const startTime = Date.now();
  
  testInfo.attachments.push({
    name: 'performance-metrics',
    body: JSON.stringify({
      test: testInfo.title,
      startTime,
    }),
    contentType: 'application/json',
  });
});

Implementation Priority

1. Phase 1 - Quick Wins (1-2 hours)

   • Enable parallel execution
   • Switch to headless mode in CI
   • Implement resource blocking
   • Optimize selectors

2. Phase 2 - Medium Impact (2-4 hours)

   • Implement test sharding
   • Add global authentication setup
   • Optimize browser contexts
   • Group related tests

3. Phase 3 - Advanced (4-8 hours)

   • Implement custom fixtures
   • Add performance monitoring
   • Optimize CI pipeline
   • Fine-tune worker allocation

Expected Results

With full implementation:

• Local development: 3-4x faster (3-4 minutes)
• CI execution: 6-8x faster (1.5-2 minutes)
• Resource usage: 40% reduction
• Flakiness: Significantly reduced

Monitoring and Maintenance

1. Track test execution times in CI
2. Monitor flaky tests with retry analytics
3. Regular review of slow tests
4. Periodic selector optimization
5. Update Playwright version quarterly

Common Pitfalls to Avoid

1. Over-parallelization causing resource contention
2. Sharing too much state between tests
3. Using static waits instead of dynamic conditions
4. Not considering CI environment limitations
5. Ignoring test isolation principles

Sequential Execution Optimizations

Since VibeTunnel tests share system-level terminal resources and cannot run in parallel, we need different optimization strategies:

1. Browser Context and Page Reuse

Impact: Save 1-2s per test

// fixtures/reusable-context.ts
let globalContext: BrowserContext | null = null;
let globalPage: Page | null = null;

export const test = base.extend({
  context: async ({ browser }, use) => {
    if (!globalContext) {
      globalContext = await browser.newContext();
    }
    await use(globalContext);
    // Don't close - reuse for next test
  },
  
  page: async ({ context }, use) => {
    if (!globalPage || globalPage.isClosed()) {
      globalPage = await context.newPage();
    } else {
      // Clear state for next test
      await globalPage.goto('about:blank');
    }
    await use(globalPage);
    // Don't close - reuse for next test
  }
});

2. Smart Test Ordering

Order tests from least destructive to most destructive:

test.describe.configure({ mode: 'serial' });

test.describe('1. Read operations', () => {
  test('view sessions', async ({ page }) => {});
});

test.describe('2. Create operations', () => {
  test('create sessions', async ({ page }) => {});
});

test.describe('3. Destructive operations', () => {
  test('kill sessions', async ({ page }) => {});
});

3. Session Pool with Pre-creation

Impact: Save 2-3s per test

test.beforeAll(async ({ page }) => {
  const pool = new SessionPool(page);
  await pool.initialize(5); // Create 5 sessions upfront
  global.sessionPool = pool;
});

4. Aggressive Resource Blocking

await context.route('**/*', (route) => {
  const url = route.request().url();
  const allowedPatterns = ['localhost:4022', '/api/', '.js', '.css'];
  
  if (!allowedPatterns.some(pattern => url.includes(pattern))) {
    route.abort();
  } else {
    route.continue();
  }
});

5. Reduced Timeouts

export const TEST_TIMEOUTS = {
  QUICK: 1000,    // Reduce from 3000
  DEFAULT: 2000,  // Reduce from 5000
  LONG: 5000,     // Reduce from 10000
};

6. Skip Animations

await page.addStyleTag({
  content: `
    *, *::before, *::after {
      animation-duration: 0s !important;
      transition-duration: 0s !important;
    }
  `
});

Sequential Implementation Plan

Phase 1 - Immediate (30 mins):

• Reduce all timeouts
• Enable headless mode
• Block unnecessary resources
• Skip animations

Phase 2 - Quick Wins (1-2 hours):

• Implement browser/page reuse
• Add smart cleanup
• Optimize waiting strategies

Phase 3 - Architecture (2-4 hours):

• Implement session pool
• Reorganize test order
• Add state persistence

Expected Results for Sequential Tests

• Current: ~12+ minutes
• Target: ~3-5 minutes (3-4x improvement)
• Key gains:
  • Page reuse: Save 1-2s per test
  • Reduced timeouts: Save 30-60s total
  • Resource blocking: Save 20-30% load time
  • Session pool: Save 2-3s per test

Conclusion

While parallel execution would provide the best performance gains, these sequential optimizations can still achieve significant improvements. Start with quick wins and progressively implement more advanced optimizations based on your specific needs.