diff --git a/web/docs/PLAYWRIGHT_OPTIMIZATION.md b/web/docs/PLAYWRIGHT_OPTIMIZATION.md deleted file mode 100644 index 8a2eb21c..00000000 --- a/web/docs/PLAYWRIGHT_OPTIMIZATION.md +++ /dev/null @@ -1,118 +0,0 @@ -# Playwright Test Optimization Guide - -## Current Issues - -1. **Test Duration**: Tests are taking 20-30+ minutes in CI -2. **Modal Tests**: Working correctly locally but may have timing issues in CI -3. **Session Creation**: Each test creates real terminal sessions which adds overhead - -## Implemented Fixes - -### Modal Implementation -- ✅ Modal functionality works correctly with the new `modal-wrapper` component -- ✅ Escape key handling works as expected -- ✅ Form interactions are responsive - -### Test Improvements -- ✅ Reduced unnecessary `waitForTimeout` calls where possible -- ✅ Optimized wait strategies for modal interactions - -## Recommendations for Further Optimization - -### 1. Parallel Test Execution -Currently tests run with `workers: 1`. Consider: -```javascript -// playwright.config.ts -workers: process.env.CI ? 2 : 4, -fullyParallel: true, -``` - -### 2. Mock Session Creation -For non-critical tests, mock the session API: -```javascript -await page.route('/api/sessions', async (route) => { - await route.fulfill({ - status: 200, - body: JSON.stringify({ - sessionId: 'mock-session-id', - // ... other session data - }) - }); -}); -``` - -### 3. Reuse Test Sessions -Create a pool of test sessions at the start and reuse them: -```javascript -// global-setup.ts -const testSessions = await createTestSessionPool(5); -process.env.TEST_SESSION_POOL = JSON.stringify(testSessions); -``` - -### 4. Reduce Animation Delays -In test mode, disable or speed up animations: -```css -/* When running tests */ -body[data-testid="playwright"] * { - animation-duration: 0s !important; - transition-duration: 0s !important; -} -``` - -### 5. Use Test-Specific Timeouts -```javascript -// For fast operations -await expect(element).toBeVisible({ timeout: 2000 }); - -// For network operations -await page.waitForResponse('/api/sessions', { timeout: 5000 }); -``` - -### 6. Skip Unnecessary Waits -Replace: -```javascript -await page.waitForLoadState('networkidle'); -``` - -With: -```javascript -await page.waitForSelector('vibetunnel-app', { state: 'attached' }); -``` - -### 7. CI-Specific Optimizations -- Use `--disable-dev-shm-usage` for Chromium in CI -- Increase `--max-old-space-size` for Node.js -- Consider using a more powerful CI runner - -## Running Tests Efficiently - -### Local Development -```bash -# Run specific test file -pnpm exec playwright test session-creation.spec.ts - -# Run with UI mode for debugging -pnpm exec playwright test --ui - -# Run with trace for debugging failures -pnpm exec playwright test --trace on -``` - -### CI Optimization -```yaml -# GitHub Actions example -- name: Run Playwright tests - run: pnpm run test:e2e - env: - NODE_OPTIONS: --max-old-space-size=4096 - PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD: 1 -``` - -## Monitoring Test Performance - -Use the provided script to identify slow tests: -```bash -./scripts/profile-playwright-tests.sh -``` - -This will show the slowest tests that need optimization. \ No newline at end of file diff --git a/web/docs/playwright-performance.md b/web/docs/playwright-performance.md deleted file mode 100644 index e17cc1ce..00000000 --- a/web/docs/playwright-performance.md +++ /dev/null @@ -1,408 +0,0 @@ -# Playwright Performance Optimization Guide - -## Overview - -This guide provides comprehensive strategies for optimizing Playwright test execution speed based on 2024-2025 best practices. Our goal is to achieve significant performance improvements while maintaining test reliability. - -## Current Performance Baseline - -Before optimization: -- 31 tests taking ~12+ minutes in CI -- Sequential execution with limited parallelization -- No sharding or distributed execution -- Full browser context creation for each test - -## Optimization Strategies - -### 1. Parallel Execution and Workers - -**Impact**: 3-4x speed improvement - -```typescript -// playwright.config.ts -export default defineConfig({ - // Optimize worker count based on CI environment - workers: process.env.CI ? 4 : undefined, - - // Enable full parallelization - fullyParallel: true, - - // Limit failures to avoid wasting resources - maxFailures: process.env.CI ? 10 : undefined, -}); -``` - -**Implementation**: -- Use CPU core count for optimal worker allocation -- Enable `fullyParallel` for test-level parallelization -- Set `maxFailures` to stop early on broken builds - -### 2. Test Sharding for CI - -**Impact**: 4-8x speed improvement with proper distribution - -```yaml -# .github/workflows/playwright.yml -strategy: - matrix: - shardIndex: [1, 2, 3, 4] - shardTotal: [4] -steps: - - run: pnpm exec playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }} -``` - -**Implementation**: -- Split tests across 4 shards for parallel CI execution -- Each shard runs on separate CI job -- Aggregate results after all shards complete - -### 3. Browser Context Optimization - -**Impact**: 20-30% speed improvement - -```typescript -// fixtures/session-fixtures.ts -export const test = base.extend<{ - authenticatedContext: BrowserContext; -}>({ - authenticatedContext: async ({ browser }, use) => { - // Create context once per worker - const context = await browser.newContext({ - storageState: 'tests/auth.json' - }); - await use(context); - await context.close(); - }, -}); -``` - -**Strategies**: -- Reuse authentication state across tests -- Share expensive setup within workers -- Use project-specific contexts for different test types - -### 4. Smart Waiting and Selectors - -**Impact**: 10-20% speed improvement - -```typescript -// Bad: Static waits -await page.waitForTimeout(5000); - -// Good: Dynamic waits -await page.waitForSelector('[data-testid="session-ready"]', { - state: 'visible', - timeout: 5000 -}); - -// Better: Wait for specific conditions -await page.waitForFunction(() => { - const terminal = document.querySelector('vibe-terminal'); - return terminal?.dataset.ready === 'true'; -}); -``` - -**Best Practices**: -- Never use static timeouts -- Use data-testid attributes for fast selection -- Avoid XPath selectors -- Wait for specific application states - -### 5. Resource Blocking - -**Impact**: 15-25% speed improvement - -```typescript -// playwright.config.ts -export default defineConfig({ - use: { - // Block unnecessary resources - launchOptions: { - args: ['--disable-web-security', '--disable-features=IsolateOrigins,site-per-process'] - }, - }, -}); - -// In tests -await context.route('**/*.{png,jpg,jpeg,gif,svg,woff,woff2,ttf}', route => route.abort()); -await context.route('**/analytics/**', route => route.abort()); -``` - -### 6. Test Organization and Isolation - -**Impact**: 30-40% improvement through smart grouping - -```typescript -// Group related tests that can share setup -test.describe('Session Management', () => { - test.beforeAll(async ({ browser }) => { - // Expensive setup once per group - }); - - test('should create session', async ({ page }) => { - // Test implementation - }); - - test('should delete session', async ({ page }) => { - // Test implementation - }); -}); -``` - -### 7. Global Setup for Authentication - -**Impact**: Saves 2-3 seconds per test - -```typescript -// global-setup.ts -import { chromium } from '@playwright/test'; - -async function globalSetup() { - const browser = await chromium.launch(); - const page = await browser.newPage(); - - // Perform authentication - await page.goto('http://localhost:4022'); - await page.fill('[name="username"]', 'testuser'); - await page.fill('[name="password"]', 'testpass'); - await page.click('[type="submit"]'); - - // Save storage state - await page.context().storageState({ path: 'tests/auth.json' }); - await browser.close(); -} - -export default globalSetup; -``` - -### 8. CI/CD Optimizations - -**Impact**: 50% faster CI builds - -```yaml -# Cache Playwright browsers -- uses: actions/cache@v3 - with: - path: ~/.cache/ms-playwright - key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }} - -# Install only required browsers -- run: pnpm exec playwright install chromium - -# Use Linux for CI (faster and cheaper) -runs-on: ubuntu-latest -``` - -### 9. Headless Mode - -**Impact**: 20-30% speed improvement - -```typescript -// playwright.config.ts -export default defineConfig({ - use: { - headless: true, // Always true in CI - video: process.env.CI ? 'retain-on-failure' : 'off', - screenshot: 'only-on-failure', - }, -}); -``` - -### 10. Performance Monitoring - -```typescript -// Add performance tracking -test.beforeEach(async ({ page }, testInfo) => { - const startTime = Date.now(); - - testInfo.attachments.push({ - name: 'performance-metrics', - body: JSON.stringify({ - test: testInfo.title, - startTime, - }), - contentType: 'application/json', - }); -}); -``` - -## Implementation Priority - -1. **Phase 1 - Quick Wins** (1-2 hours) - - Enable parallel execution - - Switch to headless mode in CI - - Implement resource blocking - - Optimize selectors - -2. **Phase 2 - Medium Impact** (2-4 hours) - - Implement test sharding - - Add global authentication setup - - Optimize browser contexts - - Group related tests - -3. **Phase 3 - Advanced** (4-8 hours) - - Implement custom fixtures - - Add performance monitoring - - Optimize CI pipeline - - Fine-tune worker allocation - -## Expected Results - -With full implementation: -- **Local development**: 3-4x faster (3-4 minutes) -- **CI execution**: 6-8x faster (1.5-2 minutes) -- **Resource usage**: 40% reduction -- **Flakiness**: Significantly reduced - -## Monitoring and Maintenance - -1. Track test execution times in CI -2. Monitor flaky tests with retry analytics -3. Regular review of slow tests -4. Periodic selector optimization -5. Update Playwright version quarterly - -## Common Pitfalls to Avoid - -1. Over-parallelization causing resource contention -2. Sharing too much state between tests -3. Using static waits instead of dynamic conditions -4. Not considering CI environment limitations -5. Ignoring test isolation principles - -## Sequential Execution Optimizations - -Since VibeTunnel tests share system-level terminal resources and cannot run in parallel, we need different optimization strategies: - -### 1. Browser Context and Page Reuse - -**Impact**: Save 1-2s per test - -```typescript -// fixtures/reusable-context.ts -let globalContext: BrowserContext | null = null; -let globalPage: Page | null = null; - -export const test = base.extend({ - context: async ({ browser }, use) => { - if (!globalContext) { - globalContext = await browser.newContext(); - } - await use(globalContext); - // Don't close - reuse for next test - }, - - page: async ({ context }, use) => { - if (!globalPage || globalPage.isClosed()) { - globalPage = await context.newPage(); - } else { - // Clear state for next test - await globalPage.goto('about:blank'); - } - await use(globalPage); - // Don't close - reuse for next test - } -}); -``` - -### 2. Smart Test Ordering - -Order tests from least destructive to most destructive: - -```typescript -test.describe.configure({ mode: 'serial' }); - -test.describe('1. Read operations', () => { - test('view sessions', async ({ page }) => {}); -}); - -test.describe('2. Create operations', () => { - test('create sessions', async ({ page }) => {}); -}); - -test.describe('3. Destructive operations', () => { - test('kill sessions', async ({ page }) => {}); -}); -``` - -### 3. Session Pool with Pre-creation - -**Impact**: Save 2-3s per test - -```typescript -test.beforeAll(async ({ page }) => { - const pool = new SessionPool(page); - await pool.initialize(5); // Create 5 sessions upfront - global.sessionPool = pool; -}); -``` - -### 4. Aggressive Resource Blocking - -```typescript -await context.route('**/*', (route) => { - const url = route.request().url(); - const allowedPatterns = ['localhost:4022', '/api/', '.js', '.css']; - - if (!allowedPatterns.some(pattern => url.includes(pattern))) { - route.abort(); - } else { - route.continue(); - } -}); -``` - -### 5. Reduced Timeouts - -```typescript -export const TEST_TIMEOUTS = { - QUICK: 1000, // Reduce from 3000 - DEFAULT: 2000, // Reduce from 5000 - LONG: 5000, // Reduce from 10000 -}; -``` - -### 6. Skip Animations - -```typescript -await page.addStyleTag({ - content: ` - *, *::before, *::after { - animation-duration: 0s !important; - transition-duration: 0s !important; - } - ` -}); -``` - -### Sequential Implementation Plan - -**Phase 1 - Immediate (30 mins)**: -- Reduce all timeouts -- Enable headless mode -- Block unnecessary resources -- Skip animations - -**Phase 2 - Quick Wins (1-2 hours)**: -- Implement browser/page reuse -- Add smart cleanup -- Optimize waiting strategies - -**Phase 3 - Architecture (2-4 hours)**: -- Implement session pool -- Reorganize test order -- Add state persistence - -### Expected Results for Sequential Tests - -- **Current**: ~12+ minutes -- **Target**: ~3-5 minutes (3-4x improvement) -- **Key gains**: - - Page reuse: Save 1-2s per test - - Reduced timeouts: Save 30-60s total - - Resource blocking: Save 20-30% load time - - Session pool: Save 2-3s per test - -## Conclusion - -While parallel execution would provide the best performance gains, these sequential optimizations can still achieve significant improvements. Start with quick wins and progressively implement more advanced optimizations based on your specific needs. \ No newline at end of file diff --git a/web/playwright.config.ts b/web/playwright.config.ts index 79a1e5c6..aa495f8e 100644 --- a/web/playwright.config.ts +++ b/web/playwright.config.ts @@ -17,14 +17,26 @@ export default defineConfig({ /* Global setup */ globalSetup: require.resolve('./src/test/playwright/global-setup.ts'), + globalTeardown: require.resolve('./src/test/playwright/global-teardown.ts'), /* Run tests in files in parallel */ - fullyParallel: false, // Keep sequential for stability + fullyParallel: true, // Enable parallel execution for better performance /* Fail the build on CI if you accidentally left test.only in the source code. */ forbidOnly: !!process.env.CI, /* Retry on CI only */ retries: process.env.CI ? 2 : 0, - /* Opt out of parallel tests on CI. */ - workers: 1, // Force single worker for stability + /* Parallel workers configuration */ + workers: (() => { + if (process.env.PLAYWRIGHT_WORKERS) { + const parsed = parseInt(process.env.PLAYWRIGHT_WORKERS, 10); + // Validate the parsed value + if (!isNaN(parsed) && parsed > 0) { + return parsed; + } + console.warn(`Invalid PLAYWRIGHT_WORKERS value: "${process.env.PLAYWRIGHT_WORKERS}". Using default.`); + } + // Default: 8 workers in CI, auto-detect locally + return process.env.CI ? 8 : undefined; + })(), /* Test timeout */ timeout: process.env.CI ? 30 * 1000 : 20 * 1000, // 30s on CI, 20s locally /* Reporter to use. See https://playwright.dev/docs/test-reporters */ @@ -74,9 +86,32 @@ export default defineConfig({ /* Configure projects for major browsers */ projects: [ + // Parallel tests - these tests create their own isolated sessions { - name: 'chromium', + name: 'chromium-parallel', use: { ...devices['Desktop Chrome'] }, + testMatch: [ + '**/session-creation.spec.ts', + '**/basic-session.spec.ts', + '**/minimal-session.spec.ts', + '**/debug-session.spec.ts', + '**/ui-features.spec.ts', + '**/test-session-persistence.spec.ts', + '**/session-navigation.spec.ts', + '**/session-management.spec.ts', + '**/session-management-advanced.spec.ts', + ], + }, + // Serial tests - these tests perform global operations or modify shared state + { + name: 'chromium-serial', + use: { ...devices['Desktop Chrome'] }, + testMatch: [ + '**/session-management-global.spec.ts', + '**/keyboard-shortcuts.spec.ts', + '**/terminal-interaction.spec.ts', + ], + fullyParallel: false, // Override global setting for serial tests }, ], diff --git a/web/scripts/test-parallel.sh b/web/scripts/test-parallel.sh new file mode 100755 index 00000000..cffa045a --- /dev/null +++ b/web/scripts/test-parallel.sh @@ -0,0 +1,35 @@ +#!/bin/bash + +# Script to run Playwright tests with parallel configuration + +echo "Running Playwright tests with parallel configuration..." +echo "" + +# Run all tests (parallel and serial) +if [ "$1" == "all" ]; then + echo "Running all tests (parallel + serial)..." + pnpm exec playwright test +elif [ "$1" == "parallel" ]; then + echo "Running only parallel tests..." + pnpm exec playwright test --project=chromium-parallel +elif [ "$1" == "serial" ]; then + echo "Running only serial tests..." + pnpm exec playwright test --project=chromium-serial +elif [ "$1" == "debug" ]; then + echo "Running tests in debug mode..." + pnpm exec playwright test --debug +elif [ "$1" == "ui" ]; then + echo "Running tests with UI mode..." + pnpm exec playwright test --ui +else + echo "Usage: ./scripts/test-parallel.sh [all|parallel|serial|debug|ui]" + echo "" + echo "Options:" + echo " all - Run all tests (parallel and serial)" + echo " parallel - Run only parallel tests" + echo " serial - Run only serial tests" + echo " debug - Run tests in debug mode" + echo " ui - Run tests with Playwright UI" + echo "" + echo "If no option is provided, this help message is shown." +fi \ No newline at end of file diff --git a/web/src/client/app.ts b/web/src/client/app.ts index 3bd444cf..479f3ead 100644 --- a/web/src/client/app.ts +++ b/web/src/client/app.ts @@ -618,9 +618,17 @@ export class VibeTunnelApp extends LitElement { private handleCreateSession() { // Check if View Transitions API is supported if ('startViewTransition' in document && typeof document.startViewTransition === 'function') { - document.startViewTransition(() => { + // Set data attribute to indicate transition is starting + document.documentElement.setAttribute('data-view-transition', 'active'); + + const transition = document.startViewTransition(() => { this.showCreateModal = true; }); + + // Clear the attribute when transition completes + transition.finished.finally(() => { + document.documentElement.removeAttribute('data-view-transition'); + }); } else { this.showCreateModal = true; } @@ -631,14 +639,17 @@ export class VibeTunnelApp extends LitElement { if ('startViewTransition' in document && typeof document.startViewTransition === 'function') { // Add a class to prevent flicker during transition document.body.classList.add('modal-closing'); + // Set data attribute to indicate transition is starting + document.documentElement.setAttribute('data-view-transition', 'active'); const transition = document.startViewTransition(() => { this.showCreateModal = false; }); - // Clean up the class after transition + // Clean up the class and attribute after transition transition.finished.finally(() => { document.body.classList.remove('modal-closing'); + document.documentElement.removeAttribute('data-view-transition'); }); } else { this.showCreateModal = false; diff --git a/web/src/client/components/session-create-form.ts b/web/src/client/components/session-create-form.ts index 01bc4ca6..fe64c6ca 100644 --- a/web/src/client/components/session-create-form.ts +++ b/web/src/client/components/session-create-form.ts @@ -176,9 +176,17 @@ export class SessionCreateForm extends LitElement { this.loadFromLocalStorage(); // Add global keyboard listener document.addEventListener('keydown', this.handleGlobalKeyDown); + + // Set data attributes for testing - both synchronously to avoid race conditions + this.setAttribute('data-modal-state', 'open'); + this.setAttribute('data-modal-rendered', 'true'); } else { // Remove global keyboard listener when hidden document.removeEventListener('keydown', this.handleGlobalKeyDown); + + // Remove data attributes synchronously + this.removeAttribute('data-modal-state'); + this.removeAttribute('data-modal-rendered'); } } } @@ -381,11 +389,12 @@ export class SessionCreateForm extends LitElement { } return html` -