Commit graph

53 commits

Author SHA1 Message Date
Peter Steinberger
40acc9669b Fix deadlock in ImageCommand by replacing semaphore with RunLoop
- Remove DispatchSemaphore usage that violated Swift concurrency rules
- Implement RunLoop-based async-to-sync bridging in runAsyncCapture()
- Convert all capture methods to async/await patterns
- Replace Thread.sleep with Task.sleep in async contexts
- Keep ParsableCommand for compatibility, avoid AsyncParsableCommand issues
- Add comprehensive tests and documentation
- Improve error handling and browser helper filtering

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 09:41:50 +01:00
Peter Steinberger
3a837c7159 test: Add comprehensive tests for browser helper filtering
Documents the expected behavior and ensures browser helper filtering works correctly:
- Tests browser-specific error messages when main browser isn't running
- Verifies successful capture when main browser is found (not helpers)
- Documents the problem this fixes (no more confusing 'no capturable windows' errors)
- Ensures non-browser applications continue to work normally

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 08:48:29 +01:00
Peter Steinberger
d5b40c1550 feat: Implement proper frontmost window capture
Adds support for capturing the frontmost window of the frontmost application
instead of falling back to screen capture mode.

Changes:
- Added 'frontmost' case to CaptureMode enum in Swift CLI
- Implemented captureFrontmostWindow() method using NSWorkspace.shared.frontmostApplication
- Updated TypeScript to use --mode frontmost instead of defaulting to screen mode
- Added comprehensive test coverage for frontmost functionality
- Updated existing tests to reflect new behavior

The frontmost mode now:
1. Detects the currently active application
2. Captures only its frontmost window (index 0)
3. Returns a single image file with proper metadata

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 08:42:43 +01:00
Peter Steinberger
34dac65d2a fix: Handle empty string item_type parameter in list tool
Fixes issue where item_type: '' was not properly defaulting to the correct operation.
Empty strings and whitespace-only strings now fall back to the proper default logic:
- If app is provided: defaults to 'application_windows'
- If no app: defaults to 'running_applications'

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 08:33:44 +01:00
Peter Steinberger
4a2d802977 fix: Handle empty provider_config gracefully and improve case-insensitive targets
- Fix "Cannot convert undefined or null to object" error when provider_config is empty
- Make frontmost target case-insensitive (frontmost, FRONTMOST, Frontmost)
- Make window specifiers case-insensitive (WINDOW_TITLE, window_title, Window_Title)
- Add comprehensive test coverage for empty/null provider_config scenarios
- Improve error handling to prevent spread operator failures on undefined _meta

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 08:20:30 +01:00
Peter Steinberger
17dea6ad79 fix: Prevent security vulnerability from malformed app targets
Addresses critical edge case where malformed app targets with multiple leading colons
(e.g., "::::::::::::::::Finder") created empty app names that would match ALL system
processes. This could potentially expose sensitive information or cause unintended
system-wide captures.

Key improvements:
- Enhanced app target parsing to validate non-empty app names
- Added fallback logic to extract valid app names from malformed inputs
- Default to screen mode when all parts are empty (security-first approach)
- Comprehensive test coverage for edge cases
- Improved backward compatibility with hidden path parameters

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 08:16:39 +01:00
Peter Steinberger
dd680eb638 feat: Improve window title matching and error messages for URLs with ports
When users search for windows with URLs containing ports (e.g., 'http://example.com:8080'),
the system now provides much better debugging information when the window isn't found.

Key improvements:
- Enhanced window not found errors now list all available window titles
- Added specific guidance for URL-based searches (try without protocol)
- New CaptureError.windowTitleNotFound with detailed debugging info
- Comprehensive test coverage for colon parsing in app targets
- Better error messages help users understand why matching failed

Example improved error:
"Window with title containing 'http://example.com:8080' not found in Google Chrome.
Available windows: 'example.com:8080 - Google Chrome', 'New Tab - Google Chrome'.
Note: For URLs, try without the protocol (e.g., 'example.com:8080' instead of 'http://example.com:8080')."

This addresses the common issue where browsers display simplified URLs in window titles
without the protocol, making it easier for users to find the correct matching pattern.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 08:09:47 +01:00
Peter Steinberger
822ea1cce7 fix: Correct error handling for path traversal and file system errors
Previously, path traversal attempts like `../../../../../../../etc/passwd` were incorrectly
reported as screen recording permission errors instead of file system errors.

Changes:
- Modified ScreenCapture error handling to distinguish between CaptureError types and ScreenCaptureKit errors
- CaptureError.fileWriteError now bypasses screen recording permission detection
- Added path validation in OutputPathResolver to detect and log path traversal attempts
- Added logging for system-sensitive path access attempts
- Comprehensive test coverage for various path traversal patterns and error scenarios

This ensures users get accurate error messages that guide them to the actual problem
(invalid paths, missing directories, file permissions) rather than misleading
screen recording permission prompts.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 08:05:03 +01:00
Peter Steinberger
4afd15279c feat: Capture all windows from multiple exact app matches instead of erroring
When multiple applications have exact matches (e.g., "claude" and "Claude"), the system now:
- Captures all windows from all matching applications instead of throwing an ambiguous match error
- Maintains sequential window indices across all matched applications
- Preserves original application names in saved file metadata
- Only returns errors for truly ambiguous fuzzy matches

This provides more useful behavior for common scenarios where users have multiple apps with
similar names (different case, etc.) and want to capture windows from all of them.

Updates:
- Added `captureWindowsFromMultipleApps` method to handle multi-app capture logic
- Modified error handling in both single window and multi-window capture modes
- Updated documentation (spec.md, CHANGELOG.md) to reflect new behavior
- Comprehensive test suite covering various multiple match scenarios

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 08:00:44 +01:00
Peter Steinberger
089d96ce22 fix: Handle edge cases for invalid screen index and JSON null paths
- Invalid screen index (e.g., screen:99) now properly falls back to capturing all screens with unique filenames
- String "null" in path parameter is now correctly treated as undefined instead of literal path
- Added fallback-aware filename generation to prevent file overwrites when screen index is out of bounds
- Comprehensive test coverage for both edge cases

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 07:53:21 +01:00
Peter Steinberger
ab882069b4 fix: Add defensive validation for invalid image formats with automatic PNG fallback
Implements robust handling for invalid image formats (like 'bmp', 'gif', 'webp') that bypass schema validation:

- Added defensive format validation in image tool handler
- Automatic path correction to ensure file extensions match actual format used
- Warning messages in response when format fallback occurs
- Comprehensive unit and integration test coverage for edge cases

This ensures invalid formats automatically fall back to PNG as requested, preventing
Swift CLI rejection and incorrect file extensions in output paths.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 07:44:17 +01:00
Peter Steinberger
977c22d37a Fix array parameter parsing for include_window_details
- Add preprocessing to handle JSON string arrays from MCP clients
- Support multiple input formats: JSON string, comma-separated, single value
- Handle empty strings and null/undefined values gracefully
- Add comprehensive test coverage for all parsing scenarios
- Fixes "Expected array, received string" error when MCP clients send JSON string arrays

This resolves the issue shown in the test screenshot where include_window_details
was sent as '["ids", "bounds", "off_screen"]' (JSON string) instead of a proper array.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 07:40:14 +01:00
Peter Steinberger
b10253ea2e Fix tests to match new executeSwiftCli signature with timeout parameter
- Update all test assertions to expect the new three-parameter signature
- Add expect.objectContaining({ timeout: expect.any(Number) }) to all executeSwiftCli assertions
- Fixed 37 test assertions across image.test.ts, image-edge-cases.test.ts, and image-tool.test.ts
- All tests now pass (297 tests passed, 17 skipped)

This completes the integration of PR #2's timeout functionality by ensuring all tests match the new function signature.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 07:24:02 +01:00
Peter Steinberger
4e20e9adbd Merge PR #2: Add timeout handling to prevent test hangs
- Adds configurable timeout support via PEEKABOO_CLI_TIMEOUT env var
- Implements proper SIGTERM/SIGKILL handling for stuck processes
- Updates tests for Linux compatibility
- Fixes hanging issues when permission dialogs appear

Co-authored-by: codegen-sh[bot] <131295404+codegen-sh[bot]@users.noreply.github.com>
2025-06-08 07:13:21 +01:00
Peter Steinberger
ed1860d546 feat: Improve error propagation and debugging for system-level failures
- Enhanced CaptureError types to include underlying system errors
- Added comprehensive error logging in debug_logs for troubleshooting
- Fixed duplicate error output from ApplicationFinder
- Improved error details for app not found to show available applications
- Updated test expectations to match new error message formats

This ensures that errors from deep within ScreenCaptureKit and file operations
are properly surfaced to users with full context in the debug logs.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 07:08:35 +01:00
codegen-sh[bot]
4b9ab04878 Fix unit tests to match current implementation
- Add timeout parameter to all executeSwiftCli calls
- Update image tool tests to include --capture-focus parameter
- All tests now pass (206 passed, 65 skipped as expected)

Fixes failing CI tests in Node.js 20.x environment.
2025-06-08 06:04:05 +00:00
codegen-sh[bot]
271814cc90 Update tests for Linux compatibility and resolve merge conflicts
 **Merge Conflicts Resolved**
- Merged latest changes from main branch
- Resolved conflicts in docs/spec.md by keeping comprehensive specification
- Added PEEKABOO_CLI_TIMEOUT environment variable documentation

🧪 **Test Suite Updates for Linux Compatibility**
- Added platform-specific test skipping for Swift-dependent tests
- Created tests/setup.ts for global test configuration
- Updated vitest.config.ts with platform detection
- Modified integration tests to skip on non-macOS platforms:
  - tests/integration/peekaboo-cli-integration.test.ts
  - tests/integration/image-tool.test.ts
  - tests/integration/analyze-tool.test.ts

📦 **New Test Scripts**
- `npm run test:unit` - Run only unit tests (any platform)
- `npm run test:typescript` - Run TypeScript tests, skip Swift (Linux-friendly)
- `npm run test:typescript:watch` - Watch mode for TypeScript-only tests

🌍 **Platform Support**
- **macOS**: All tests run (unit + integration + Swift)
- **Linux/CI**: Only TypeScript tests run (Swift tests auto-skipped)
- **Environment Variables**:
  - `SKIP_SWIFT_TESTS=true` - Force skip Swift tests
  - `CI=true` - Auto-skip Swift tests in CI

📚 **Documentation Updates**
- Added comprehensive testing section to README.md
- Documented platform-specific test behavior
- Added environment variable documentation for test control

This allows the TypeScript parts of Peekaboo to be tested on Linux while maintaining full test coverage on macOS.
2025-06-08 06:03:49 +00:00
Peter Steinberger
94060963d0 test: Add comprehensive edge case tests for image and analyze tools
Added tests for:
- Whitespace trimming in app_target parameter
- Format parameter case-insensitivity and aliases
- Empty question handling (skips analysis for empty strings)
- Screen index parsing edge cases (float, hex, negative values)
- Special filesystem characters in filenames (|, :, *)
- Analyze tool edge cases (empty questions, error handling)
- Provider configuration edge cases
- Very long questions and special characters in responses

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 06:50:10 +01:00
Peter Steinberger
0301df2608 fix: Trim whitespace from app_target parameter
- Add .trim() to app_target when passing to Swift CLI
- Handles cases like "   Spotify   " correctly matching "Spotify"
- Applies to all app name formats including window specifiers

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 06:47:56 +01:00
Peter Steinberger
2676decf51 feat: Enhanced error messages for ambiguous app identifiers
- Error messages now include the list of matching applications when multiple apps match an identifier
- Shows bundle IDs alongside app names to help users disambiguate (e.g., Calendar (com.apple.iCal))
- Applies to both image and list tools for consistent user experience
- Added comprehensive tests for error detail handling

This makes it much easier for users to understand which specific application to target when there are multiple matches.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 06:16:15 +01:00
Peter Steinberger
dbb68e4294 feat: Auto-fallback to PNG for invalid format values and screen captures
- Screen captures with format 'data' now auto-fallback to PNG to prevent stack overflow
- Invalid format values (empty strings, null, unrecognized) gracefully fall back to PNG
- Added comprehensive tests for format validation
- Updated documentation to reflect the new behavior

This provides a better user experience by handling edge cases gracefully instead of returning errors.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 06:04:09 +01:00
Peter Steinberger
2e65e000f0 fallback to png for full screen captures. 2025-06-08 06:04:09 +01:00
Peter Steinberger
30277bbf6c fix: Prevent format 'data' for screen captures to avoid stack overflow
- Screen captures now reject format: 'data' with clear error message
- Large screen images cause JavaScript stack overflow when base64 encoded
- Application window captures can still use format: 'data'
- Update tests and documentation to reflect this limitation
2025-06-08 06:04:09 +01:00
Peter Steinberger
94e966fa98 feat: Add fuzzy matching for application names
- Implement Levenshtein distance algorithm for fuzzy app name matching
- Handle common typos like "Chromee" → "Google Chrome"
- Add window-specific labels in analysis results
- Improve error messages with app name suggestions
- Fix TypeScript JSON parsing for error responses
- Update tests for new error message formats

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 05:22:58 +01:00
Peter Steinberger
b3ec918363 Improve analyze description for multiple windows 2025-06-08 05:22:58 +01:00
Peter Steinberger
282d00f5d9 Add auto capture focus mode and fix list tool validation
- Added new "auto" capture focus mode that intelligently brings windows to foreground only when needed
- Changed default capture_focus from "background" to "auto" for better screenshot success rates
- Fixed list tool server_status validation to allow empty include_window_details arrays
- Added comprehensive tests for new auto mode functionality
- Enhanced error messages for better user experience

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 04:31:28 +01:00
Peter Steinberger
17b74b4f1f accept path as silent fallback parameter 2025-06-08 04:05:13 +01:00
Peter Steinberger
76d0faef42 image + analyze keeps the temp files 2025-06-08 04:02:04 +01:00
Peter Steinberger
e5e8123445 add more tests 2025-06-08 03:49:11 +01:00
Peter Steinberger
10672e57c0 Prepare v1.0.0-beta.15: Improved list tool usability and robustness
### Improved
- The list tool is now more lenient and user-friendly
- item_type parameter is now optional (defaults to 'running_applications')
- Intelligent auto-detection when app parameter is provided
- Enhanced error handling and validation

### Fixed
- Fixed crash when list tool called with empty item_type
- Improved image tool path handling for temporary files
- Better error messages and validation throughout

### Tests
- Added comprehensive test coverage for new list tool features
- Enhanced integration tests for improved scenarios
- Total test count increased from 223 to 228 tests

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-08 02:36:13 +01:00
Peter Steinberger
f5ad072bc8 chore: bump version to 1.0.0-beta.13 2025-06-08 01:28:12 +01:00
Peter Steinberger
6597b12e3c Fix TypeScript test error message expectation 2025-06-08 00:27:12 +01:00
Peter Steinberger
b1ddf6f1b6 Enhance Swift testing framework and test coverage
- Update CI configuration to use macOS-15 runner with Xcode 16.3
- Expand test coverage with comprehensive new test suites:
  * JSONOutputTests.swift - JSON encoding/decoding and MCP compliance
  * LoggerTests.swift - Thread-safe logging functionality
  * ImageCaptureLogicTests.swift - Image capture command logic
  * TestTags.swift - Centralized test tagging system
- Improve existing tests with Swift Testing patterns and async support
- Make Logger thread-safe with concurrent dispatch queue
- Add performance, concurrency, and edge case testing
- Fix compilation issues and optimize test performance

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-07 23:57:26 +01:00
Peter Steinberger
4e5e15c5a5 Prepare for 1.0.0-beta.11 release
- Update version to 1.0.0-beta.11 in package.json and Swift version file
- Update CHANGELOG.md with today's date
- Fix test expectations for new error message format
- Build universal Swift binary with latest changes

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-06-07 22:56:33 +01:00
Peter Steinberger
4636aec9d4 Fix test expectation for unknown type fallback
Changed from 'any' to 'string' to match the updated implementation

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-27 01:11:10 +02:00
Peter Steinberger
9295dd1ed0 Fix server status test by properly mocking fs module
- Mock fs.readFile in beforeEach to return valid package.json
- Mock fs module with existsSync and accessSync for CLI checks

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-27 00:44:33 +02:00
Peter Steinberger
c6abc9d168 Fix test failures after refactoring
- Fix buildSwiftCliArgs import in image tests
- Update buildSwiftCliArgs to match expected behavior (capture-focus default, warning messages)
- Fix image summary builder logic to properly generate summaries
- Update server status test to match new implementation that doesn't use executeSwiftCli

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-27 00:42:46 +02:00
Peter Steinberger
7bf63a225c Implement missing best practices
- Add npm run inspector script for MCP inspector tool
- Synchronize Swift CLI version with package.json (1.0.0-beta.9)
- Update macOS version requirement to v14 (Sonoma) for n-1 support
- Add Swift compiler warnings check in prepare-release script
- Convert tests/setup.ts from Jest to Vitest syntax
- Update server status tests to match new format

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-26 23:46:03 +02:00
Peter Steinberger
72819798ff feat: Simplify image tool API and add screen index support\n\n- MCP image tool API simplified with app_target, new format:data behavior, and automatic AI provider selection.\n- Swift CLI image command now supports --screen-index for specific display capture.\n- Updated image.ts, relevant documentation, and tests accordingly.\n- Version bumped to 1.0.0-beta.4. 2025-05-25 18:31:50 +02:00
Peter Steinberger
53ec5ef9a4 Add Swift linting and enhance image capture features
- Add SwiftLint and SwiftFormat configuration with npm scripts
- Refactor Swift code to comply with linting rules:
  - Fix identifier naming (x/y → xCoordinate/yCoordinate)
  - Extract long functions into smaller methods
  - Fix code style violations
- Enhance image capture tool:
  - Add blur detection parameter
  - Support custom image formats and quality
  - Add flexible naming patterns for saved files
- Add comprehensive integration tests for image tool
- Update documentation with new linting commands

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-25 18:02:39 +02:00
Peter Steinberger
f41e70e23e Add proper tool description and work around a bug in Gemini’s parser 2025-05-25 18:02:05 +02:00
Peter Steinberger
6396e299be Prepare for v1.0.0-beta.3 release 2025-05-25 14:07:21 +02:00
Peter Steinberger
ed59bb58dc Combine image + analyze 2025-05-25 13:32:39 +02:00
Peter Steinberger
6c2c327a2c Add E2E tests 2025-05-25 03:20:15 +02:00
Peter Steinberger
bda349ea29 Improve error messages 2025-05-25 02:27:19 +02:00
Peter Steinberger
3e9bfcdf4e Fixes a test 2025-05-25 01:43:47 +02:00
Peter Steinberger
c366be75e6 Test explainer 2025-05-25 01:43:43 +02:00
Peter Steinberger
cac7975bbe Update tests 2025-05-25 01:34:58 +02:00
Peter Steinberger
8209adc3d0 Update to Swift 6 and rename swift-cli to peekaboo-cli
- Migrate Swift code to Swift 6 compatibility
- Rename swift-cli folder to peekaboo-cli for better naming consistency
- Update all references throughout the codebase (build scripts, imports, tests)
- Fix .gitignore to properly track Swift source files
- Update environment variable names to use PEEKABOO_ prefix

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-25 01:31:43 +02:00
Peter Steinberger
97eba5e97e Add tests; finish peekabo cli rename 2025-05-25 01:28:34 +02:00