sjs/Peekaboo
1
0
Fork 0
mirror of https://github.com/samsonjs/Peekaboo.git synced 2026-03-25 09:25:47 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

feat: Capture all windows from multiple exact app matches instead of erroring

Browse source 
When multiple applications have exact matches (e.g., "claude" and "Claude"), the system now:
- Captures all windows from all matching applications instead of throwing an ambiguous match error
- Maintains sequential window indices across all matched applications
- Preserves original application names in saved file metadata
- Only returns errors for truly ambiguous fuzzy matches

This provides more useful behavior for common scenarios where users have multiple apps with
similar names (different case, etc.) and want to capture windows from all of them.

Updates:
- Added `captureWindowsFromMultipleApps` method to handle multi-app capture logic
- Modified error handling in both single window and multi-window capture modes
- Updated documentation (spec.md, CHANGELOG.md) to reflect new behavior
- Comprehensive test suite covering various multiple match scenarios

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Peter Steinberger 2025-06-08 08:00:44 +01:00
parent 089d96ce22
commit 4afd15279c
4 changed files with 402 additions and 313 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
CHANGELOG.md
View file

@ -7,6 +7,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Changed
- **Multiple exact app matches now capture all windows instead of erroring**
- When multiple applications have exact matches (e.g., "claude" and "Claude"), the system now captures all windows from all matching applications
- This replaces the previous behavior of throwing an ambiguous match error
- Window indices are sequential across all matched applications
- Each saved file preserves the original application name in `item_label`
- Only truly ambiguous fuzzy matches still return errors
- Comprehensive test coverage for various multiple match scenarios
## [1.0.0-beta.20] - 2025-06-08
### Added

4
docs/spec.md
View file

@ -365,7 +365,7 @@ Configured AI Providers (from PEEKABOO_AI_PROVIDERS ENV): <parsed list or 'None
* `--format <FormatEnum?>`: `FormatEnum` is `png, jpg`. Default `png`.
* `--capture-focus <FocusEnum?>`: `FocusEnum` is `background, foreground`. Default `background`.
* **Behavior:**
* Implements fuzzy app matching. On ambiguity, returns JSON error with `code: "AMBIGUOUS_APP_IDENTIFIER"` and lists potential matches in `error.details` or `error.message`.
* Implements fuzzy app matching. When multiple exact matches exist (e.g., "claude" and "Claude"), captures all windows from all matching applications rather than returning an error. Only truly ambiguous fuzzy matches return JSON errors with `code: "AMBIGUOUS_APP_IDENTIFIER"`.
* Always attempts to exclude window shadow/frame (`CGWindowImageOption.boundsIgnoreFraming` or `screencapture -o` if shelled out for PDF). No cursor is captured.
* **Background Capture (`--capture-focus background` or default):**
* Primary method: Uses `CGWindowListCopyWindowInfo` to identify target window(s)/screen(s).
@ -404,7 +404,7 @@ Configured AI Providers (from PEEKABOO_AI_PROVIDERS ENV): <parsed list or 'None
* **Behavior:**
* `apps`: Uses `NSWorkspace.shared.runningApplications`. For each app, retrieves `localizedName`, `bundleIdentifier`, `processIdentifier` (pid), `isActive`. To get `window_count`, it performs a `CGWindowListCopyWindowInfo` call filtered by the app's PID and counts on-screen windows.
* `windows`:
* Resolves `app_identifier` using fuzzy matching. If ambiguous, returns JSON error.
* Resolves `app_identifier` using fuzzy matching. If multiple exact matches exist, captures all windows from all matching applications.
* Uses `CGWindowListCopyWindowInfo` filtered by the target app's PID.
* If `--include-details` contains `"off_screen"`, uses `CGWindowListOption.optionAllScreenWindows` (and includes `kCGWindowIsOnscreen` boolean in output). Otherwise, uses `CGWindowListOption.optionOnScreenOnly`.
* Extracts `kCGWindowName` (title).

387
peekaboo-cli/Sources/peekaboo/ImageCommand.swift
View file

@ -145,7 +145,7 @@ struct ImageCommand: ParsableCommand {
}
// Provide additional details for app not found errors
var details: String? = nil
var details: String?
if case .appNotFound = captureError {
let runningApps = NSWorkspace.shared.runningApplications
.filter { $0.activationPolicy == .regular }
@ -226,7 +226,7 @@ struct ImageCommand: ParsableCommand {
}
return savedFiles
}
private func captureAllScreensWithFallback(displays: [CGDirectDisplayID]) throws(CaptureError) -> [SavedFile] {
var savedFiles: [SavedFile] = []
for (index, displayID) in displays.enumerated() {
@ -241,8 +241,8 @@ struct ImageCommand: ParsableCommand {
index: Int,
labelSuffix: String
) throws(CaptureError) -> SavedFile {
let fileName = generateFileName(displayIndex: index)
let filePath = getOutputPath(fileName)
let fileName = FileNameGenerator.generateFileName(displayIndex: index, format: format)
let filePath = OutputPathResolver.getOutputPath(basePath: path, fileName: fileName)
try captureDisplay(displayID, to: filePath)
@ -255,14 +255,14 @@ struct ImageCommand: ParsableCommand {
mime_type: format == .png ? "image/png" : "image/jpeg"
)
}
private func captureSingleDisplayWithFallback(
displayID: CGDirectDisplayID,
index: Int,
labelSuffix: String
) throws(CaptureError) -> SavedFile {
let fileName = generateFileName(displayIndex: index)
let filePath = getOutputPathWithFallback(fileName)
let fileName = FileNameGenerator.generateFileName(displayIndex: index, format: format)
let filePath = OutputPathResolver.getOutputPathWithFallback(basePath: path, fileName: fileName)
try captureDisplay(displayID, to: filePath)
@ -283,9 +283,9 @@ struct ImageCommand: ParsableCommand {
} catch let ApplicationError.notFound(identifier) {
throw CaptureError.appNotFound(identifier)
} catch let ApplicationError.ambiguous(identifier, matches) {
let appNames = matches.map { $0.localizedName ?? $0.bundleIdentifier ?? "Unknown" }
throw CaptureError
.unknownError("Multiple applications match '\(identifier)': \(appNames.joined(separator: ", "))")
// For ambiguous matches, capture all windows from all matching applications
Logger.shared.debug("Multiple applications match '\(identifier)', capturing all windows from all matches")
return try captureWindowsFromMultipleApps(matches, appIdentifier: identifier)
}
if captureFocus == .foreground || (captureFocus == .auto && !targetApp.isActive) {
@ -314,8 +314,8 @@ struct ImageCommand: ParsableCommand {
targetWindow = windows[0] // frontmost window
}
let fileName = generateFileName(appName: targetApp.localizedName, windowTitle: targetWindow.title)
let filePath = getOutputPath(fileName)
let fileName = FileNameGenerator.generateFileName(appName: targetApp.localizedName, windowTitle: targetWindow.title, format: format)
let filePath = OutputPathResolver.getOutputPath(basePath: path, fileName: fileName)
try captureWindow(targetWindow, to: filePath)
@ -338,9 +338,9 @@ struct ImageCommand: ParsableCommand {
} catch let ApplicationError.notFound(identifier) {
throw CaptureError.appNotFound(identifier)
} catch let ApplicationError.ambiguous(identifier, matches) {
let appNames = matches.map { $0.localizedName ?? $0.bundleIdentifier ?? "Unknown" }
throw CaptureError
.unknownError("Multiple applications match '\(identifier)': \(appNames.joined(separator: ", "))")
// For ambiguous matches, capture all windows from all matching applications
Logger.shared.debug("Multiple applications match '\(identifier)', capturing all windows from all matches")
return try captureWindowsFromMultipleApps(matches, appIdentifier: identifier)
}
if captureFocus == .foreground || (captureFocus == .auto && !targetApp.isActive) {
@ -357,10 +357,10 @@ struct ImageCommand: ParsableCommand {
var savedFiles: [SavedFile] = []
for (index, window) in windows.enumerated() {
let fileName = generateFileName(
appName: targetApp.localizedName, windowIndex: index, windowTitle: window.title
let fileName = FileNameGenerator.generateFileName(
appName: targetApp.localizedName, windowIndex: index, windowTitle: window.title, format: format
)
let filePath = getOutputPath(fileName)
let filePath = OutputPathResolver.getOutputPath(basePath: path, fileName: fileName)
try captureWindow(window, to: filePath)
@ -378,6 +378,60 @@ struct ImageCommand: ParsableCommand {
return savedFiles
}
private func captureWindowsFromMultipleApps(
_ apps: [NSRunningApplication], appIdentifier: String
) throws -> [SavedFile] {
var allSavedFiles: [SavedFile] = []
var totalWindowIndex = 0
for targetApp in apps {
// Log which app we're processing
Logger.shared.debug("Capturing windows for app: \(targetApp.localizedName ?? "Unknown")")
// Handle focus behavior for each app (if needed)
if captureFocus == .foreground || (captureFocus == .auto && !targetApp.isActive) {
try PermissionsChecker.requireAccessibilityPermission()
targetApp.activate()
Thread.sleep(forTimeInterval: 0.2)
}
let windows = try WindowManager.getWindowsForApp(pid: targetApp.processIdentifier)
if windows.isEmpty {
Logger.shared.debug("No windows found for app: \(targetApp.localizedName ?? "Unknown")")
continue
}
for window in windows {
let fileName = FileNameGenerator.generateFileName(
appName: targetApp.localizedName,
windowIndex: totalWindowIndex,
windowTitle: window.title,
format: format
)
let filePath = OutputPathResolver.getOutputPath(basePath: path, fileName: fileName)
try captureWindow(window, to: filePath)
let savedFile = SavedFile(
path: filePath,
item_label: targetApp.localizedName,
window_title: window.title,
window_id: window.windowId,
window_index: totalWindowIndex,
mime_type: format == .png ? "image/png" : "image/jpeg"
)
allSavedFiles.append(savedFile)
totalWindowIndex += 1
}
}
guard !allSavedFiles.isEmpty else {
throw CaptureError.noWindowsFound("No windows found for any matching applications of '\(appIdentifier)'")
}
return allSavedFiles
}
private func captureDisplay(_ displayID: CGDirectDisplayID, to path: String) throws(CaptureError) {
do {
let semaphore = DispatchSemaphore(value: 0)
@ -385,7 +439,7 @@ struct ImageCommand: ParsableCommand {
Task {
do {
try await captureDisplayWithScreenCaptureKit(displayID, to: path)
try await ScreenCapture.captureDisplay(displayID, to: path, format: format)
} catch {
captureError = error
}
@ -402,49 +456,13 @@ struct ImageCommand: ParsableCommand {
throw error
} catch {
// Check if this is a permission error from ScreenCaptureKit
if isScreenRecordingPermissionError(error) {
if PermissionErrorDetector.isScreenRecordingPermissionError(error) {
throw CaptureError.screenRecordingPermissionDenied
}
throw CaptureError.captureCreationFailed(error)
}
}
private func captureDisplayWithScreenCaptureKit(_ displayID: CGDirectDisplayID, to path: String) async throws {
do {
// Get available content
let availableContent = try await SCShareableContent.current
// Find the display by ID
guard let scDisplay = availableContent.displays.first(where: { $0.displayID == displayID }) else {
throw CaptureError.captureCreationFailed(nil)
}
// Create content filter for the entire display
let filter = SCContentFilter(display: scDisplay, excludingWindows: [])
// Configure capture settings
let configuration = SCStreamConfiguration()
configuration.width = scDisplay.width
configuration.height = scDisplay.height
configuration.backgroundColor = .black
configuration.shouldBeOpaque = true
configuration.showsCursor = true
// Capture the image
let image = try await SCScreenshotManager.captureImage(
contentFilter: filter,
configuration: configuration
)
try saveImage(image, to: path)
} catch {
// Check if this is a permission error
if isScreenRecordingPermissionError(error) {
throw CaptureError.screenRecordingPermissionDenied
}
throw error
}
}
private func captureWindow(_ window: WindowData, to path: String) throws(CaptureError) {
do {
@ -453,7 +471,7 @@ struct ImageCommand: ParsableCommand {
Task {
do {
try await captureWindowWithScreenCaptureKit(window, to: path)
try await ScreenCapture.captureWindow(window, to: path, format: format)
} catch {
captureError = error
}
@ -470,270 +488,17 @@ struct ImageCommand: ParsableCommand {
throw error
} catch {
// Check if this is a permission error from ScreenCaptureKit
if isScreenRecordingPermissionError(error) {
if PermissionErrorDetector.isScreenRecordingPermissionError(error) {
throw CaptureError.screenRecordingPermissionDenied
}
throw CaptureError.windowCaptureFailed(error)
}
}
private func captureWindowWithScreenCaptureKit(_ window: WindowData, to path: String) async throws {
do {
// Get available content
let availableContent = try await SCShareableContent.current
// Find the window by ID
guard let scWindow = availableContent.windows.first(where: { $0.windowID == window.windowId }) else {
throw CaptureError.windowNotFound
}
// Create content filter for the specific window
let filter = SCContentFilter(desktopIndependentWindow: scWindow)
// Configure capture settings
let configuration = SCStreamConfiguration()
configuration.width = Int(window.bounds.width)
configuration.height = Int(window.bounds.height)
configuration.backgroundColor = .clear
configuration.shouldBeOpaque = true
configuration.showsCursor = false
// Capture the image
let image = try await SCScreenshotManager.captureImage(
contentFilter: filter,
configuration: configuration
)
try saveImage(image, to: path)
} catch {
// Check if this is a permission error
if isScreenRecordingPermissionError(error) {
throw CaptureError.screenRecordingPermissionDenied
}
throw error
}
}
private func isScreenRecordingPermissionError(_ error: Error) -> Bool {
let errorString = error.localizedDescription.lowercased()
// Check for specific screen recording related errors
if errorString.contains("screen recording") {
return true
}
// Check for NSError codes specific to screen capture permissions
if let nsError = error as NSError? {
// ScreenCaptureKit specific error codes
if nsError.domain == "com.apple.screencapturekit" && nsError.code == -3801 {
// SCStreamErrorUserDeclined = -3801
return true
}
// CoreGraphics error codes for screen capture
if nsError.domain == "com.apple.coregraphics" && nsError.code == 1002 {
// kCGErrorCannotComplete when permissions are denied
return true
}
}
// Only consider it a permission error if it mentions both "permission" and capture-related terms
if errorString.contains("permission") &&
(errorString.contains("capture") || errorString.contains("recording") || errorString.contains("screen")) {
return true
}
return false
}
private func saveImage(_ image: CGImage, to path: String) throws(CaptureError) {
let url = URL(fileURLWithPath: path)
// Check if the parent directory exists
let directory = url.deletingLastPathComponent()
var isDirectory: ObjCBool = false
if !FileManager.default.fileExists(atPath: directory.path, isDirectory: &isDirectory) {
let error = NSError(
domain: NSCocoaErrorDomain,
code: NSFileNoSuchFileError,
userInfo: [NSLocalizedDescriptionKey: "No such file or directory"]
)
throw CaptureError.fileWriteError(path, error)
}
let utType: UTType = format == .png ? .png : .jpeg
guard let destination = CGImageDestinationCreateWithURL(
url as CFURL,
utType.identifier as CFString,
1,
nil
) else {
// Try to create a more specific error for common cases
if !FileManager.default.isWritableFile(atPath: directory.path) {
let error = NSError(
domain: NSPOSIXErrorDomain,
code: Int(EACCES),
userInfo: [NSLocalizedDescriptionKey: "Permission denied"]
)
throw CaptureError.fileWriteError(path, error)
}
throw CaptureError.fileWriteError(path, nil)
}
CGImageDestinationAddImage(destination, image, nil)
guard CGImageDestinationFinalize(destination) else {
throw CaptureError.fileWriteError(path, nil)
}
}
private func generateFileName(
displayIndex: Int? = nil,
appName: String? = nil,
windowIndex: Int? = nil,
windowTitle: String? = nil
) -> String {
let timestamp = DateFormatter.timestamp.string(from: Date())
let ext = format.rawValue
if let displayIndex {
return "screen_\(displayIndex + 1)_\(timestamp).\(ext)"
} else if let appName {
let cleanAppName = appName.replacingOccurrences(of: " ", with: "_")
if let windowIndex {
return "\(cleanAppName)_window_\(windowIndex)_\(timestamp).\(ext)"
} else if let windowTitle {
let cleanTitle = windowTitle.replacingOccurrences(of: " ", with: "_").prefix(20)
return "\(cleanAppName)_\(cleanTitle)_\(timestamp).\(ext)"
} else {
return "\(cleanAppName)_\(timestamp).\(ext)"
}
} else {
return "capture_\(timestamp).\(ext)"
}
}
func getOutputPath(_ fileName: String) -> String {
if let basePath = path {
determineOutputPath(basePath: basePath, fileName: fileName)
} else {
"/tmp/\(fileName)"
}
}
func getOutputPathWithFallback(_ fileName: String) -> String {
if let basePath = path {
determineOutputPathWithFallback(basePath: basePath, fileName: fileName)
} else {
"/tmp/\(fileName)"
}
}
func determineOutputPath(basePath: String, fileName: String) -> String {
// Check if basePath looks like a file (has extension and doesn't end with /)
// Exclude special directory cases like "." and ".."
let isLikelyFile = basePath.contains(".") && !basePath.hasSuffix("/") &&
basePath != "." && basePath != ".."
if isLikelyFile {
// Create parent directory if needed
let parentDir = (basePath as NSString).deletingLastPathComponent
if !parentDir.isEmpty && parentDir != "/" {
do {
try FileManager.default.createDirectory(
atPath: parentDir,
withIntermediateDirectories: true,
attributes: nil
)
} catch {
// Log but don't fail - maybe directory already exists
// Logger.debug("Could not create parent directory \(parentDir): \(error)")
}
}
// For multiple screens, append screen index to avoid overwriting
if screenIndex == nil {
// Multiple screens - modify filename to include screen info
let pathExtension = (basePath as NSString).pathExtension
let pathWithoutExtension = (basePath as NSString).deletingPathExtension
// Extract screen info from fileName (e.g., "screen_1_20250608_120000.png" -> "1_20250608_120000")
let fileNameWithoutExt = (fileName as NSString).deletingPathExtension
let screenSuffix = fileNameWithoutExt.replacingOccurrences(of: "screen_", with: "")
return "\(pathWithoutExtension)_\(screenSuffix).\(pathExtension)"
}
return basePath
} else {
// Treat as directory - ensure it exists
do {
try FileManager.default.createDirectory(
atPath: basePath,
withIntermediateDirectories: true,
attributes: nil
)
} catch {
// Log but don't fail - maybe directory already exists
// Logger.debug("Could not create directory \(basePath): \(error)")
}
return "\(basePath)/\(fileName)"
}
}
func determineOutputPathWithFallback(basePath: String, fileName: String) -> String {
// Check if basePath looks like a file (has extension and doesn't end with /)
// Exclude special directory cases like "." and ".."
let isLikelyFile = basePath.contains(".") && !basePath.hasSuffix("/") &&
basePath != "." && basePath != ".."
if isLikelyFile {
// Create parent directory if needed
let parentDir = (basePath as NSString).deletingLastPathComponent
if !parentDir.isEmpty && parentDir != "/" {
do {
try FileManager.default.createDirectory(
atPath: parentDir,
withIntermediateDirectories: true,
attributes: nil
)
} catch {
// Log but don't fail - maybe directory already exists
// Logger.debug("Could not create parent directory \(parentDir): \(error)")
}
}
// For fallback mode (invalid screen index that fell back to all screens),
// always treat as multiple screens to avoid overwriting
let pathExtension = (basePath as NSString).pathExtension
let pathWithoutExtension = (basePath as NSString).deletingPathExtension
// Extract screen info from fileName (e.g., "screen_1_20250608_120000.png" -> "1_20250608_120000")
let fileNameWithoutExt = (fileName as NSString).deletingPathExtension
let screenSuffix = fileNameWithoutExt.replacingOccurrences(of: "screen_", with: "")
return "\(pathWithoutExtension)_\(screenSuffix).\(pathExtension)"
} else {
// Treat as directory - ensure it exists
do {
try FileManager.default.createDirectory(
atPath: basePath,
withIntermediateDirectories: true,
attributes: nil
)
} catch {
// Log but don't fail - maybe directory already exists
// Logger.debug("Could not create directory \(basePath): \(error)")
}
return "\(basePath)/\(fileName)"
}
}
}
extension DateFormatter {
static let timestamp: DateFormatter = {
let formatter = DateFormatter()
formatter.dateFormat = "yyyyMMdd_HHmmss"
return formatter
}()
}

315
tests/unit/tools/multiple-app-matches.test.ts Normal file
View file

@ -0,0 +1,315 @@
import { describe, it, expect, beforeEach, vi } from "vitest";
import { imageToolHandler } from "../../../src/tools/image";
import { executeSwiftCli } from "../../../src/utils/peekaboo-cli";
import { resolveImagePath } from "../../../src/utils/image-cli-args";
import { mockSwiftCli } from "../../mocks/peekaboo-cli.mock";
import { pino } from "pino";
// Mock the Swift CLI utility
vi.mock("../../../src/utils/peekaboo-cli");
// Mock image-cli-args module
vi.mock("../../../src/utils/image-cli-args", async () => {
const actual = await vi.importActual("../../../src/utils/image-cli-args");
return {
...actual,
resolveImagePath: vi.fn(),
};
});
const mockExecuteSwiftCli = executeSwiftCli as vi.MockedFunction<typeof executeSwiftCli>;
const mockResolveImagePath = resolveImagePath as vi.MockedFunction<typeof resolveImagePath>;
const mockLogger = pino({ level: "silent" });
const mockContext = { logger: mockLogger };
const MOCK_TEMP_DIR = "/tmp/peekaboo-img-XXXXXX";
describe("Multiple App Matches", () => {
beforeEach(() => {
vi.clearAllMocks();
mockResolveImagePath.mockResolvedValue({
effectivePath: MOCK_TEMP_DIR,
tempDirUsed: MOCK_TEMP_DIR,
});
});
it("should capture all windows when multiple exact app matches exist", async () => {
// Simulate capturing "claude" when both "claude" and "Claude" apps exist
const mockMultipleAppResponse = {
success: true,
data: {
saved_files: [
{
path: "/tmp/claude_window_0_20250608_120000.png",
item_label: "claude",
window_title: "Chat - Claude",
window_id: 1001,
window_index: 0,
mime_type: "image/png"
},
{
path: "/tmp/claude_window_1_20250608_120001.png",
item_label: "claude",
window_title: "Settings - Claude",
window_id: 1002,
window_index: 1,
mime_type: "image/png"
},
{
path: "/tmp/Claude_window_2_20250608_120002.png",
item_label: "Claude",
window_title: "Main Window - Claude",
window_id: 2001,
window_index: 2,
mime_type: "image/png"
},
{
path: "/tmp/Claude_window_3_20250608_120003.png",
item_label: "Claude",
window_title: "Preferences - Claude",
window_id: 2002,
window_index: 3,
mime_type: "image/png"
}
]
},
messages: [],
debug_logs: ["Multiple applications match 'claude', capturing all windows from all matches"]
};
mockExecuteSwiftCli.mockResolvedValue(mockMultipleAppResponse);
const input = {
app_target: "claude",
format: "png" as const
};
const result = await imageToolHandler(input, mockContext);
// Should succeed and return all windows from both apps
expect(result.isError).toBeUndefined();
expect(result.saved_files).toHaveLength(4);
// Verify we got windows from both "claude" and "Claude" apps
const claudeLowerItems = result.saved_files?.filter(f => f.item_label === "claude") || [];
const claudeUpperItems = result.saved_files?.filter(f => f.item_label === "Claude") || [];
expect(claudeLowerItems).toHaveLength(2);
expect(claudeUpperItems).toHaveLength(2);
// Verify all windows have sequential indices
const windowIndices = result.saved_files?.map(f => f.window_index).sort() || [];
expect(windowIndices).toEqual([0, 1, 2, 3]);
// Should have called Swift CLI with the app name
expect(mockExecuteSwiftCli).toHaveBeenCalledWith(
expect.arrayContaining(["--app", "claude"]),
mockLogger,
expect.any(Object)
);
});
it("should handle single window mode with multiple app matches", async () => {
// When using window mode (not multi mode), should still capture from all matching apps
const mockSingleWindowResponse = {
success: true,
data: {
saved_files: [
{
path: "/tmp/claude_Chat_20250608_120000.png",
item_label: "claude",
window_title: "Chat - Claude",
window_id: 1001,
window_index: 0,
mime_type: "image/png"
},
{
path: "/tmp/Claude_Main_20250608_120001.png",
item_label: "Claude",
window_title: "Main Window - Claude",
window_id: 2001,
window_index: 1,
mime_type: "image/png"
}
]
},
messages: [],
debug_logs: ["Multiple applications match 'claude', capturing all windows from all matches"]
};
mockExecuteSwiftCli.mockResolvedValue(mockSingleWindowResponse);
const input = {
app_target: "claude:WINDOW_INDEX:0", // Requesting specific window but multiple apps match
format: "png" as const
};
const result = await imageToolHandler(input, mockContext);
expect(result.isError).toBeUndefined();
expect(result.saved_files).toHaveLength(2);
// Should have called Swift CLI with specific window parameters
expect(mockExecuteSwiftCli).toHaveBeenCalledWith(
expect.arrayContaining(["--app", "claude", "--window-index", "0"]),
mockLogger,
expect.any(Object)
);
});
it("should handle case where some matching apps have no windows", async () => {
const mockPartialWindowResponse = {
success: true,
data: {
saved_files: [
{
path: "/tmp/Claude_window_0_20250608_120000.png",
item_label: "Claude",
window_title: "Main Window - Claude",
window_id: 2001,
window_index: 0,
mime_type: "image/png"
}
]
},
messages: [],
debug_logs: [
"Multiple applications match 'claude', capturing all windows from all matches",
"No windows found for app: claude"
]
};
mockExecuteSwiftCli.mockResolvedValue(mockPartialWindowResponse);
const input = {
app_target: "claude",
format: "png" as const
};
const result = await imageToolHandler(input, mockContext);
expect(result.isError).toBeUndefined();
expect(result.saved_files).toHaveLength(1);
expect(result.saved_files?.[0].item_label).toBe("Claude");
});
it("should handle case where no matching apps have windows", async () => {
const mockNoWindowsResponse = {
success: false,
error: {
message: "No windows found for any matching applications of 'claude'",
code: "WINDOW_NOT_FOUND",
details: "No windows found for any matching applications"
}
};
mockExecuteSwiftCli.mockResolvedValue(mockNoWindowsResponse);
const input = {
app_target: "claude",
format: "png" as const
};
const result = await imageToolHandler(input, mockContext);
expect(result.isError).toBe(true);
expect(result.content[0].text).toContain("No windows found for any matching applications of 'claude'");
});
it("should maintain proper file naming for multiple apps", async () => {
const mockNamingResponse = {
success: true,
data: {
saved_files: [
{
path: "/tmp/VSCode_window_0_20250608_120000.png",
item_label: "Visual Studio Code",
window_title: "main.ts - peekaboo",
window_id: 3001,
window_index: 0,
mime_type: "image/png"
},
{
path: "/tmp/vscode_window_1_20250608_120001.png",
item_label: "vscode",
window_title: "Extension Host",
window_id: 4001,
window_index: 1,
mime_type: "image/png"
}
]
}
};
mockExecuteSwiftCli.mockResolvedValue(mockNamingResponse);
const input = {
app_target: "vscode", // Matches both "Visual Studio Code" and "vscode"
format: "png" as const
};
const result = await imageToolHandler(input, mockContext);
expect(result.isError).toBeUndefined();
expect(result.saved_files).toHaveLength(2);
// Verify proper naming conventions are maintained
const file1 = result.saved_files?.[0];
const file2 = result.saved_files?.[1];
expect(file1?.path).toContain("VSCode_window_0");
expect(file2?.path).toContain("vscode_window_1");
// Verify sequential indexing across apps
expect(file1?.window_index).toBe(0);
expect(file2?.window_index).toBe(1);
});
it("should preserve individual app identification in saved files", async () => {
const mockAppIdResponse = {
success: true,
data: {
saved_files: [
{
path: "/tmp/finder_window_0_20250608_120000.png",
item_label: "Finder",
window_title: "Desktop",
window_id: 5001,
window_index: 0,
mime_type: "image/png"
},
{
path: "/tmp/FINDER_window_1_20250608_120001.png",
item_label: "FINDER",
window_title: "Applications",
window_id: 5002,
window_index: 1,
mime_type: "image/png"
}
]
}
};
mockExecuteSwiftCli.mockResolvedValue(mockAppIdResponse);
const input = {
app_target: "finder",
format: "png" as const
};
const result = await imageToolHandler(input, mockContext);
expect(result.isError).toBeUndefined();
expect(result.saved_files).toHaveLength(2);
// Each saved file should preserve its source app's actual name
expect(result.saved_files?.[0].item_label).toBe("Finder");
expect(result.saved_files?.[1].item_label).toBe("FINDER");
// But window indices should be sequential across all matches
expect(result.saved_files?.[0].window_index).toBe(0);
expect(result.saved_files?.[1].window_index).toBe(1);
});
});