Capture screenshots of the desktop for visual verification, debugging, and documentation.
Overview
Screenshot capture enables:
- Full screen screenshots
- Region-specific screenshots
- Visual verification of GUI state
- Documentation and debugging
Desktop automation requires a template with desktop support. Ensure your sandbox has desktop capabilities enabled.
Full Screen Screenshot
Capture the entire screen:
from hopx_ai import Sandbox
sandbox = Sandbox.create(template="desktop")
# Capture full screen
img_bytes = sandbox.desktop.screenshot()
# Save to file
with open('screenshot.png', 'wb') as f:
f.write(img_bytes)
import { Sandbox } from '@hopx-ai/sdk';
import fs from 'fs';
const sandbox = await Sandbox.create({ template: 'desktop' });
// Capture full screen
const screenshot = await sandbox.desktop.screenshot();
// Save to file (screenshot may be Buffer or object with data property)
if (screenshot instanceof Buffer) {
fs.writeFileSync('screenshot.png', screenshot);
} else {
fs.writeFileSync('screenshot.png', Buffer.from(screenshot.data, 'base64'));
}
Region Screenshot
Capture a specific region of the screen:
# Capture region: x=100, y=100, width=500, height=300
img_bytes = sandbox.desktop.screenshot_region(100, 100, 500, 300)
# Save to file
with open('region.png', 'wb') as f:
f.write(img_bytes)
// Capture region: x=100, y=100, width=500, height=300
const region = await sandbox.desktop.screenshotRegion(100, 100, 500, 300);
// Save to file
fs.writeFileSync('region.png', region);
Window Screenshot
Capture a specific window (X11 advanced feature):
# Capture active window
img_bytes = sandbox.desktop.capture_window()
# Capture specific window
windows = sandbox.desktop.get_windows()
if windows:
window_id = windows[0].id
img_bytes = sandbox.desktop.capture_window(window_id)
# Save to file
with open('window.png', 'wb') as f:
f.write(img_bytes)
// Capture active window
const windowImg = await sandbox.desktop.captureWindow();
// Capture specific window
const windows = await sandbox.desktop.listWindows();
if (windows.length > 0) {
const windowId = windows[0].id;
const windowImg = await sandbox.desktop.captureWindow(windowId);
}
// Save to file
fs.writeFileSync('window.png', windowImg);
Screenshot Workflow
Complete screenshot workflow:
from hopx_ai import Sandbox
import time
sandbox = Sandbox.create(template="desktop")
try:
# Start application (example)
# ... application setup ...
# Wait for application to load
time.sleep(2)
# Capture initial state
initial = sandbox.desktop.screenshot()
with open('initial.png', 'wb') as f:
f.write(initial)
# Perform actions
sandbox.desktop.click(400, 300)
time.sleep(1)
# Capture after action
after = sandbox.desktop.screenshot()
with open('after.png', 'wb') as f:
f.write(after)
# Capture specific region
region = sandbox.desktop.screenshot_region(100, 100, 400, 300)
with open('region.png', 'wb') as f:
f.write(region)
finally:
sandbox.kill()
import { Sandbox } from '@hopx-ai/sdk';
import fs from 'fs';
const sandbox = await Sandbox.create({ template: 'desktop' });
try {
// Start application (example)
// ... application setup ...
// Wait for application to load
await new Promise(resolve => setTimeout(resolve, 2000));
// Capture initial state
const initial = await sandbox.desktop.screenshot();
fs.writeFileSync('initial.png', Buffer.from(initial.data, 'base64'));
// Perform actions
await sandbox.desktop.mouseClick(400, 300);
await new Promise(resolve => setTimeout(resolve, 1000));
// Capture after action
const after = await sandbox.desktop.screenshot();
fs.writeFileSync('after.png', Buffer.from(after.data, 'base64'));
// Capture specific region
const region = await sandbox.desktop.screenshotRegion(100, 100, 400, 300);
fs.writeFileSync('region.png', region);
} finally {
await sandbox.kill();
}
- Format: PNG
- Color depth: 24-bit RGB
- Compression: PNG compression
Finding Coordinates
Use screenshots to find coordinates for mouse operations:
# Take screenshot
img_bytes = sandbox.desktop.screenshot()
# Save and inspect in image viewer
with open('reference.png', 'wb') as f:
f.write(img_bytes)
# Use coordinates from image viewer
sandbox.desktop.click(400, 300) # Coordinates from reference.png
// Take screenshot
const screenshot = await sandbox.desktop.screenshot();
// Save and inspect in image viewer
fs.writeFileSync('reference.png', Buffer.from(screenshot.data, 'base64'));
// Use coordinates from image viewer
await sandbox.desktop.mouseClick(400, 300); // Coordinates from reference.png
Next Steps
