Rich Output

Rich output capture automatically detects and captures visual outputs like matplotlib plots, pandas DataFrames, and plotly charts from your code execution.

Prerequisites

Before you begin, make sure you have:

Active sandbox - A running sandbox (see Creating Sandboxes)
Understanding of code execution - Familiarity with Synchronous Execution is helpful
Code that generates visualizations - Code that creates plots, DataFrames, or charts

Overview

When you execute code with run_code(), HopX automatically captures:

Matplotlib plots → PNG images (base64-encoded)
Pandas DataFrames → HTML tables
Plotly charts → HTML interactive visualizations
JSON outputs → Structured data

Rich output capture is enabled by default in run_code(). All visual outputs are automatically detected and included in the ExecutionResult.rich_outputs array.

Automatic Capture

Rich outputs are captured automatically:

Python
JavaScript

from hopx_ai import Sandbox

sandbox = Sandbox.create(template="code-interpreter")

# Execute code that generates a plot
result = sandbox.run_code('''
import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(0, 10, 100)
y = np.sin(x)

plt.plot(x, y)
plt.title("Sine Wave")
plt.savefig('/workspace/plot.png')
''')

# Check for rich outputs
print(f"Rich outputs captured: {len(result.rich_outputs)}")
for output in result.rich_outputs:
    print(f"Type: {output.type}")
    print(f"Data keys: {list(output.data.keys())}")

sandbox.kill()

import { Sandbox } from '@hopx-ai/sdk';

const sandbox = await Sandbox.create({ template: 'code-interpreter' });

// Execute code that generates a plot
const result = await sandbox.runCode(`
import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(0, 10, 100)
y = np.sin(x)

plt.plot(x, y)
plt.title("Sine Wave")
plt.savefig('/workspace/plot.png')
`);

// Check for rich outputs
console.log(`Rich outputs captured: ${result.richOutputs?.length || 0}`);
for (const output of result.richOutputs || []) {
  console.log(`Type: ${output.type}`);
  console.log(`Data keys: ${Object.keys(output.data)}`);
}

await sandbox.kill();

Matplotlib Plots

Capture matplotlib plots as PNG images:

Python
JavaScript

from hopx_ai import Sandbox
import base64

sandbox = Sandbox.create(template="code-interpreter")

result = sandbox.run_code('''
import matplotlib.pyplot as plt
import numpy as np

# Create a simple plot
x = np.linspace(0, 2*np.pi, 100)
y = np.sin(x)

plt.figure(figsize=(10, 6))
plt.plot(x, y, 'b-', linewidth=2, label='sin(x)')
plt.plot(x, np.cos(x), 'r--', linewidth=2, label='cos(x)')
plt.xlabel('x')
plt.ylabel('y')
plt.title('Trigonometric Functions')
plt.legend()
plt.grid(True)
plt.savefig('/workspace/trig_plot.png')
''')

# Access captured plot
for output in result.rich_outputs:
    if 'image/png' in output.data:
        png_data = output.data['image/png']
        print(f"Plot captured: {len(png_data)} bytes")
        # png_data is base64-encoded PNG image

sandbox.kill()

import { Sandbox } from '@hopx-ai/sdk';

const sandbox = await Sandbox.create({ template: 'code-interpreter' });

const result = await sandbox.runCode(`
import matplotlib.pyplot as plt
import numpy as np

# Create a simple plot
x = np.linspace(0, 2*np.pi, 100)
y = np.sin(x)

plt.figure(figsize=(10, 6))
plt.plot(x, y, 'b-', linewidth=2, label='sin(x)')
plt.plot(x, np.cos(x), 'r--', linewidth=2, label='cos(x)')
plt.xlabel('x')
plt.ylabel('y')
plt.title('Trigonometric Functions')
plt.legend()
plt.grid(True)
plt.savefig('/workspace/trig_plot.png')
`);

// Access captured plot
for (const output of result.richOutputs || []) {
  if (output.data['image/png']) {
    const pngData = output.data['image/png'];
    console.log(`Plot captured: ${pngData.length} bytes`);
    // pngData is base64-encoded PNG image
  }
}

await sandbox.kill();

Pandas DataFrames

Capture pandas DataFrames as HTML tables:

Python
JavaScript

from hopx_ai import Sandbox

sandbox = Sandbox.create(template="code-interpreter")

result = sandbox.run_code('''
import pandas as pd
import numpy as np

# Create a DataFrame
data = {
    'Name': ['Alice', 'Bob', 'Charlie', 'Diana'],
    'Age': [25, 30, 35, 28],
    'Score': [85, 92, 78, 95]
}

df = pd.DataFrame(data)
print(df)  # DataFrame is automatically captured as HTML
''')

# Access captured DataFrame
for output in result.rich_outputs:
    if 'text/html' in output.data:
        html_data = output.data['text/html']
        print(f"DataFrame captured as HTML: {len(html_data)} chars")
        # html_data contains HTML table representation

sandbox.kill()

import { Sandbox } from '@hopx-ai/sdk';

const sandbox = await Sandbox.create({ template: 'code-interpreter' });

const result = await sandbox.runCode(`
import pandas as pd
import numpy as np

# Create a DataFrame
data = {
    'Name': ['Alice', 'Bob', 'Charlie', 'Diana'],
    'Age': [25, 30, 35, 28],
    'Score': [85, 92, 78, 95]
}

df = pd.DataFrame(data)
print(df)  # DataFrame is automatically captured as HTML
`);

// Access captured DataFrame
for (const output of result.richOutputs || []) {
  if (output.data['text/html']) {
    const htmlData = output.data['text/html'];
    console.log(`DataFrame captured as HTML: ${htmlData.length} chars`);
    // htmlData contains HTML table representation
  }
}

await sandbox.kill();

Plotly Charts

Capture interactive Plotly charts as HTML:

Python
JavaScript

from hopx_ai import Sandbox

sandbox = Sandbox.create(template="code-interpreter")

result = sandbox.run_code('''
import plotly.graph_objects as go
import numpy as np

# Create interactive plot
x = np.linspace(0, 10, 100)
y = np.sin(x)

fig = go.Figure()
fig.add_trace(go.Scatter(x=x, y=y, mode='lines', name='sin(x)'))
fig.update_layout(title='Interactive Plot', xaxis_title='x', yaxis_title='y')
fig.write_html('/workspace/plot.html')
''')

# Access captured Plotly chart
for output in result.rich_outputs:
    if 'text/html' in output.data:
        html_data = output.data['text/html']
        print(f"Plotly chart captured: {len(html_data)} chars")
        # html_data contains interactive HTML chart

sandbox.kill()

import { Sandbox } from '@hopx-ai/sdk';

const sandbox = await Sandbox.create({ template: 'code-interpreter' });

const result = await sandbox.runCode(`
import plotly.graph_objects as go
import numpy as np

# Create interactive plot
x = np.linspace(0, 10, 100)
y = np.sin(x)

fig = go.Figure()
fig.add_trace(go.Scatter(x=x, y=y, mode='lines', name='sin(x)'))
fig.update_layout(title='Interactive Plot', xaxis_title='x', yaxis_title='y')
fig.write_html('/workspace/plot.html')
`);

// Access captured Plotly chart
for (const output of result.richOutputs || []) {
  if (output.data['text/html']) {
    const htmlData = output.data['text/html'];
    console.log(`Plotly chart captured: ${htmlData.length} chars`);
    // htmlData contains interactive HTML chart
  }
}

await sandbox.kill();

Multiple Rich Outputs

Capture multiple outputs in a single execution:

Python
JavaScript

from hopx_ai import Sandbox

sandbox = Sandbox.create(template="code-interpreter")

result = sandbox.run_code('''
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# Create a plot
plt.plot([1, 2, 3, 4], [1, 4, 9, 16])
plt.savefig('/workspace/plot1.png')

# Create a DataFrame
df = pd.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6]})
print(df)

# Create another plot
plt.figure()
plt.scatter([1, 2, 3], [4, 5, 6])
plt.savefig('/workspace/plot2.png')
''')

# Access all captured outputs
print(f"Total rich outputs: {len(result.rich_outputs)}")
for i, output in enumerate(result.rich_outputs):
    print(f"Output {i+1}: {output.type}")
    print(f"  Data types: {list(output.data.keys())}")

sandbox.kill()

import { Sandbox } from '@hopx-ai/sdk';

const sandbox = await Sandbox.create({ template: 'code-interpreter' });

const result = await sandbox.runCode(`
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# Create a plot
plt.plot([1, 2, 3, 4], [1, 4, 9, 16])
plt.savefig('/workspace/plot1.png')

# Create a DataFrame
df = pd.DataFrame({'x': [1, 2, 3], 'y': [4, 5, 6]})
print(df)

# Create another plot
plt.figure()
plt.scatter([1, 2, 3], [4, 5, 6])
plt.savefig('/workspace/plot2.png')
`);

// Access all captured outputs
console.log(`Total rich outputs: ${result.richOutputs?.length || 0}`);
for (let i = 0; i < (result.richOutputs?.length || 0); i++) {
  const output = result.richOutputs[i];
  console.log(`Output ${i + 1}: ${output.type}`);
  console.log(`  Data types: ${Object.keys(output.data)}`);
}

await sandbox.kill();

Rich Output Structure

Each rich output has the following structure:

Python
JavaScript

from hopx_ai import Sandbox

sandbox = Sandbox.create(template="code-interpreter")

result = sandbox.run_code('''
import matplotlib.pyplot as plt
plt.plot([1, 2, 3])
plt.savefig('/workspace/plot.png')
''')

# Inspect rich output structure
for output in result.rich_outputs:
    print(f"Type: {output.type}")  # e.g., "image/png", "text/html"
    print(f"Data: {type(output.data)}")  # Dict[str, str]
    
    # Access specific data format
    if 'image/png' in output.data:
        png_base64 = output.data['image/png']
        # Decode if needed: import base64; image_bytes = base64.b64decode(png_base64)
    
    if 'text/html' in output.data:
        html_content = output.data['text/html']
        # Use HTML content directly

sandbox.kill()

import { Sandbox } from '@hopx-ai/sdk';

const sandbox = await Sandbox.create({ template: 'code-interpreter' });

const result = await sandbox.runCode(`
import matplotlib.pyplot as plt
plt.plot([1, 2, 3])
plt.savefig('/workspace/plot.png')
`);

// Inspect rich output structure
for (const output of result.richOutputs || []) {
  console.log(`Type: ${output.type}`);  // e.g., "image/png", "text/html"
  console.log(`Data: ${typeof output.data}`);  // object
  
  // Access specific data format
  if (output.data['image/png']) {
    const pngBase64 = output.data['image/png'];
    // Decode if needed: const imageBytes = Buffer.from(pngBase64, 'base64');
  }
  
  if (output.data['text/html']) {
    const htmlContent = output.data['text/html'];
    // Use HTML content directly
  }
}

await sandbox.kill();

Complete Example

Here’s a complete example with all rich output types:

Python
JavaScript

from hopx_ai import Sandbox
import base64

sandbox = Sandbox.create(template="code-interpreter")

result = sandbox.run_code('''
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# 1. Matplotlib plot
x = np.linspace(0, 10, 100)
plt.plot(x, np.sin(x))
plt.title("Sine Wave")
plt.savefig('/workspace/sine.png')

# 2. Pandas DataFrame
df = pd.DataFrame({
    'x': x[::10],
    'sin(x)': np.sin(x[::10])
})
print(df)

# 3. Multiple plots
plt.figure()
plt.subplot(2, 1, 1)
plt.plot(x, np.sin(x))
plt.title("Sin")

plt.subplot(2, 1, 2)
plt.plot(x, np.cos(x))
plt.title("Cos")
plt.savefig('/workspace/multiplot.png')
''')

# Process all rich outputs
print(f"Captured {len(result.rich_outputs)} rich outputs\n")

for i, output in enumerate(result.rich_outputs, 1):
    print(f"Output {i}:")
    print(f"  Type: {output.type}")
    
    if 'image/png' in output.data:
        png_data = output.data['image/png']
        print(f"  PNG size: {len(png_data)} bytes (base64)")
    
    if 'text/html' in output.data:
        html_data = output.data['text/html']
        print(f"  HTML size: {len(html_data)} chars")

sandbox.kill()

import { Sandbox } from '@hopx-ai/sdk';

const sandbox = await Sandbox.create({ template: 'code-interpreter' });

const result = await sandbox.runCode(`
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

# 1. Matplotlib plot
x = np.linspace(0, 10, 100)
plt.plot(x, np.sin(x))
plt.title("Sine Wave")
plt.savefig('/workspace/sine.png')

# 2. Pandas DataFrame
df = pd.DataFrame({
    'x': x[::10],
    'sin(x)': np.sin(x[::10])
})
print(df)

# 3. Multiple plots
plt.figure()
plt.subplot(2, 1, 1)
plt.plot(x, np.sin(x))
plt.title("Sin")

plt.subplot(2, 1, 2)
plt.plot(x, np.cos(x))
plt.title("Cos")
plt.savefig('/workspace/multiplot.png')
`);

// Process all rich outputs
console.log(`Captured ${result.richOutputs?.length || 0} rich outputs\n`);

for (let i = 0; i < (result.richOutputs?.length || 0); i++) {
  const output = result.richOutputs[i];
  console.log(`Output ${i + 1}:`);
  console.log(`  Type: ${output.type}`);
  
  if (output.data['image/png']) {
    const pngData = output.data['image/png'];
    console.log(`  PNG size: ${pngData.length} bytes (base64)`);
  }
  
  if (output.data['text/html']) {
    const htmlData = output.data['text/html'];
    console.log(`  HTML size: ${htmlData.length} chars`);
  }
}

await sandbox.kill();

Best Practices

1. Save Files Explicitly

Always save plots to files (e.g., plt.savefig()) to ensure they’re captured. Rich output capture works best when files are explicitly saved.

2. Use Appropriate Formats

Matplotlib: Save as PNG for best compatibility
Pandas: Print DataFrames to trigger HTML capture
Plotly: Use write_html() for interactive charts

3. Check Output Count

Use len(result.rich_outputs) to verify all expected outputs were captured.

4. Handle Base64 Encoding

PNG images are base64-encoded. Decode them before displaying or saving locally.

5. Multiple Outputs

You can capture multiple plots/DataFrames in a single execution. All will be included in rich_outputs.

Synchronous Execution - Basic code execution
Background Execution - Long-running tasks
SDK: sandbox.run_code() - Python SDK method (rich output enabled by default)
API: POST /execute - VM Agent API endpoint
CLI Code Execution - Execute code from CLI

Next Steps

Learn about Synchronous Execution basics
Explore Background Execution for long-running visualizations
Review Streaming Execution for real-time output

Home

Get Started

Sandboxes

Code Execution

Filesystem

Commands

Environment Variables

Templates

Desktop Automation

Terminal

Observability

Cache

Prerequisites

Overview

Automatic Capture

Matplotlib Plots

Pandas DataFrames

Plotly Charts

Multiple Rich Outputs

Rich Output Structure

Complete Example

Best Practices

Next Steps

​Prerequisites

​Overview

​Automatic Capture

​Matplotlib Plots

​Pandas DataFrames

​Plotly Charts

​Multiple Rich Outputs

​Rich Output Structure

​Complete Example

​Best Practices

​Related

​Next Steps

Prerequisites

Overview

Automatic Capture

Matplotlib Plots

Pandas DataFrames

Plotly Charts

Multiple Rich Outputs

Rich Output Structure

Complete Example

Best Practices

Related

Next Steps