Skip to content

Scenes

Scenes are the building blocks of MUG experiments. Each scene represents a stage in your experiment, from welcome screens to interactive gameplay to final thank-you pages.

Scene Types

MUG provides four types of scenes:

StartScene

The entry point for participants. Every experiment must begin with a StartScene.

from mug.scenes import static_scene

start_scene = (
    static_scene.StartScene()
    .scene(scene_id="welcome")
    .display(
        scene_header="Welcome to the Experiment",
        scene_body="<p>Thank you for participating...</p>"
    )
)

Use for:

  • Welcome messages
  • Initial instructions
  • Consent forms (with custom HTML)

Key features:

  • Always shows a "Continue" button to advance
  • Can include HTML in scene_body
  • Required as first scene in every Stager

GymScene

Interactive environment where participants engage with a Gymnasium-based environment.

from mug.scenes import gym_scene
from mug.configurations import configuration_constants

game_scene = (
    gym_scene.GymScene()
    .scene(scene_id="gameplay")
    .environment(env_creator=make_env, env_config={})
    .rendering(fps=30, game_width=600, game_height=400)
    .gameplay(
        num_episodes=5,
        action_mapping={"ArrowLeft": 0, "ArrowRight": 1},
        default_action=0,
    )
    .policies(policy_mapping={"human": configuration_constants.PolicyTypes.Human})
)

Use for:

  • Interactive gameplay
  • Data collection during environment interaction
  • Human-only, AI-only, or human-AI experiments

Key features:

  • Supports Gymnasium environments
  • Real-time rendering
  • Flexible action mappings
  • Episode management
  • Policy execution (human, AI, or mixed)

EndScene

The final scene shown to participants. Every experiment must end with an EndScene.

end_scene = (
    static_scene.EndScene()
    .scene(scene_id="thanks")
    .display(
        scene_header="Thank You!",
        scene_body="<p>Your participation is complete.</p>"
    )
)

Use for:

  • Thank you messages
  • Redirecting to external surveys (e.g., Prolific, MTurk)
  • Final instructions or debriefing

Key features:

  • No "Continue" button (experiment ends here)
  • Can trigger redirect after timeout
  • Required as last scene in every Stager

StaticScene

Custom HTML pages for non-interactive content.

survey_scene = (
    static_scene.StaticScene()
    .scene(scene_id="demographics")
    .display(
        scene_header="Demographics Survey",
        scene_body="""
            <form id="demographics">
                <label>Age: <input type="number" name="age"></label>
                <label>Gender: <input type="text" name="gender"></label>
            </form>
        """
    )
)

Use for:

  • Surveys and questionnaires
  • Additional instructions between games
  • Attention checks
  • Custom interactive HTML

Key features:

  • Full HTML/CSS/JavaScript support
  • Can disable "Continue" button until form completion
  • Form data is automatically collected via element_ids

Scene Configuration

All scenes share common configuration methods:

.scene()

Identify and configure the scene:

.scene(
    scene_id="unique_identifier",        # Required: unique ID for this scene
    experiment_config={},                 # Optional: scene-specific metadata
    should_export_metadata=True,          # Optional: save scene config to file
)

.display()

Set the visual content:

.display(
    scene_header="Scene Title",           # Displayed at top
    scene_body="<p>HTML content</p>",    # Main content area
    scene_body_filepath="path/to/file.html",  # Or load from file
)

Note: Use either scene_body OR scene_body_filepath, not both.

GymScene-Specific Configuration

GymScene has additional configuration methods for interactive gameplay:

.environment()

Define what environment to run:

.environment(
    env_creator=make_my_env,              # Function that returns a Gym env
    env_config={"difficulty": "hard"},    # Kwargs passed to env_creator
    seed=42,                               # Random seed for reproducibility
)

.rendering()

Control visual display:

.rendering(
    fps=30,                                # Frames per second
    game_width=600,                        # Canvas width in pixels
    game_height=400,                       # Canvas height in pixels
    env_to_state_fn=my_render_fn,        # Custom rendering function
    hud_text_fn=my_hud_fn,                # Function to generate HUD text
    location_representation="relative",    # "relative" (0-1) or "pixels"
    background="#FFFFFF",                  # Background color
)

.gameplay()

Configure game mechanics:

.gameplay(
    num_episodes=5,                        # Number of episodes to play
    max_steps=1000,                        # Max steps per episode
    action_mapping={                       # Map keys to actions
        "ArrowLeft": 0,
        "ArrowRight": 1,
    },
    default_action=0,                      # Action when no key pressed
    action_population_method=              # How to handle missing actions
        configuration_constants.ActionSettings.DefaultAction,
    input_mode=                            # How to collect input
        configuration_constants.InputModes.PressedKeys,
    reset_freeze_s=0,                      # Freeze time after episode ends
)

.policies()

Define who/what controls each agent:

.policies(
    policy_mapping={                       # Map agent IDs to policies
        "player_0": configuration_constants.PolicyTypes.Human,
        "player_1": "my_ai_policy",
    },
    load_policy_fn=load_policy,           # Function to load AI policies
    policy_inference_fn=run_inference,    # Function to run policy inference
    frame_skip=4,                          # Actions applied every N frames
)

.content()

Customize participant-facing text:

.content(
    scene_header="Game Title",
    scene_body="<p>Loading...</p>",       # Shown before game starts
    in_game_scene_body="<p>Instructions during game</p>",
    scene_body_filepath="instructions.html",  # Or load from file
    in_game_scene_body_filepath="hud.html",
)

.runtime()

Configure browser-based execution:

.runtime(
    run_through_pyodide=True,              # Enable browser-side execution
    environment_initialization_code="import gym\nenv = gym.make('CartPole-v1')",
    environment_initialization_code_filepath="path/to/env.py",
    packages_to_install=["gymnasium==1.0.0", "numpy"],
    restart_pyodide=False,                 # Restart Pyodide between scenes
)

Scene Lifecycle

Each scene goes through a lifecycle:

  1. Build: Scene configuration is finalized
  2. Activate: Scene becomes active for a participant
  3. Interact: Participant engages with the scene
  4. Deactivate: Participant advances, scene cleanup occurs

Lifecycle Hooks:

class CustomScene(gym_scene.GymScene):

    def on_connect(self, socketio, room):
        """Called when participant connects to server"""
        pass

    def activate(self, socketio, room):
        """Called when scene becomes active"""
        super().activate(socketio, room)
        # Custom activation logic

    def deactivate(self):
        """Called when participant leaves scene"""
        # Cleanup logic
        super().deactivate()

Scene Metadata

Scenes can export metadata for analysis:

.scene(
    scene_id="my_scene",
    experiment_config={"version": "1.0", "condition": "A"},
    should_export_metadata=True,
)

This saves a JSON file with:

  • Scene ID
  • Scene type
  • All configuration parameters
  • Timestamp
  • Custom experiment_config data

Metadata is saved to data/{scene_id}/{subject_id}_metadata.json.

Custom HTML in Scenes

StartScene, EndScene, and StaticScene support full HTML:

scene = (
    static_scene.StaticScene()
    .scene(scene_id="survey")
    .display(
        scene_header="Quick Survey",
        scene_body="""
            <style>
                .question { margin: 20px 0; }
                label { display: block; margin: 5px 0; }
            </style>

            <div class="question">
                <p>How much did you enjoy the game?</p>
                <label><input type="radio" name="enjoy" value="1"> Not at all</label>
                <label><input type="radio" name="enjoy" value="5"> Very much</label>
            </div>

            <script>
                // Custom JavaScript for validation, etc.
                document.querySelector('form').addEventListener('submit', (e) => {
                    // Validation logic
                });
            </script>
        """
    )
)

Collecting form data:

To capture data from HTML form elements, add their id attributes to the scene's element_ids list. When the participant advances past the scene, MUG automatically reads the value of each listed element from the DOM and saves it to a CSV file at data/{experiment_id}/{scene_id}/{subject_id}.csv.

scene = (
    static_scene.StaticScene()
    .scene(scene_id="survey")
    .display(
        scene_header="Quick Survey",
        scene_body="""
            <p>How much did you enjoy the game?</p>
            <input type="range" id="enjoyment" min="1" max="7" value="4">

            <p>Any comments?</p>
            <textarea id="comments"></textarea>
        """
    )
)
scene.element_ids = ["enjoyment", "comments"]

The built-in survey scene classes (e.g., OptionBoxesWithScalesAndTextBox, ScalesAndTextBox, MultipleChoice) set element_ids automatically based on the form elements they generate. See mug/scenes/static_scene.py for available survey components.

You can also store arbitrary client-side data by writing to the window.mugGlobals JavaScript object from within your scene_body HTML. These globals are synced to the server and saved alongside the form data as {subject_id}_globals.json.

Multi-Scene Experiments

Experiments can have any number of scenes between Start and End:

from mug.scenes import stager

experiment = stager.Stager(scenes=[
    start_scene,                 # Required first
    instructions_scene,          # StaticScene
    practice_game_scene,         # GymScene
    survey_scene_1,              # StaticScene
    main_game_scene,             # GymScene
    survey_scene_2,              # StaticScene
    end_scene,                   # Required last
])

Participants progress through scenes by clicking "Continue" or completing episodes.

Scene IDs and Data Organization

Scene IDs determine data organization:

data/
├── welcome/                    # StartScene data
│   └── subject_123_metadata.json
├── game_scene_1/              # GymScene data
│   ├── subject_123.csv
│   ├── subject_123_globals.json
│   └── subject_123_metadata.json
├── survey/                     # StaticScene data
│   └── subject_123.csv
└── thanks/                     # EndScene data
    └── subject_123_metadata.json

Use descriptive scene IDs to organize your data clearly.

Best Practices

  1. Use descriptive scene_ids: "tutorial_level_1" not "scene1"
  2. Keep instructions clear: Participants won't ask for clarification
  3. Test the flow: Complete the experiment yourself before running with participants
  4. Export metadata: Set should_export_metadata=True for reproducibility
  5. Validate forms: Use JavaScript to validate StaticScene forms before allowing "Continue"
  6. Handle errors: Test what happens when participants refresh, go back, etc.

Common Patterns

Practice + Main Game:

practice_scene = (
    gym_scene.GymScene()
    .scene(scene_id="practice")
    .gameplay(num_episodes=3)  # Fewer episodes
    # ... other config
)

main_scene = (
    gym_scene.GymScene()
    .scene(scene_id="main_game")
    .gameplay(num_episodes=10)  # Full experiment
    # ... other config
)

Conditional Scene Content:

.content(
    game_page_html_fn=lambda game, subject_id:
        f"<p>Your current score: {game.total_rewards[subject_id]}</p>"
)

Multiple Conditions:

Use RandomizeOrder with keep_n=1 to randomly assign each participant to one condition. Each condition is a separate scene, and the wrapper selects one at random when the Stager builds the scene sequence for a participant:

from mug.scenes import scene, stager

stager = stager.Stager(scenes=[
    start_scene,
    scene.RandomizeOrder(
        [condition_a_scene, condition_b_scene, condition_c_scene],
        keep_n=1,  # Each participant sees exactly one
    ),
    survey_scene,
    end_scene,
])